Tag Archives: using


Tokenization: Tokenization is the process of breaking a string of text into smaller units called tokens. These tokens are usually words, numbers, or punctuation marks. The nltk module provides several tokenizers that can be used for tokenizing text. For example, the word_tokenize() function uses simple regex-based rules to tokenize a string into words. The sent_tokenize() function splits a text into a list of sentences.

Part-of-Speech (POS) Tagging: POS tagging involves assigning part-of-speech tags like noun, verb, adjective etc. to each token in a sentence. This helps in syntactic parsing and many other tasks. The nltk.pos_tag() function takes tokenized text as input and returns the same text with each token tagged with its part-of-speech. It uses probabilistic taggers trained on large corpora.

Named Entity Recognition (NER): NER is the task of locating and classifying named entities like persons, organizations, locations etc. mentioned in unstructured text into pre-defined categories. The nltk.ne_chunk() method recognizes named entities using optional regexes and can output grammatical structures. This information helps in applications like information extraction.

Stemming: Stemming is the process of reducing words to their root/stem form. For example, reducing “studying”, “studied” to the root word “stud”. Nltk provides a PorterStemmer class that performs morphological stemmer for English words. It removes common morphological and inflectional endings from words. Stemming helps in reducing data sparsity for applications like text classification.

Lemmatization: Lemmatization goes beyond stemming and brings words to their base/dictionary form. For example, it reduces “studying”, “studied” to the lemma “study”. It takes into account morphological analysis of words and tries to remove inflectional endings. Nltk provides WordNetLemmatizer which performs morphological analysis and returns the lemmatized form of words. Lemmatization helps improve Information Retrieval tasks.

Text Classification: Text classification involves classifying documents or sentences into predefined categories based on their content. Using features extracted from documents and machine learning algorithms like Naive Bayes Classifier, documents can be classified. Nltk provides functions to extract features like word counts,presence/absence of words etc. from texts that can be used for classification.

Sentiment Analysis: Sentiment analysis determines whether the sentiment expressed in a document or a sentence is positive, negative or neutral. This helps in understanding peoples opinions and reactions. Nltk has several pre-trained sentiment classifiers like Naive Bayes Classifier that can be used to determine sentiment polarity at document or sentence level. Features like presence of positive/negative words, emoticons etc are used for classification.

Language Identification: Identifying the language that a text is written in is an important subtask of many NLP applications. Nltk provides language identification functionality using n-gram character models. Functions like detect() can identify languages given a text sample. This helps in routing texts further processing based on language.

Text Summarization: Automatic text summarization involves condensing a text document into a shorter version preserving its meaning and most important ideas. Summary generation works by identifying important concepts and sentences in a document using features like word/sentence frequency, dialogue etc. Techniques like centroid-based summarization can be implemented using Nltk to generate summaries of documents.

Information Extraction: IE is the task of extracting structured information like entities, relationships between entities etc from unstructured text. Using methods like regex matching, entity clustering, open IE techniques and parsers, key information can be extracted from texts. Nltk provides functionalities and wrappers around open source IE tools that can be leveraged for tasks like building knowledge bases from documents.

Named Entity Translation: Translating named entities like person names, locations etc accurately across languages is a challenging task. Nltk provides methods and data to transliterate named entities from one language to another phonetically or by mapping entity with same meaning across languages. This helps in cross-lingual applications like question answering over multi-lingual data.

Topic Modeling: Topic modeling is a statistical modeling technique to discover abstract “topics” that occur in a collection of documents. It involves grouping together words that co-occur frequently to form topics. Using algorithms like Latent Dirichlet Allocation(LDA) implemented methods in Nltk, topics can be automatically discovered from document collections that best explains the co-occurrence of words.

These are some of the common NLP tasks that can be accomplished using the Python modules – string, re and nltk. Nltk provides a comprehensive set of utilities and data for many NLP tasks right from basic text processing like tokenization, stemming, parsing to higher level tasks like sentiment analysis, text classification, topic modeling etc. The regular expression module (re) helps in building custom patterns for tasks like named entity recognition, normalization etc. These Python libraries form a powerful toolkit for rapid development of NLP applications.


One of the biggest potential limitations of self-report measures is biases related to social desirability and impression management. There is a risk that participants may not report private or sensitive information accurately because they want to present themselves in a favorable light or avoid embarrassment. For example, if a study is examining symptoms of depression, participants may under-report how frequently they experience certain feelings or behaviors because admitting to them would make them feel badly about themselves. This type of bias can threaten the validity of conclusions drawn from the data.

Another limitation is recall bias, or errors in a person’s memory of past events, behaviors, or feelings. Many self-report measures ask participants to reflect on periods of time in the past, sometimes going back years. Human memory is fallible and can be inaccurate or incomplete. For events farther back in time, details may be forgotten or reconstructed differently than how they actually occurred. This is a particular problem for retrospective self-reports but can also influence current self-reports if questions require remembering specific instances rather than overall frequencies. Recall bias introduces noise and potential inaccuracy into the data.

Response biases related to self-presentation are not the only potential for socially desirable responding. There is also a risk of participants wanting to satisfy the researcher or meet perceived demands of the study. They may provide answers they think the experimenter wants to hear or will make the study turn out as expected, rather than answers that fully reflect their genuine thoughts, feelings, and experiences. This threatens the validity of inferences about psychologically meaningful constructs if responses are skewed by a desire to please rather than a candid report of subjective experience.

Self-report measures also rely on the assumption that individuals have reliable insight into their own thoughts, behaviors, traits, and other private psychological experiences. There are many reasons why a person’s self-perceptions may not correspond perfectly with reality or with objective behavioral observations. People are not always fully self-aware or capable of accurate self-analysis and self-diagnosis. Their self-views can be biased by numerous cognitive and emotional factors like self-serving biases, selective attention and memory, projection, denial and reaction formation, and more. Relying only on self-report removes the capability for cross-validation against more objective measures or reports from knowledgeable others.

Practical difficulties inherent to the self-report format pose additional limitations. Ensuring participants interpret vague or complex questions as intended can be challenging without opportunity for clarification or explanation by the researcher. Response scales may not provide optimal sensitivity and precision for measuring psychological constructs. Question order effects, question wording choices, and other superficial qualities of the measure itself can unduly influence responses independent of the intended latent variables. And low literacy levels, language barriers, or limited attention and motivation in some participants may compromise reliability and validity if questions are misunderstood.

An issue that affects not just the accuracy but also the generalizability of self-report findings is that the psychological experience of completing questionnaires may itself shape responses in unforeseen ways. The act of self-reflection and item consideration activates certain cognitive and affective processes that do not mirror real-world behavior. And researchers cannot be sure whether measured constructs are elicited temporarily within the artificial context of research participation or indicative of patterns that generalize to daily life outside the lab. Ecological validity is challenging to establish for self-report data.

Practical difficulties also emerge from logistical demands of obtaining and interpreting self-report data. Large sample sizes are usually required to achieve sufficient statistical power given the noisiness of self-report. But recruitment and full participation across numerous multi-item measures poses challenges for both researchers and subjects. Substantial time, resources and effort are required on the part of researchers to develop quality measures, administer them properly, screen responses for quality, handle missing data, and quantitatively reduce information from numerous items into interpretable scores on underlying dimensions.

Some key limitations of self-report methods include issues with biases that threaten validity like social desirability, recall bias, and response bias to please researchers. Additional difficulties emerge from lack of objective behavioral measures for comparison or validation, imperfect self-awareness and insight, susceptibility to superficial qualities and context of the measures themselves, questionable generalizability beyond research contexts, and substantial logistical and resource demands for quality data collection and analysis. Many of these are challenging, though not impossible, to control for or address through research design features and statistical methods. Researchers using self-report must carefully consider these issues and their potential impact on drawing sound scientific conclusions from the results obtained.


Kotter’s 8-step process for leading change is one of the most widely accepted change management frameworks used by organizations and consultants worldwide to help ensure change initiatives are successful. When undertaking a change management consulting project, closely following Kotter’s 8 steps can help smooth the process and increase the chances of achieving the desired outcomes.

The first step in Kotter’s model is to establish a sense of urgency. At the start of the project, the change management consultant would work with senior leadership to assess why change is needed and help create a compelling case for action. Diagnosing the need for change based on market factors, competitive threats, productivity issues or other challenges facing the organization helps convince stakeholders change is imperative. The consultant would work with leaders to communicate this urgency through meetings, presentations and other forums to gain buy-in.

Step two involves creating a guiding coalition. The consultant facilitates formation of a high-powered, cross-functional team consisting of influential leaders, managers and change agents whose help is needed to drive the change. Their positional power and combined expertise helps provide change momentum. Coalition members get fully engaged by understanding the opportunity for their business areas and being involved in strategic planning.

In step three, the consultant helps the coalition develop and communicate a powerful vision for change. An inspiring new vision is crafted that offers a clear picture of what the future could look like after successful transformation. This vision aims to simplify the complex change process and direct the efforts of people in a unified way. Communication tools such as memos, speeches, discussion guides and websites ensure the vision is repeatedly shared across divisions and levels.

Forming strategic initiatives to achieve the vision is step four. Based on assessments, the consultant works with the coalition to identify essential projects and tasks needed to bring the transformation to life. These initiate platforms include new processes, technologies, products, services, capabilities or organizational forms that are tangible representations of achieving the vision. Clear milestones, timelines and deliverables are defined to build momentum.

Step five involves enabling action by removing barriers and empowering others to act on the initiatives. The consultant helps empower broad-based action by assessing perceived obstacles to change, obtaining resources and ensure training, systems and structures are in place. Policies are adjusted, direct reports are enabled with new skills and tools, and new performance management and reward systems recognize successes.

Generating short-term wins is step six. After initial thrusts, the consultant works with leaders to recognize and reward achievements that demonstrate visible progress. Highlighting success stories that resulted from early initiatives helps build confidence and momentum for further change, while motivating continued efforts needed to consolidate gains and propel additional progress.

Consolidating improvements and sustaining acceleration is step seven. As deeper changes take root, formal plans with goals and milestones guide ongoing efforts to ensure initiatives become standard practice. New approaches are continuously developed, leaders are coach to increase progress and hold the line against complacency. The consultant helps assess what’s working well and where more work is needed.

Institutionalizing new approaches is the final step eight. The transformation is complete when behaviors, systems and structures fully support the new state as the ‘new normal’. With the consultant’s guidance, leadership focuses on anchoring cultural changes through succession planning, performance evaluations, job descriptions and retirements to cement the transformation. Feedback from staff is gathered to understand what continues to work and where small adaptations are still warranted to sustain momentum.

The 8 step model guides change management efforts from start to finish over time. As a consultant, working closely with leadership using Kotter’s framework helps overcome barriers, move initiatives ahead and drive increasing buy-in. Continually monitoring each step ensures activities remain aligned and pace of progress is kept up. Completing all phases leads to a higher potential for achieving desired business outcomes. Consultants provide objective facilitation to help leaders make well-informed decisions and skillfully manage people side of change for sustainable success.

In conclusion, Kotter’s 8 step change management model offers a proven approach for consultants to structure engagement, guide planning and ensure activities are implemented to realize goals. By keeping leadership accountable to achieve defined outcomes at each stage, likelihood of overcoming resistance increases. And change becomes embedded rather than a one-time event. Combined with assessment-driven recommendations, facilitation of key stakeholder workshops and status reporting, consultants help organizations transform in a way that lasts.


One of the most important insights the dashboard provided was visibility into how different departments and product categories were performing. By having sales visualized by department, executives could easily see which areas of the store were most successful and driving the majority of revenue. They likely noticed a few star departments that were strong performers and deserved more investment and focus. Meanwhile, underperforming departments that had lower sales numbers became immediately apparent and possibly warranted examining reasons for poor performance to identify opportunities for improvement.

Breaking sales down by product category offered a similar view into top moving and bottom moving categories. Executives could make data-driven decisions about discontinuing slow categories to free up shelf space for better sellers. Or they may have identified untapped potential in niche categories experiencing growth that deserved expansion. Simply knowing metrics like average sales per item and dollar sales by category armed managers with intelligence on where to focus merchandising and promotion efforts.

Another key insight the dashboard provided was visibility into sales trends over time. By viewing month-over-month or quarter-over-quarter sales figures, executives could easily identify seasonal patterns and determine when sales typically peaked and valleys. They likely noticed strong correlation between certain holidays or times of year and higher sales. These trend insights allowed managers to more accurately predict sales and strategically plan inventory levels, staffing needs, promotions and new product launches during anticipated high-traffic periods.

Analyzing sales by region or territory on the dashboard surely revealed to executives how different individual stores or groups of stores were faring. Underperforming stores with noticeably lower sales numbers may have needed troubleshooting to determine causes like undesirable location attributes, lack of experienced management, poor merchandising, etc. Top performing stores with higher sales densities per square foot could serve as benchmarks to learn successful tactics from and replicate elsewhere. Regional managers likely used these localized sales views to make data-driven decisions about new store sites as well.

Sales broken down by day of the week and hour of the day provided timely insights into peak and off-peak trading periods. Executives no doubt noticed much higher sales on certain common shopping days like Fridays, Saturdays and the days leading up to major holidays. Identifying the busiest shopping hours, typically early evening weekday hours after work, allowed better deployment of staff during high volumes. Conversely, very low sales late at night signified opportunity to adjust or reduce staff during graveyard shifts with little customer traffic.

Unit sales versus dollar sales metrics revealed to executives important intelligence about average transaction sizes and demand for higher-priced items. Stores seeing larger average order values most likely meant these locations were appealing to customers with more disposable income, carried higher-end product assortments or offered services promoting larger baskets. This type of insight helped shape purchasing, pricing, assortment and service strategies tailored to local demographics.

Granular sales data analyzed at the zip code or neighborhood level exposed micro-trends within territories that store-level views alone could not. Some surrounding areas clearly sent more patrons than others based on geo-location analysis. These neighborhood hotspots represented untapped opportunities for targeted marketing or even consideration of opening new stores. Weaker neighborhoods alerted managers to explore reasons for lack of uptake.

Customer behavior metrics provided via loyalty program data empowered executives to profile best customers and tailor the experience. Knowing top-spending customer demographics, preferred products, responsiveness to promotions allowed developing one-to-one engagement programs to deepen loyalty. Customer lifetime value insights quantified the long-term impact of converting occasional to returning shoppers through enhanced experiences based on data-driven segmentation and personalization.

In aggregate, the dashboard’s consolidated sales views, trend reporting and detailed metrics enabled managers to uncover otherwise obscured correlations, see the big picture across departments and regions, make more strategic resource allocation decisions with confidence, and continuously optimize operations with ongoing data-driven experimentation andfine-tuning. These dashboard-delivered insights aimed to drive overall top and bottom line growth for the entire retail organization.

Having access to such a robust sales and performance reporting tool allowed the company’s leadership to truly know their business inside and out. Regular examination of key metrics meant continual learning opportunities to stay ahead of industry changes and economic cycles. The insights gained surely helped superstore executives and managers make the most effective operational and strategic moves to profitably growth their multi-unit business for years to come.


Customer Self-Service Portal – Develop a customer self-service portal that allows external users like customers or clients to log support requests, check the status of existing requests, search a knowledge base for solutions, and view certain reports. The portal would integrate with the ServiceNow incident, problem, change, and knowledge management modules. Key aspects would include customizing the user interface and workflow, enabling authentication/authorization, and configuring data security access controls.

Enterprise Asset Management Application – Build out a comprehensive asset lifecycle management solution in ServiceNow for tracking all organizational assets from purchase to disposal. The application would provide capabilities for procurement, install base management, maintenance scheduling, software license tracking, and asset retirement. Multiple tables and views would need to be configured along with relating assets to locations, financial data, contracts, and users/roles. Workflows would be designed to automate tasks like notifying stakeholders of expiring warranties or maintenance due dates. Custom fields, catalogs, and approval processes could extend the solution for an organization’s specific asset types like IT, facilities, manufacturing equipment etc.

HR Service Delivery Platform – Create an HR service delivery platform where both employees and HR representatives can manage HR related tasks and requests entirely through ServiceNow. Modules could include a self-service portal, recruitment, onboarding, performance management, learning management, benefits administration, payroll processing, and more. New catalog items, workflows, and navigation menus would be required along with integrations to back-end HRIS and payroll systems. Dashboards and reports would provide metrics on things like time to hire, open positions, performance review completion, compensation, leaves and attendance.

IT Operations Automation – Automate various repetitive IT operations tasks through the development of custom workflows, applications, and integrations in ServiceNow. Examples include automatic password resets on user requests, approval-driven provisioning of new systems or services, security incident response checklists, virtual machine image deployment, cloud infrastructure provisioning via APIs, or application release management. Dashboards could track key metrics like mean time to repair/restore service, open tickets by priority, change failure rate. This consolidates what were likely manual, disconnected tasks across teams.

Integration Hub – Create ServiceNow as an integration hub to consolidate data and automate processes across various organizational systems. This could include building connectors and adapters to pull or sync data from HR, Finance, CRM and other line of business applications. Requirements gathering, data mapping, designing filters and transformations are key. Workflows are developed to trigger on events or data changes in source systems to initiate related actions in ServiceNow or downstream target systems. Administrative tools provide visibility and control over integrations. This centralizes and simplifies integrations versus point-to-point interfaces between each individual pair of systems.

Mobile Workforce Management – Build a mobile workforce management solution where field technicians use mobile applications and an optimised worker portal to manage their workload and tasks. The solution schedules and dispatches work orders to technicians based on their skills and availability. It provides turn-by-turn navigation, parts inventory lookup, issue resolution assistance, and time/expense tracking. Administrators can view performance metrics and job status. Features include geofencing, offline data capture, custom object extensions for work types, integration to inventory and scheduling systems. This brings paper-based processes digital for improved productivity and insight.

Each of these examples would require extensive configuration and customization within the ServiceNow platform to meet the specific requirements. Capstone implementation projects would focus on one of these use cases to really demonstrate a strong understanding of ServiceNow’s capabilities and best practices for application development. The key aspects to address with each project would include detailed requirements analysis, data modeling, UI/UX design, integration architecture, testing methodology, change management planning, and documentation/training. Substantial configuration, coding and development efforts would be needed to implement the necessary custom applications, workflows, dashboards and integrate with external systems. The project would culminate in deploying the solution to a test/pilot environment and demoing the features and benefits.

There are many opportunities for robust and meaningful capstone implementations leveraging the ServiceNow platform to automate processes, integrate systems and deliver modern service experiences across the enterprise. Projects that provide real business value through process optimization, data consolidation or improved workforce enablement allow students to apply their technical, analytical and project management skills at an advanced level. ServiceNow’s low code environment facilitates rapid prototyping and validation of concepts before going through the full development lifecycle.