This document discusses common NLP problems, including sentiment analysis, chatbots, translation services, and document summarization. It then presents four case studies applying NLP techniques: 1) Using chatbot conversation data and topology to understand customer groups, 2) Using text classification for product types, 3) Using topic modeling on poetry to classify poems by genre, 4) Using linguistic analysis and changepoint detection on public statements to understand changes in a leader's behavior during war. Finally, it lists helpful Python packages for NLP, topology, and modeling.

1. NLP IN THE WILD
COLLEEN M. FARRELLY, DATASEMBLY

2. COMMON INDUSTRY NLP
PROBLEMS
• Sentiment analysis/tracking of customer
feedback
• Computational linguistics/psychology of
language usage
• Chatbots
• Translation services
• Supervised learning
• Document summary

3. Problem
formulation
Data
collection
Choice of
tools
(math/ML)
Application Results

4. CASE STUDY 1: CONSUMER GROUP
CLUSTERING
• Want to understand how
different groups interact
with a chatbot
• Sales implications
• Groups-specific needs
for future feature builds
• Chatbot conversation data
sample
• NLP to derive salient text
features
• Persistent homology to

5. CASE 2: SUPERVISED LEARNING
• Want to classify products by
type (such as fruit or canned
soup) using title text
• Data includes a small sample of
scraped titles from a sample of
retailers with manual annotation
of product type
• Text cleaning and embedding
algorithms to prepare the text
data for machine learning
• Supervised learning algorithm
to create the classier

6. CASE 3: TOPIC
MODELING
• Want to find main
topics discussed in a
corpus of documents
(poems)
• Poetry data sample
across genres of
poetry by a single
author
• Topic modeling to
classify poems

7. CASE 4:
TIME-
BASED
ANALYSIS
OF
MINDSET
• Want to quickly understand
changes in leader’s behavior at
onset of war
• Public statement sample by
president over course of several
weeks as input data
• NLP to derive linguistic features
• Longitudinal models and topology-
based changepoint algorithm on
linguistic feature time series

8. HELPFUL
PYTHON
PACKAGES
• NLP:
• NLTK (parts of speech tagging,
munging data…)
• Gensim (topic models)
• Vader (sentiment analysis)
• TDA
• Persim/ripser (persistent
homology)
• Kmapper (Mapper algorithm)
• Structural equation modeling/latent
class modeling
• Semopy (similar to lavaan in R)

9. CONTACT ME
• cfarrelly@med.miami.edu
• LinkedIn (Colleen M. Farrelly)