1.
SENTIMENT
ANALYSIS
Why and How
Swiss Group for Artificial Intelligence and Cognitive Science - 2nd SGAICO at HSLU – May 23rd, 2017
https://sgaico.swissinformatics.org/

2.
Who am I
Dr. Amancio Bouza
Intrapreneur & Artificial Intelligence Expert
Let’s connect:
https://ch.linkedin.com/in/amanciobouza
Follow me:
https://twitter.com/AmancioBouza

3.
3
WHY?Do we want Sentiment Analysis

4.
4
Identify (un)happy customers?

5.
Simple Alternative: Net Promoter Score (NPS)
How likely do you recommend
Travelito to your friends?
5
very unlikely very likely

6.
6
So, WHY
then?

7.
7
1. Improve Products
2. Enable Closed-Loop
Customer Experience
3. Understand Competition

8.
8
How to improve Products

9.
9
“The bed in the City Hotel was not
comfortable.
However, the breakfast was
excellent with good food and
excellent service.
We had a wonderful time and will
travel with Travelito again.”
Example of customer feedback for anonymized company in tourism industry

10.
Analyze Entities Related to Negative Feedbacks and Drill Down
10
Named Entity
Recognition:
- Organization
- Location
- Person
Can you
spot the
problem? ;)

11.
11
Enable Closed-Loop Customer
Experiences

12.
React in Real-Time to Customer Disatisfaction
12
End-Customers tell you
if they are unhappy
Rest assured
they tell the
world too!

13.
13
Happy Customers
Come Back
Happy Customers
Become Fans and
Spread the Word

14.
Use Social Media Channels for Real-Time Sentiment
Analysis
14
Mitigate
Dissatisfaction at
the right time
Keep
retention
(loyalty) and
referrals high

15.
15
Competition Intelligence

16.
Identify Strength and Weaknesses of Competition
16
Find out Competitive Advantage of
your Competition…
…and
Make it Better or
Make it Irrelevant

17.
Find Info on Review Platforms
17

18.
18
WHAT?Is the Challenge

19.
Customers do
provide feedback,
lots of feedback

20.
… and that’s GREAT!
20

21.
But How to Process so
Many Customer
Feedbacks?
21

22.
22
HOW?To do Sentiment Analysis

23.
Let’s try
Sentiment
Analysis with
Machine Learning

24.
24
Disclaimer
Sentiment Analysis PoC was done in few days
No extra Cognitive Services used besides ML Studio
Dataset consisted of 40k customer feedbacks
Prior-Probability of negative Feedback ~45%
Evaluation was done with 10-fold Cross Validation

25.
Proposal
25
Use
whatever
you want

26.
Following the Lean Approach Follow the
Lean Approach
CRISP-DM
Cross Industry Standard
Process for Data Mining
26

27.
27

28.
Sentiment Analysis PoC in a Nutshell
Data Cleaning
Remove uncomplete data
Data Preparation
What to learn , i.e., sentiment
How to learn, i.e., cognitive computing
Machine Learning
Learn to identify sentiments
Sentiment Analysis
Identify sentiments in customer feedback
28

29.
29
Common Text Preprocessing applied
- Lemmatization
- Normalize Lower Case
- and much more
Best Performer for this dataset
Two-Class Boosted
Decision Tree
500 Features were selected for this dataset selected with 𝜒2
Extraction of N-Gram Features
from Text

30.
Key Experiences with Azure ML Studio for Sentiment Analysis
+ 1-Gram Features provided best results
+ Preprocessing and Normalization of Text is simple
- Data Preparation was cumbersome with built-in function blocks -> use R
- Key Phrase Extraction was useless to extract sentiments
- Handover of ML Workspace to others is complicated
- Automated creation of Web Service isn’t intuitive. Requires rethinking of the ML workflow
+ However, it provides great API Documentation, Sandbox and examples on the fly
+ Intuitive Machine Learning Studio
+ Fast & simple setup of (scientifically valid) experiments
+ High Computational Performance
+ Support for R / R required for advanced stuff, data preparation in particular
30

31.
31
WHAT?Is the Result

32.
32
“The bed in the City Hotel was not
comfortable.
However, the breakfast was
excellent with good food and
excellent service.
We had a wonderful time and will
travel with Travelito again.”
Example of customer feedback for anonymized company in tourism industry

33.
Sentiment Analysis PoC ranks with human to interpret sentiment
33
79%
Accuracy

34.
Evaluation results in detail: 79% Accuracy
34
Accuracy Precision Recall F-Score AUC
0.788 0.751 0.733 0.742 0.853

35.
Take Home Message: WHY do we do Sentiment Analysis
35
Improve Products Closed-Loop
Customer Experience
Competition
Intelligence

36.
Who am I
Dr. Amancio Bouza
Intrapreneur & Artificial Intelligence Expert
Let’s connect:
https://ch.linkedin.com/in/amanciobouza
Follow me:
https://twitter.com/AmancioBouza