Visa

1. Natural Language Processing
Concepts, and Comparison of
Commercial Platforms
Sumit Amar
Sr. Director, Innovation Development, Visa
sumitamar@hotmail.com

2. Disclaimer
The opinions expressed in this presentation and on the following slides are solely those of the
presenter, and not necessarily those of Visa. Products, Sites, and tools mentioned focus on
commercial available technology, and are for illustrative, educational purposes only. The views
expressed in this presentation are solely those of the presenter, who do not endorse or recommend
any commercial products, process, or service.

3. Agenda
Introduction to Natural Language Processing
Human Language vs Machine Language
NLP Applications
Commercial Off-the-shelf Platforms
Summary
Questions

4. Natural Language Processing
A way for computers to analyze,
understand, and derive meaning from
human language in a smart and useful way.
Intersection of artificial intelligence,
computational linguistics and computer
science

5. Human Language vs Machine Language
“The spirit is willing but the flesh is weak” =>
To Russian
“The vodka is good but the meat is rotten”
“Out of sight, out of mind” =>
To Russian
“Blind and insane”

6. Human Language vs Machine Language
(Ambiguity)
• The Pope's Baby Step on Gays.
• The man saw a boy on a hill with a telescope.
• Boy paralysed after tumour fights back to gain a black belt.
• Will Will will the will to Will?

7. NLP Levels
Phonological Analysis - Speech
Tokenization - Text

8. NLP Applications
Automatic Summarization
Text Classification
Sentiment Analysis
Named Entity Extraction
Machine Translation
Topic Extraction
Stemmer
Conversational Agents (Chatbots)

9. Automatic Summarization
Extraction
Abstraction
Problem of Dangling Anaphora
Multi-document Summarization

10. Text Classification
Spam Detection
Category classification
Authorship Attribution

11. Sentiment Analysis (Opinion Mining)
Polarity Detection
General Inquirer
Subjective vs Objective Analysis
Combined with Recommender Systems

12. Named Entity Recognition
Linguistic Grammar
Statistical Learning
Conditional Random Fields

13. Machine Translation
• Rule Based
• Transfer Based
• Interlingual Based
• Dictionary Based
• Statistical
• Example Based
• Hybrid (Statistical and Rule Based)
• Neural (Deep Learning)
Approaches:

14. Topic Extraction
Extraction of important keywords
from text
Precursor to Machine Translation
and Conversational bots

15. Stemmer
Production Technique
Suffix Stripping
Lemmatization
Stochastic Method
Matching words
Affix

16. Conversational Agents / Chat bots
Part of Speech Tagging (PoS Tagging) Identification of:
• Utterance – Form of sentence part to
convey a message
• Intent – Intended task to accomplish
• Slot (aka Entity or Parameter) – Details
required for fulfilling an intent
• Context – A workspace that keeps alive
meaning and parameters in a session
• Session – An entire conversation until
an Intent is satisfied

17. Off the shelf NLP Platforms
Google Api.ai (DialogFlow)
Microsoft LUIS (Language Understanding Intelligent Service)
IBM Watson Conversation Service
Facebook Wit.ai
Amazon Lex

18. Table Source: Digiteum
Comparing
Commercial
Platforms

19. Summary
NLP Basics
NLP Applications
NLP Application Approaches
Problems in NLP caused by ambiguity
Commercial NLP Platforms

20. References
http://content.time.com/time/magazine/article/0,9171,2148640,00.html
https://www.express.co.uk/life-style/health/450562/Boy-paralysed-after-tumour-fights-back-to-gain-a-black-belt
https://computerlanguageisp.wordpress.com/2013/01/25/ambiguity-the-man-saw-the-boy-on-the-hill-with-the-telescope/
http://www.businessinsider.com/weird-sentences-2014-1
https://monkeylearn.com/blog/definitive-guide-natural-language-processing/
http://www.digiteum.com/nlp-engines-for-chatbots/
https://www.youtube.com/watch?v=OQQ-W_63UgQ

21. Thank you