Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
DeepCare Chatbot
Generating answers to customers using a hybrid approach
of
Deep Learning and NLP
Pascal van Kooten
2
Introduction
● First conference; first talk
● MSc Methods & Statistics, Utrecht University
● Innovating @ Jibes Data Ana...
3
Gartner’s Hype Cycle 2016
4
And this is where you can find me...
5
Fun projects
● whereami
Uses WiFi signals and machine learning to predict where you are
● deep_eye2mouse
Move the mouse ...
6
Chatbots
7
Chatbots
8
Chatbots
● These chatbots take the initiative
● AT&T experimented with chatbots, but cannot
see one online
● Fun experim...
9
Data
● Find “big QA data” in the English language
● @ATTCares, 1.3M tweets as of May 2017
● 100k Q&A collected through t...
10
Problem
● They receive ~3000 tweets per day
● Corporations want an automated solution
● Very versatile problems:
– Info...
11
Goal
● Without business constraints…
● Try to see if a bot can reliably reply like a human
● Combine NLP and Deep Learn...
12
http://deepcare.online
(available during PyData)
13
Examples (1/2)
Q: I want an iphone for xmas
A: santa heard you, pvkooten order an iphone at att.com/search?
q=iphone & ...
14
Examples (2/2)
Q: I want to sell an iphone
A: I'm not sure if an iphone is what you really want. I can only help you if...
15
It’s not only deep learning
● Logical mistake:
Q: Stop my samsung contract
A: Ok, I stopped your iphone contract
16
Overview
● Pre-processing
– Generic / Twitter related
– Domain specific
● Phase 1: NLP Model
● Phase 2: Deep Learning a...
Preprocessing
Pascal van Kooten
18
Case study
Q: i want a case for my iphone 6
A: we have got you covered, pvkooten get a fab case @
att.com/search?q=ipho...
19
Generic preprocessing
● Name (API provides actual name) NAME→
● Employee name+time AVAILABILITY→
● Tags (@something) TA...
20
Specific preprocessing
● Brand name
● Products
● Competitors
21
Preprocessing
● Preprocessing to help generalize
● Only preprocess things that can be reused in the
domain
● Plus a cru...
NLP Model
Restrict answering
Pascal van Kooten
23
Traditional NLP
● Depending on language model
– Lemmatization (cats → cat)
– Part of Speech (cat → NOUN)
– Dependency t...
24
Model explanation
● spaCy (https://spacy.io/)
● Rule uses:
– Part of Speech
– Lemma
– Negation
For a sentence,
- need/w...
25
Case study
Q: i want a case for my iphone 6
A: we have got you covered, pvkooten get a fab case @ att.com/search?q=ipho...
26
Examples
want galaxy not iphone
not iphone ✓ want iphone!
Generative model
Generating answers
Pascal van Kooten
28
Generative neural network model
● Popularized by Google’s seq2seq:
A general-purpose encoder-decoder framework for Tens...
29
Generative neural network model
30
Generative neural network model
https://indico.io/blog/sequence-modeling-neuralnets-part1/
31
Code
● tensorflow
● Started generative model code from
Conchylicultor/DeepQA
32
Neural network parameters
● batch_size: 128
● glob_step: 3186
● max_length: 25
● learning_rate: 0.001
● embedding_size:...
33
Conclusion
●
It makes small grammar mistakes
●
It can still make mistakes in logic
– Mostly with unseen data
● Not opti...
34
Questions?
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
What to Upload to SlideShare
Next
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

DeepCare Chatbot - Generating answers to customers using a hybrid approach of Deep Learning and NLP

Download to read offline

by Pascal van Kooten at PyData Barcelona 2017
Description:

The DeepCare chatbot is capable of learning to answer customer questions. Using a hybrid approach of NLP and Deep Learning, it tries to combat logical fallacies that occur in pure deep learning bots, while still coming up with unique answers. A live demo will be available at http://deepcare.online on the day of the talk.

  • Be the first to like this

DeepCare Chatbot - Generating answers to customers using a hybrid approach of Deep Learning and NLP

  1. 1. DeepCare Chatbot Generating answers to customers using a hybrid approach of Deep Learning and NLP Pascal van Kooten
  2. 2. 2 Introduction ● First conference; first talk ● MSc Methods & Statistics, Utrecht University ● Innovating @ Jibes Data Analytics for ~3 years ● Working for companies to investigate: – Blockchain – Computer Vision – Machine Learning / NLP ● Human Machine Interaction
  3. 3. 3 Gartner’s Hype Cycle 2016
  4. 4. 4 And this is where you can find me...
  5. 5. 5 Fun projects ● whereami Uses WiFi signals and machine learning to predict where you are ● deep_eye2mouse Move the mouse by your webcam + gaze ● sky Using machine intelligence for web crawling ● Neural Complete (presentation in 2 weeks in Moscow) Neural network to auto complete neural network code ● And…. DeepCare github.com/kootenpv
  6. 6. 6 Chatbots
  7. 7. 7 Chatbots
  8. 8. 8 Chatbots ● These chatbots take the initiative ● AT&T experimented with chatbots, but cannot see one online ● Fun experiment to have a reactive chatbot
  9. 9. 9 Data ● Find “big QA data” in the English language ● @ATTCares, 1.3M tweets as of May 2017 ● 100k Q&A collected through twitter api ● ~15k related to “wanting/needing iphone”
  10. 10. 10 Problem ● They receive ~3000 tweets per day ● Corporations want an automated solution ● Very versatile problems: – Information requests – Complaints – Ask for change in system ● Bot needs to detect when it should be quiet ● First task: “social selling”
  11. 11. 11 Goal ● Without business constraints… ● Try to see if a bot can reliably reply like a human ● Combine NLP and Deep Learning – NLP for transparency, answer yes/no – Deep learning for variation in response
  12. 12. 12 http://deepcare.online (available during PyData)
  13. 13. 13 Examples (1/2) Q: I want an iphone for xmas A: santa heard you, pvkooten order an iphone at att.com/search? q=iphone & get #rolloverdata asap. ^fakeAT&Tbot t.co/J4c8G3KR1S Q: can't decide if I want an iphone 7 or samsung note 4 A: we have great deals on both! get both this your lifestyle! check them out here att.com/search?q=iphone 7 ^fakeAT&Tbot Q: I want to see a movie A: I'm not sure if an iphone is what you really want. I can only help you if you want an iphone.
  14. 14. 14 Examples (2/2) Q: I want to sell an iphone A: I'm not sure if an iphone is what you really want. I can only help you if you want an iphone. Q: I want to buy an iphone A: we have got what you need, pvkooten grab an iphone w/ #rolloverdata @ att.com/search? q=iphone today! t.co/J4c8G3KR1S Q: I need a charger for my iphone A: we have one for you, pvkooten get a new charger & many other code tweet-@ att.com/search?q=iphone charger for 20%!^fakeAT&Tbot Q: i want a case for my iphone 6 A: we have got you covered, pvkooten get a fab case @ att.com/search?q=iphone 6 case %20cover%20 g & dm me for $$$ off! ^fakeAT&Tbot t.co/J4c8G3KR1S
  15. 15. 15 It’s not only deep learning ● Logical mistake: Q: Stop my samsung contract A: Ok, I stopped your iphone contract
  16. 16. 16 Overview ● Pre-processing – Generic / Twitter related – Domain specific ● Phase 1: NLP Model ● Phase 2: Deep Learning answer generation ● Post-processing
  17. 17. Preprocessing Pascal van Kooten
  18. 18. 18 Case study Q: i want a case for my iphone 6 A: we have got you covered, pvkooten get a fab case @ att.com/search?q=iphone 6 case%20cover%20 g & dm me for $ $$ off! ^fakeAT&Tbot t.co/J4c8G3KR1S is seen after the preprocessing step as: Q: i want a case for my PRODUCT A: we have got you covered, NAME get a fab case @ OFFER_LINK & dm me for MONEY_AMOUNT off! INITIALS WEBCARE_LINK
  19. 19. 19 Generic preprocessing ● Name (API provides actual name) NAME→ ● Employee name+time AVAILABILITY→ ● Tags (@something) TAG→ ● Links LINK (experimented)→ ● Emoticons
  20. 20. 20 Specific preprocessing ● Brand name ● Products ● Competitors
  21. 21. 21 Preprocessing ● Preprocessing to help generalize ● Only preprocess things that can be reused in the domain ● Plus a crucial component to a company: PRODUCT ● ….
  22. 22. NLP Model Restrict answering Pascal van Kooten
  23. 23. 23 Traditional NLP ● Depending on language model – Lemmatization (cats → cat) – Part of Speech (cat → NOUN) – Dependency tree (I → subject) ✓Allows generalization! ✗Depends on language model getting it right ✗Feature explosion makes it difficult to generalize ✗Creating rules can be time consuming ✓ It’s very transparant ✓ Rules can give guarantees deep learning can’t
  24. 24. 24 Model explanation ● spaCy (https://spacy.io/) ● Rule uses: – Part of Speech – Lemma – Negation For a sentence, - need/want... is the lemmatized verb - which has PRODUCT as a nearby child - but no negation dependency on the same level or above as PRODUCT - the subject is “I” - ???
  25. 25. 25 Case study Q: i want a case for my iphone 6 A: we have got you covered, pvkooten get a fab case @ att.com/search?q=iphone 6 case%20cover%20 g & dm me for $$$ off! ^fakeAT&Tbot t.co/J4c8G3KR1S want (VERB, ROOT) ✓ ---- I (PRON, nsubj) ✓ ---- case (NOUN, dobj) -------- a (DET, det) -------- for (ADP, prep) ------------ iphone (NOUN, pobj) ✓ ---------------- my (ADJ, poss) ---------------- 6 (NUM, nummod)
  26. 26. 26 Examples want galaxy not iphone not iphone ✓ want iphone!
  27. 27. Generative model Generating answers Pascal van Kooten
  28. 28. 28 Generative neural network model ● Popularized by Google’s seq2seq: A general-purpose encoder-decoder framework for Tensorflow that can be used for Machine Translation, Text Summarization, Conversational Modeling, Image Captioning, and more. ● Map sequence of words to another sequence of words ● Chinese sentence to English sentence ● Given training examples, minimize loss ● Can also be used for creating a chatbot ● Most likely a variation on this model is being used by Google Translate
  29. 29. 29 Generative neural network model
  30. 30. 30 Generative neural network model https://indico.io/blog/sequence-modeling-neuralnets-part1/
  31. 31. 31 Code ● tensorflow ● Started generative model code from Conchylicultor/DeepQA
  32. 32. 32 Neural network parameters ● batch_size: 128 ● glob_step: 3186 ● max_length: 25 ● learning_rate: 0.001 ● embedding_size: 128 ● hidden_size: 512 ● num_layers: 2 --------------------- loss: 0.06 time: 24 hours
  33. 33. 33 Conclusion ● It makes small grammar mistakes ● It can still make mistakes in logic – Mostly with unseen data ● Not optimal because 2 models, but fun experiment ● Like OpenAI…. ● Shows signs of sentiment: Q: I want an iphone but my mom says iphones are the devil A: do not get upset! order an iphone at att.com/search?q=iphone iphones with…
  34. 34. 34 Questions?

by Pascal van Kooten at PyData Barcelona 2017 Description: The DeepCare chatbot is capable of learning to answer customer questions. Using a hybrid approach of NLP and Deep Learning, it tries to combat logical fallacies that occur in pure deep learning bots, while still coming up with unique answers. A live demo will be available at http://deepcare.online on the day of the talk.

Views

Total views

470

On Slideshare

0

From embeds

0

Number of embeds

1

Actions

Downloads

15

Shares

0

Comments

0

Likes

0

×