This presentation offers an introduction to dialogue machines/chatbots. Speaker: Batool Arhamna Haider (Natural Language Understanding Researcher - Silicon Valley, CA, USA) LinkedIn: https://www.linkedin.com/in/batool-arhamna-haider-85655b18

1. Batool Arhamna Haider
Lecture prepared for Habib University, Pakistan, Nov 7th 2019
Photo: https://www.freepik.com/premium-vector/fly-robot_2430923.htm
A Bird Eye View of Dialogue Machines

2. About me
Applied Scientist at Amazon – AI (2019)
Data Scientist @UHG (2017)
Masters at Stanford University (2015)
Bachelors at NED University (2011)
2

3. They were always there in our imagination…
Rosiefrom the Jetsons
TARS of the Interstellar
Baymax of the Big Hero 6
3

4. Dialogue Machines
4

5. In today’s talk
5
Warm-up (30 min)
1. What are dialogue machines? Their evolution…
2. Meet Alexa

6. In today’s talk
Nuts and Bolts (30 min)
3. Common parts of most dialogue systems
4. Meet Lex
6
Warm-up (20 min)
1. What are dialogue machines? Their evolution…
2. Meet Alexa

7. In today’s talk
Warm-up (20 min)
1. What are dialogue machines? Their evolution…
2. Meet Alexa
Nuts and Bolts (20 min)
3. Common parts of most dialogue systems Meet Lex
4. Meet Lex
The Future (15 min)
4. Challenges & Research
7

8. In today’s talk
Warm-up (20 min)
1. What are dialogue machines? Their evolution…
2. Meet Alexa
Nuts and Bolts (20 min)
3. Common parts of most dialogue systems
4. Meet Lex
The Future (15 min)
4. Challenges & Research
8Bonus: Win ½ million dollars building chats!

9. Meet Alexa 
9
What is a Dialogue Machine?

10. The world is exploding with Chatbots!
Add reference
More than 100 million devices with Alexa assistant have been sold
Google Home Sales Exploded by 483% in 2017
10

11. 11Picture: https://medium.com/searce/paving-the-way-to-a-chatbot-revolution-c7ea21fa2f2f

12. Broad Categories of Dialogue Machines
12
•Chitchat bots
•Goal Oriented bots

13. ChitChat Bots
13
Bot
User
What have you been upto?
Was listening to music…
What music were you listening to?
Jazz. What music do you like?

14. Meet Mitsuku!
• Live demo: https://www.pandorabots.com/mitsuku/
14

15. Can be broadly divided into 2 categories
•Chitchat bots
•Goal Oriented bots
15

16. Goal-Oriented Chat Bots
16
Bot
User
Can you be a flight?
Sure! Where and when?
To Boston. I want to fly tomorrow.
Sure! Ticket booked!
intent

17. Evolution of Chatbots – 3 Phases
17Photo: https://www.pinterest.com/pin/421297740124501580/ https://emerj.com/ai-sector-overviews/chatbots-for-customer-service/

18. Evolution of Chatbots
•Phase I: Rules
18

19. Evolution of Chatbots
•Phase I: Purely rule based
User: “My mother cooks good meal”
Bot: “My mother likes to sing”
19
User: “I love to sing too! Whitney is my favorite!”
Bot: “Oh! I love Whitney’s One Moment in Time song”

20. Evolution of Chatbots
•Phase I: Purely rule based
User: “Book flight to USA”
Bot: book_flight_intent
20

21. So many rules!!
21

22. Evolution of Chatbots
•Phase II: Statistical Models
22

23. Evolution of Chatbots
•Phase II: Statistical Model (aka machine learning)
User: “Book flight to USA”
Bot: book_flight_intent
Logistic
Regression
Words as features Intent Label
23

24. Words as Features
Sentence 1: “Book flight to USA”
Sentence 2: “Reserve flight”
24

25. Words as Features
Sentence Book Flight to USA Reserve fly flights … Label
1 1 1 1 1 0 0 0 0 Book_flight
2 0 1 0 0 1 0 0 0 Book_flight
X = features
Y=labels
Could also use more sophisticated techniques such as TF-IDF etc 25

26. Challenges
•Requires a lot of feature engineering
•Does not benefit with more and more data
26

27. Evolution of Chatbots
•Phase II: Deep learning
27

28. Phase III: Deep Learning
User: “Book flight to USA”
User: “Reserve for me flights to USA”
User: “I would like to purchase a ticket”
Bot: book_flight_intent
Neural NetsWords as features Intent Label
28

29. Examples
Reference
29

30. Word Embeddings
30
“You shall know a word by the company it keeps!”
John R. Firth

31. Phase III: Deep Learning
User: “Book flight to USA”
User: “Reserve flight”
User: “I would like to purchase a ticket”
Bot: book_flight_intent
Neural NetsWords as features Intent Label
31

32. Word Features for Neural Networks
32
0 0 0 …. 1 …. 0 ... 0 0 0Book
Vocabulary
One-hot vectors
Utterance : “Book Flight”

33. Word Features for Neural Networks
33
0 0 0 …. 1 …. 0 ... 0 0 0Book
Flight 0 0 0 …. 0 …. 1 ... 0 0 0

34. Embeddings
34
Book flight
Embedding Layer
Covert to indices

35. Embedding Layer
35
Book flight
Embedding Layer
0.3
0.4
0.1
0.1
-0.4
0.2
0.0
0.2
Word Embeddings
Book flight
Covert to indices

36. Encoding Layer
36
Book flight
Embedding Layer
0.3
0.4
0.1
0.1
-0.4
0.2
0.0
0.2
Word Embeddings
Book flight
Covert to indices
Encoding Layers
0.4
0.1
0.0
0.2
Sentence Embeddin
Book flight

37. Classification Layer
37
Encoding Layers
0.4
0.1
0.0
0.2
Sentence Embedding
Book flight Classification Layer
Output
Intent Label

38. New to Neural-Nets?
38
1.Neural Networks and Deep Learning
2.Improving Deep Neural Networks
3.Structuring Machine Learning Projects
4.Convolutional Neural Networks
5.Sequence Models
5 Course Series taught by Prof. Andrew Ng
https://www.coursera.org/specializations/deep-learning

39. New to Neural-Nets?
39
By Aurélien Géron

40. What do State-of-the Art Chatbots Look like?
• Video Demo
https://www.facebook.com/zubairahmed.a.i/videos/1015618705682
6517/UzpfSTEzMTgyNDk5Njg6MTAyMTg4ODY2ODMzMDA5NDY/
40

41. Nuts and Bolts (20 min)
4. Common parts of most dialogue systems
3. Meet Lex
41

42. Parts of a Dialogue System
42
Dialogue State
Tracker
Natural Language
Understanding
Speech or Text Action
Natural Language
Generation
Speech Synthesis
Input
Speech
Identification
Third Party Software
Sentiment
Output
Speech -> Text

43. Natural Language Understanding
43
How are you?
Can I book a flight to Boston?
Intent detected: book_flight_intent
Slots detected: Arrival_city = Boston
Bot Prompts User Utterances
Sure! When would you like to fly?
Dec 21st

44. Meet Lex! 
• Live Demo
44

45. Natural Language Understanding
45
Can I book a flight to Boston?
Neural Network
Block
Book_flight_intent O O O O O O Arrival_City
Intent Classification Slot Labelling

46. Parts of a Dialogue System
46
Dialogue State
Tracker
Natural Language
Understanding
Speech or Text Action
Natural Language
Generation
Speech Synthesis
Input
Speech
Identification
Third Party Software
Sentiment
Output
Speech -> Text

47. Dialogue State Tracker
47
How are you?
Can I book a flight to Boston?
Sure! When would you like to fly?
Current Intent detected: book_flight_intent
Slots To Elicit: arrival city, arrival date
Previous Intent: None
Previous utterances: None
Dec 21st
Bot User

48. Natural Language Understanding
48
Can I book a flight to Boston?
Neural Network
Block
Book_flight_intent O O O O O O Arrival_City
Intent Classification Slot Labelling
Context Signals

49. Parts of a Dialogue System
49
Dialogue State
Tracker
Natural Language
Understanding
Speech or Text Action
Natural Language
Generation
Speech Synthesis
Input
Speech
Identification
Third Party Software
Sentiment
Output
Speech -> Text

50. Google books an Appointment
• Video Demo
https://www.youtube.com/watch?v=YCWJ0z6_z34
50

51. Future
Challenges & Researches
51

52. A few of many challenges & research areas
52
1. They forget!
Pretrained models, meta-learning, research around catastrophic
forgetting etc
2. Require a lot of annotated data; real time supervision is hard
Pretrained models, weak supervision, reinforcement learning etc
3. Do they really understand our world? Will they ever?
4. Multi-modality

53. Visual Q&A
• http://vqa.cloudcv.org/
53

54. How to win 1/5 Million Dollars?
54

55. Alexa Prize 2020!
55
https://developer.amazon.com/alexaprize?fbclid=IwAR1Uj6DwDb67Bxqiijw1ZjWWHtTTQ43FXrAKnf-AxouVezKdvXJVCdjGqpk

56. Questions? Bring it on!
56

57. TARS from the Movie Interstellar
“Everybody good? Plenty of slaves for my robot colony?” (Tars) 57