1. Tom Bocklisch
Head of Engineering
Proprietary Material.
Any use of this material without specific permission of Rasa is strictly prohibited.
Conversational AI with
Rasa NLU & Rasa Core

2. Conversational AI will
dramatically change how
your users interact with
you.
Example:
A customer can change her
address via Facebook Messenger
Play demo video

3. An open source, highly scalable ML
framework to build
conversational software

4. Rasa the OSS to build conversational software with ML
Introduction
Backend,
database,
API, etc.
Dialogue
Management
“Brain”
Input Modules
“Ears”
NLU, GUI elements,
context, personal
info
Output
Modules
“Mouth”
NLG, GUI elements
Connector
Modules
Connector to any
conversational
platforms
“What’s the weather
like tomorrow?”
(User Request via
text or voice)
“It will be sunny and
20°C.”
(AI response via
text or voice)
Alternatives:

5. Why Rasa?
Introduction
Runs Locally
● No Network
Overhead
● Control QoS
● Deploy anywhere
Hackable
● Tune models for your
use case
Own Your Data
● Don’t hand data over
to big tech co’s
● Avoid vendor lock-in

6. About Me
Introduction
● Studied Computer Science in Germany
● Went on to found a consultancy focused on image and text
analysis - https://scalableminds.com
● Joined the Rasa Team http://rasa.com - leading the development
of the open source stack as well as internal tools
If you have any questions about this talk, Rasa or me - I’ll be around.

7. What we are focusing on today
Introduction
Goal:
build & understand a bot based on machine learning
Roadmap:
1. Natural Language Understanding
i. Theory
ii. Let’s Code
2. Dialogue Handling
i. Theory
ii. Let’s Code
3. Research
4. Questions

8. Setup
Introduction
1. Jupyther notebook in python 3.6 (2.7 should work as well)
2. Download:
Ipython Notebook: http://tiny.cc/rasa-tut
Repository: https://github.com/tmbo/rasa-demo-pydata18

9. Under The Hood
Natural Language Understanding

10. Rasa NLU: Natural Language Understanding
Under the Hood
Goal: create structured data
I have a new address, it’s
709 King St, San Francisco

11. Rasa NLU: Natural Language Understanding
Under the Hood
Natural Language
Understanding
What’s the
weather like
tomorrow?
Example Intent Classification Pipeline
”What’s the weather like tomorrow?” { “intent”: “request_weather” }
Vectorization Intent Classification

12. Rasa NLU: Natural Language Understanding
Under the Hood
Bags are your friend
Bag
of
words
SVM
greet
goodbye
thank_you
request_weather
confirm
What’s the
weather like
tomorrow?

13. Rasa NLU: Natural Language Understanding
Under the Hood
Natural Language
Understanding
What’s the
weather like
tomorrow?
Example Entity Extraction Pipeline
”What’s the weather like tomorrow?” { “date”: “tomorrow” }
Tokenizer
Part of Speech
Tagger
Chunker
Named Entity
Recognition
Entity Extraction
Example Intent Classification Pipeline
”What’s the weather like tomorrow?” { “intent”: “request_weather” }
Vectorization Intent Classification

14. Rasa NLU: Entity Extraction
Under the Hood
Where can I get a burrito in the 2nd arrondissement ?
cuisine location
1. Binary classifier is_entity & then entity_classifier
2. Direct structured prediction
averaged perceptron

15. Let’s Code
Natural Language Understanding

16. Let’s build a fun little bot
Under the Hood

17. Let’s build a fun little bot
Under the Hood

18. Under The Hood
Dialogue Handling

19. Let’s get you awake again!
Everyone get a partner and an arms length of space.

20. Under The Hood
Dialogue Handling

21. Why Dialogue Handling with Rasa Core?
Under The Hood
● No more state machines!
● Reinforcement Learning: too much
data, reward functions...
● Need a simple solution for everyone

22. Why Machine Learning?
Under The Hood

23. State Machines are infeasible
Under The Hood

24. Rasa Core: Dialogue Handling
Under The Hood
“What’s the weather
like tomorrow?”
Intent
Entities
next
ActionState
previous
Action
“Thanks.”
after next
Action
updated
State
“It will be sunny
and 20°C.”
SVM
Recurrent NN
...

25. Rasa Core: Dialogue Handling
Under The Hood
“What’s the weather
like tomorrow?”
“It will be sunny and
20°C.”
Entity
Input
Action Mask
Renormal-
ization
Sample
action
Action
type?
Response
API Call
Recurrent
NN
API Call
Entity
Output
Intent
Classification
Entity
Extraction
Similar to LSTM-dialogue prediction paper: https://arxiv.org/abs/1606.01269

26. Let’s Code
Dialogue Engine

27. Rasa Core: Dialogue Training
Under The Hood
Issue: How to get started? Online Learning→
What’s the weather
like tomorrow?
How did you like it?
Correct wrong
behaviour
Retrain model
It will be sunny and
20°C.

28. Let’s Code
Interactive Learning

29. Research

30. Training NLU models without initial word vectors
Research
Goal: Learn an embedding for the intent labels based on the user messages
https://medium.com/rasa-blog/supervised-word-vectors-from-scratch-in-rasa-nlu-6daf794efcd8
● Learns joined embeddings for intents & words at the same time
● Allows multi-intent labels
● Knows about similarity between intent labels
● Based on Starspace Paper

31. Training NLU models without initial word vectors
Research
Goal: Learn an embedding for the intent labels based on the user messages
Evaluation:
Multi-Intent:

32. Generalisation across dialogue tasks
Research
Why do we need this complex architecture? For generalisation between domains!

33. Extensions

34. Rule Based Models (e.g. Duckling):
● Extract entities with restricted formats using e.g. patterns
● Allow for normalization (e.g. “tomorrow” → 26.05.18)
Pre-Trained Entity Models
Extensions
Reusing Trained ML models (e.g. spaCy builtins):
● Trained on large corpora on common entities (persons, cities)
● Usually restricted to the language they are trained on

35. Form Filling
Extensions
● Make some form logic deterministic
● Request all fields and then call API
● Simplifies action space

36. Final Thoughts

37. Closing The Loop
Final Thoughts
“What’s the weather
like tomorrow?”
(User Request via
text or voice)
New Training
Data
Retrain ML
Models
Correct Training
Data
Relabel
Collected Data
Use Improved
Model

38. Open challenges
Final Thoughts
● Handling OOV words
● Multi language entity recognition
● Combination of dialogue models
We’re constantly working on improving our models!
For those that are curious:

39. Current Research
Final Thoughts
Good reads for a rainy day:
● Last Words: Computational Linguistics and Deep Learning (blog)
https://goo.gl/lGSRuj
● Starspace Embeddings (paper)
https://arxiv.org/abs/1709.03856
● End-to-End dialogue system using RNN (paper)
https://arxiv.org/pdf/1604.04562.pdf
● MemN2N in python (github)
https://github.com/vinhkhuc/MemN2N-babi-python
● Sentence Embeddings (blog)
https://medium.com/huggingface/universal-word-sentence-embeddings-ce48ddc8fc3a

40. ● Techniques to handle small data sets are key to get started with
conversational AI
● Deep ML techniques help advance state of the art NLU and
conversational AI
● Combine ML with traditional Programming and Rules where
appropriate
● Abandon flow charts
Summary
Final Thoughts
4 take home thoughts:

41. Thanks! Tom Bocklisch
Head of Engineering
tom@rasa.com