Slides from the workshop 'Building AI assistants that scale using machine learning and OSS tools' ad O'Reilly AI conference by Justina Petraityte, Developer Advocate at Rasa

1. Building AI assistants that scale using
machine learning and OSS tools
Justina Petraityte, Developer Advocate @ Rasa
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

2. What are we focusing on during this workshop
Introduction
Goal:
Build a ML-powered AI assistant
Roadmap:
1. Natural Language Understanding
i. Introduction and theory
ii. Coding
2. Dialogue Handling
i. Introduction and theory
ii. Coding
3. Closing the feedback loop
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

3. Setup
Introduction
1. Jupyter notebook in python 3.6
2. Download:
Repository: https://github.com/RasaHQ/rasa-workshop
3. Install:
i. Dependencies specified in a requirements.txt file of the repo
ii. MongoDB (optional step)
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

4. Key challenges for enterprises to build in-house
conversational AI:
1. ML experts → Hard to find and retain
2. ML infrastructure → Hard engineering problem
3. Integration into apps / backend systems →
Cumbersome
Developing AI assistant to handle text and voice conversations is difficult
PROBLEM
Internal Customer-facing
Emails
Messages
Calls
Apps & websites
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

5. INTRODUCTION: THE RASA STACK
The Rasa Stack is a set of open source machine learning
tools for developers for conversational AI:
● Core: framework for machine learning-based,
contextual decision making
● NLU: a library for natural language
understanding with intent classification and
entity extraction
The Rasa Stack: Open source tools for developers to automate
contextual text and voice conversations
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

6. Open source and free
No risk of vendor lock-in. No “AI
fairy dust” - can see all ML
components.
Why do developers love Rasa today?
WHY RASA
Empowers developers
Get started with very little ML
knowledge, look under the hood
and learn more about ML.
Customizable
Tune models and get higher
accuracy on your data set. Integrate
into your existing infrastructure.
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

7. Rasa’s technology can understand natural language and decide about the next best
action based on the context of the conversation using Machine Learning
Backend, CRM,
database, API, etc.
INTRODUCTION: THE RASA STACK
ML-based
Dialogue
Management
“The Brain”
Input Modules
“The Ears”
Natural Language
Understanding
Output
Modules
“The Mouth”
Response Generator
Connector
Modules
Live Chat, App,
Messenger, Phone
SMS, etc.
“I want to change
my address”
(User Request via
text or voice)
“What’s your new
address?”
(AI response via
text or voice)
User Backend
Data Control
Companies keep
complete control of
their data and customer
interaction
Via communication channel:
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

8. Developing AI assistants with Rasa
Stack
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

9. Rasa NLU: Natural Language Understanding
Under the Hood
Goal: extract the structured data from unstructured user inputs
I have a new address, it’s
709 King St, San Francisco
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

10. Rasa NLU: Natural Language Understanding
Under the Hood
greet
goodbye
thank_you
request_weather
confirm
What’s the weather
like tomorrow?
Bag
of words
SVM
rasa.com/docs
Data:
intent: weather
- What’s the weather like tomorrow?
- Tell me the weather forecast
intent: greet
- Hello
- hi
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

11. Understanding multi-intents
Under the Hood
Message Intent
Sounds great! Can you also tell me what is the price? affirm+ask_price
What about tomorrow? I feel too tired today. inform+chitchat
Yes, book it. Also, please book me a taxi. affirm+book_taxi
A user input can have more than one intention. Enabling the assistant to understand them leads to more
natural conversations.
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

12. Supervised Word Vectors from scratch
Text Classification beyond word2vec
References:
WSABIE (Weston, Bengio, Usunier)
StarSpace (Wu et al)
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

13. Rasa NLU: Entity Extraction
Under the Hood
Natural
Language
Understanding
What’s the
weather like
tomorrow?
Example Entity Extraction Pipeline
”What’s the weather like tomorrow?” { “date”: “tomorrow” }
Tokenizer
Named Entity
Recognition
Entity Extraction
Feature extractor
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

14. Rasa NLU: Entity Extraction
Under the Hood
[low, title, upper]
word afterbefore
CRF
Entities:
date: tomorrow
location: Berlin
low,
title,
upper
low,
title,
upper
bias,
low,
prefix5,
…,
suffix3,
digit
What’s the weather like tomorrow in Berlin?
rasa.com/docs
date location
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

15. Handling synonyms
Challenges
I moved to New York City.
city: nyc
I moved to NYC.
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

16. Identifying out-of-scope inputs
Challenges
greet
goodbye
thank_you
book_hotel
inform
I want some pizza
Intent
classifier
The best way to handle out-of-scope inputs is to actually teach your bot to identify them.
out-of-scope
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

17. Under the hood
Dialogue Management
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

18. Rasa Core learns to Converse from real conversational data
Scalable Dialogue
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

19. Rasa Core: Dialogue Handling
Under The Hood
“What’s the weather
like tomorrow?”
Intent
Classification
Entity
Extraction
Similar to LSTM-dialogue prediction paper: https://arxiv.org/abs/1606.01269
next_best_action:
action_weather, 87%
utter_greet, 5%
utter_goodbye, 4%
Entities:
date: tomorrow, 75%
Intent:
weather: 98%
“It will be
sunny and
20°C.”
Response
generator
Response
Slot:
temp: 25 deg., 85%
Action
type?
API Call
Recurrent
NN
previous_states:
max_history: 3
current_state
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

20. Let’s zoom in on a action server
Under The Hood
“What’s the weather
like tomorrow?”
next_best_action:
action_weather, 87%
utter_greet, 5%
utter_goodbye, 4%
Post request
Run a custom action
SDK
“It will be
sunny and
20°C.”
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

21. Handling out of scope inputs
Challenges
Book me a hotel in
NYC.
Sure! For how many
people should I book?
It’s a beautiful day
outside, isn’t it?
I didn’t quite get you...
For how many people
should I book?
You should always aim to identify out-of-scope inputs and enable your bot to take charge of the conversation.
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

22. Closing the feedback loop
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

23. Rasa Core: Dialogue Training
Under The Hood
Issue: How to get started? Interactive Learning→
What’s the weather
like tomorrow?
How did you like it?
Correct wrong
behaviour
Retrain model
It will be sunny and
20°C.
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

24. Key takeaways and tips
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

25. Key takeaways
Let the real users help you improve your assistant.
Use the user interaction metadata to get started.
Don’t be afraid to customize the tools you use.
Start small and make sure your assistant nails the happy path first.
NLP is far from being a solved problem.
rasa.com/docs
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

26. Open Source
+
Community
+
Applied Research
Join the Rasa open source community!
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.

27. Justina Petraityte
Developer Advocate
juste@rasa.com
@juste_petr
Get in touch!
Confidential and proprietary.
Any use of this material without specific permission of Rasa is strictly prohibited.