This document discusses different types of conversational agents, including older chatbots like IBM Watson and modern chatbots using APIs. It also covers more advanced generative chatbots using LSTMs on large datasets as well as intelligent agents that can access information and perform actions. The document provides examples of how different agents work and considerations in building agents, from defining intents and slots to using neural networks, knowledge graphs, and APIs.

1. Conversational Agents
By Sam Witteveen

2. About me
• I currently spend most my time working on deep
learning that supports algorithmic trading
• Kids educational app company
• Deep Learning/AI by far the most interesting
thing worth working on

3. Conversational Agents
• Old style Chatbots - IBM, AIML, Eliza
• Simple Modern Chatbot - Facebook bots, API.ai
• Generative Chatbot - Twitter Bots
• Intelligent Agent - Google Assistant, Kate

4. Considerations
• Open domain vs. closed domain
• Pre-made vs dynamic responses
• Just chat vs trigger actions
• Access to what information / Web

5. Old Style Chatbots: AIML
• XML
• Eliza / Alice bot
• Pandorabots, Watson
• Regex
• Outdated and very fixed in the way that the work

6. AIML example

7. Simple Modern Chatbot

8. Simple Modern Chatbot
• Intents and slots/variables
“can i order a pizza”
“can i order a hamburger”
• intent = ‘order’
• item = ‘pizza/hamburger’ - slot variable

9. API.AI

10. Do it yourself
• Simple Classification
• Needs you to define classes first
• Convert text in pipeline
Logistic Regression Character CNNs
RNNsMLPs

11. Pipeline
• Tokenize
• Vectorize words
• Embeddings
• Word2vec / GloVE

12. Example Text Pipeline
TensorFlow
processing
Tokenization
get words
Speech Speech to Text
Text to speech
convert to
embedding
Beam Search
Response
Remove words
(sometimes)
convert to
One-Hot
Word2vec/GloVE
Back to Words

13. Simple Chatbots
• API.ai / Google does these kinds of bots pretty
simply
• They only tend to work on a very closed domain
• Good for simple tasks like ordering pizza,
booking flights etc

14. Generative Chatbots

15. Generative Chatbot
• Twitter bots made from speeches etc
• LSTMs (char-RNN) or Seq2Seq
• Large amount of training and data needed
• Beam search on the response
• Some token system for variables like names

16. Seq2Seq
<start>
Input
How
Input
are
Input
you
Input
<EOS>
Input
<Go>
Input
<Go>
output
I
output
am
output
fine
output
<EOS>
output

17. Generative Chatbot
• Great for fun
• Not ideal for business cases unless you have
amazing data
• Still a long way to go

18. Hybrid Agent
• Speech to Text
• Has the ability to access wider info
• Custom NER, POS for extracting variables (deep
learning )
• Graph system - for info and memory

19. Intent Ensemble
• Ensemble system for classifying the input
• LSTM Classification
• Character CNN Classification
• Bag of words

20. Intelligent Agent
Response
generation
First Layer of
Intent detection
Actions
Knowledge
graph
Speech STT
TTS
Check Layer of
Intent detection
Beam Search
Response
Entity
Extraction
Action
Extraction
Logic
unstructured
retrieval
translation
translation summarization

21. Knowledge bases
• Get info for returning to user
• Alternative: Freebase, DBpedia

22. Google Knowledge API

23. Translation text to query
• Similar to language translation
• “How old is Taylor Swift?”
• Entity: ’taylor swift’ , info.age

24. The really hard stuff
• Stateful Conversations (G-Assistant)
• Context
• Detecting different users (G-Home)

25. Tips
• Try to limit the domain of conversation
• Use Google Cloud for speech
• Make use of off the shelf wherever possible
• Use your own models for anything unique eg.
NER, vocab etc.
• Get as many example conversations as possible

26. Next Time
• 1. Neural Machine Translation - Seq2Seq with attention
• 2. CNNs for Text Class
• 3. Advanced Embeddings
• 4. Babl Q&A
• 5. Advanced Summarization
• 6. ParseySaurus - Google’s new NLP
• 7. Hotdog / Not Hotdog

27. The End
Contact
email: s@maclea.ai
twitter: sam_witteveen
https://github.com/samwit/