This document discusses the history and development of chatbots from early rule-based systems to modern deep learning approaches. It defines chatbots as programs that simulate human conversation and outlines some of the early systems from the 1960s to 2000s that were rule-based. It then introduces natural language processing and machine learning, focusing on neural network models and word embeddings. Finally, it provides a brief overview of how to build a neural conversational model using a sequence to sequence approach and train it on a dialog corpus, noting some caveats like needing large amounts of data and compute resources.

1. Chatbots and Deep Learning:
From theory to practice

2. Andherson C. Maeda
Developer Consultant @ thoughtworks
$ whoami

3. Chatbots

4. Definition
What's chatbot means?
1

5. “
Program that simulate human
behavior though questions and
answers.

6. Can machines think?
◉ Computing Machinery and Intelligence
(TURING, 1950)
◉ Imitation game

7. In 50 years
An interrogator will not have more than 70%
chance to make the right identification after 5
minutes conversation

8. Chatbots timeline2

9. Chatbots timeline
60's: Eliza
Simulation of
rogerian
psychologist. Use
pieces of input to
build output phrase
(Wallace)
90's: Julia
TinyMUD bot. Based
on rules recognition
and neural networks
(Mauldin)
2000: Alice
Based on rules also,
but with more
features. Inspired by
XML: AIML (Wallace)
Natural Language Processing
Neural Networks

10. What's that?
Natural Language
Processing3

11. “
Field of Computer Science and
Linguistics concerned with
interactions between computers
and human natural languages

12. NLP: example

13. How can I build my own bot?
Chatbots toolbox4

14. Alice: AIML
<category>
<pattern>WHAT ARE YOU</pattern>
<template>
<think><set name="topic">Me</set></think>
I am the latest result in artificial intelligence,
which can reproduce the capabilities of the
human brain
with greater speed and accuracy.
</template>
</category>

15. Platforms, APIs and Libs

16. or not...
Caveats5

17. Caveats (or not)
◉ Handcrafted rules
◉ Unknown questions will generate default
answers

18. Machine Learning
Deep, embeddings, etc.

19. Definition
What's Machine Learning?
6

20. “
"A computer program is said to learn from experience E with respect
to some class of tasks T and performance measure P if its
performance at tasks in T, as measured by P, improves with
experience E" (Mitchel, 1997)

21. Machine Learning Algorithms

22. A Neural Conversational Model

23. Word Embeddings
A Neural Conversational Model
Seq2Seq

24. Building...
Building my own neural bot <3
7

25. Place your screenshot here
Dialog corpus selection
Find a large dialog
corpus

26. NLP Tasks
Apply NLP tasks to clean
your dataset like:
- Lowercase
- Tokenization

27. Word Embeddings
Build the Word2Vec
embedding from your
dialog corpus
- 40k most frequent
words could be
good enough

28. Train Seq2Seq Model
Train your model with a
dialog corpus.
- Use some testing
technique
(70%/30%, cross
validation, etc)
- GPU!

29. Demo (portuguese version)
Sequence to Sequence trained with StarWars
corpus:
◉ https://webot.personaltek.com

30. Seq2Seq StarWars Bot

31. or not...
Caveats8

32. Caveats (or not)
◉ Handcrafted rules
◉ Unknown questions will generate default
answers answers!
○ Weird answers, but answers...
◉ Long training time…
◉ Lot of resources needed
◉ Hard to test the model = (

33. Any questions ?
You can find me at
◉ @maedabr
◉ maeda.br@gmail.com
◉ amaeda@thoughtworks.com
Thanks!