From Chatbots to
Augmented
Conversational Assistants:
An Experimental Study combining AI and Crowdsourcing
Frederic Jacquet
Chief Innovation Officer
Lao-dja S. Tchala
Data-Scientist
Agenda
§ Talk Overview: What can you expect
§ AI/ML to Augment Chatbots
capabilities
§ Methods & Human to improve bots
knowledge base
§ Perspective: Making an Inclusive
Chatbot
§ Wraping Up
• Talk Overview: What can you expect
• AI/ML to Augment Chatbots capabilities
• Methods & Human to improve bot knowledge base
• Perspectives: Making an Inclusive Chatbot
• Wrapping Up
"By 2020, the average
person will have more
conversations with bots
than with their spouse"
Gartner 2016
Gartner's Top 10 Strategic Predictions for 2017 and Beyond: Surviving the
Storm Winds of Digital Disruption
An Experimental Study combining AI
and Crowdsourcing
Identifying the
most relevant AI/ML
features that bring
advanced conversational
capabilities to Chatbots.
Bringing more
understanding and replying
abilities to Bots defining a
progressive methodology
where data are improved
based on crowdsourcing.
• Methods and Human
• AI/ML to Augment Bots
Take
aways from
this talk
• Talk Overview: What can you expect
• AI/ML to Augment Chatbots capabilities
• Methods & Human to improve bot knowledge base
• Perspectives: Making an Inclusive Chatbot
• Wrapping Up
AI : Computer Vision –
multiple choices
• What is it?
• Why?
• Our hypothesis
• Which technologies?
AI : Computer Vision –
multiple choices
• How it works?
AI : Computer Vision – multiple choices
Train a model with a set of images
1 Track a hand on the images
2 Calculate the center of the cercle around the hand
3 Coding an action depending on the center’s position
4
ACTION 1
ACTION 2
AI : Computer Vision –
multiple choices
• Demo
AI : Computer Vision –
multiple choise
• Future improvements
ML : Recommendation
algorithm
• What is it?
• Why?
• Our hypothesis
• Which technologies?
ML : Recommendation
algorithm
• How it works?
ML : Recommendation algorithm
• General
Qi
Qj
Qk
Full Text
Full Text
Full Text
Ai
Aj
Ak
Ai
Aj
Ak
Ai Qi
Aj Qj
Ak Qk
Qi
Qk
Key_word_i1 Key_word_i2
Key_word_k1 Key_word_k2
User send message
1 Bot answers
2 Mapping
3 Key words extraction
4 Model prediction
5
Qi
Qk
Ai
Aj
• Helpdesk use case : equipment's recommendation
Qj
Full Text
Full Text
Full Text
Qi
Qk
Manager
Equipment_i1 Equipment_i2
Equipment_k1 Equipment_k2
Authentificated users send messages
1 Key words extraction
4
Developer
Administrator
Qi
Qk
2 3 Rôle prediction
5
Administrator
Developer
Administrator
tips
Bot predicts the user’s rôle and proposes supplement equipments
6
Equipment_i1
Equipment_i2
You must be an Administrator
You might also need
Administrator
ML : Recommendation
algorithm
• Demo
ML : Recommendation
algorithm
• Future improvements
• Talk Overview: What can you expect
• AI/ML to Augment Chatbots capabilities
• Methods & Human to improve bot knowledge base
• Perspectives: Making an Inclusive Chatbot
• Wrapping Up
Humans’ contribution
• What is it?
• Why?
• Our hypothesis
• Which technologies?
Humans’ contribution
• How it works?
Humans’ contribution
Qi
Qj
Qk
Ai
Aj
Ak
Qi Ai
Qi1
Qi1
Qi1
Ai
Qi1
Qi1
Ai
Ai
Ai
Qi Ai
Qi1
Qi1
Qi1
Ai
Qi1
Qi1
Ai
Ai
Ai
Qi Ai
Qi1
Qi1
Ai
Qi1
Ai
Ai
Augmenting Data
1 Cleaning Data
2 Feeding the bot
3
Ask for clarification
Reformulating answers
Adaptive answers
Humans’ contribution
• Demo
Humans’ contribution
• Future improvements
Adaptive answers
• What is it?
• Why?
• Our hypothesis
• Which technologies?
Adaptive answers
• How it works?
Adaptive answers
Qi
Qj
Qk
Full Text
Full Text
Full Text
Ai
Aj
Ak
Ai
Aj
Ak
Ai
Aj
Ak
Ai
Ak
User send message
1 Bot answers
2 Training Data from user
3 Model predict which answers the user may like or not
4 Model training
5
Aj
Al
Ak
New answers’ prediction
True values Predcitons
Memory to use in next conversations
Al
Ak
6
Adaptive answers
• Demo
Adaptive answers
• Future improvements
Reformulating answers
• What is it?
• Why?
• Our hypothesis
• Which technologies?
Reformulating answers
• How it works?
Reformulating answers
Qj
User send message
1
Ai
Ai1
Ai2
Ai3
Ai Ai1 Ai2
Ai3
Bot chose a response and send it to the user
2 The already given responses are removed from the ones the bot will chose from
3
Qj
Ai
Ai1
Ai2
Ai3 OR
4
Reformulating answers
• Demo
Reformulating answers
• Future improvements
Ask for clarification
• What is it?
• Why?
• Our hypothesis
• Which technologies?
Ask for clarification
• How it works?
Ask for clarification
Qi
2 questions are similar
0
Qi
Qi1
Qi1
Qj
Qj1
Qj1
Qi
Qj
Qf
Qe
Qf
User ask one of them
1
Key_word_i1 Key_word_i2
Key_word_k1 Key_word_k2
Key_word_i1 Key_word_i2
Key_word_k1 Key_word_k2
Keys words are extracted from each of those questions
2 Keys words are used to ask clarification to the user
3
Do you mean
Key_word_i1 Key_word_i2
Key_word_k1 Key_word_k2
or
Key_word_i1 Key_word_i2
Key_word_k1 Key_word_k2
?
Ask for clarification
• Demo
Ask for clarification
• Future improvements
• Talk Overview: What can you expect
• AI/ML to Augment Chatbots capabilities
• Methods & Human to improve bot knowledge base
• Perspectives: Making an Inclusive Chatbot
• Wrapping Up
Inclusive Chatbots
Because texting or
talking may prove to
be temporarily or
permanently
challenging because of
morphology specifics,
trauma, mobility
deficiency, stroke,
diseases or any
deficiency.
• Talk Overview: What can you expect
• AI/ML to Augment Chatbots capabilities
• Methods & Human to improve bot knowledge base
• Perspectives: Making an Inclusive Chatbot
• Wrapping Up
Feedback / Questions
Do you have any questions?
Your feedback is important to us.
Don’t forget to rate and review the sessions.

From Chatbots to Augmented Conversational Assistants

  • 1.
    From Chatbots to Augmented ConversationalAssistants: An Experimental Study combining AI and Crowdsourcing Frederic Jacquet Chief Innovation Officer Lao-dja S. Tchala Data-Scientist
  • 2.
    Agenda § Talk Overview:What can you expect § AI/ML to Augment Chatbots capabilities § Methods & Human to improve bots knowledge base § Perspective: Making an Inclusive Chatbot § Wraping Up
  • 3.
    • Talk Overview:What can you expect • AI/ML to Augment Chatbots capabilities • Methods & Human to improve bot knowledge base • Perspectives: Making an Inclusive Chatbot • Wrapping Up
  • 4.
    "By 2020, theaverage person will have more conversations with bots than with their spouse" Gartner 2016 Gartner's Top 10 Strategic Predictions for 2017 and Beyond: Surviving the Storm Winds of Digital Disruption
  • 5.
    An Experimental Studycombining AI and Crowdsourcing Identifying the most relevant AI/ML features that bring advanced conversational capabilities to Chatbots. Bringing more understanding and replying abilities to Bots defining a progressive methodology where data are improved based on crowdsourcing. • Methods and Human • AI/ML to Augment Bots
  • 6.
  • 7.
    • Talk Overview:What can you expect • AI/ML to Augment Chatbots capabilities • Methods & Human to improve bot knowledge base • Perspectives: Making an Inclusive Chatbot • Wrapping Up
  • 8.
    AI : ComputerVision – multiple choices • What is it? • Why? • Our hypothesis • Which technologies?
  • 9.
    AI : ComputerVision – multiple choices • How it works?
  • 10.
    AI : ComputerVision – multiple choices Train a model with a set of images 1 Track a hand on the images 2 Calculate the center of the cercle around the hand 3 Coding an action depending on the center’s position 4 ACTION 1 ACTION 2
  • 11.
    AI : ComputerVision – multiple choices • Demo
  • 13.
    AI : ComputerVision – multiple choise • Future improvements
  • 14.
    ML : Recommendation algorithm •What is it? • Why? • Our hypothesis • Which technologies?
  • 15.
  • 16.
    ML : Recommendationalgorithm • General Qi Qj Qk Full Text Full Text Full Text Ai Aj Ak Ai Aj Ak Ai Qi Aj Qj Ak Qk Qi Qk Key_word_i1 Key_word_i2 Key_word_k1 Key_word_k2 User send message 1 Bot answers 2 Mapping 3 Key words extraction 4 Model prediction 5 Qi Qk Ai Aj • Helpdesk use case : equipment's recommendation Qj Full Text Full Text Full Text Qi Qk Manager Equipment_i1 Equipment_i2 Equipment_k1 Equipment_k2 Authentificated users send messages 1 Key words extraction 4 Developer Administrator Qi Qk 2 3 Rôle prediction 5 Administrator Developer Administrator tips Bot predicts the user’s rôle and proposes supplement equipments 6 Equipment_i1 Equipment_i2 You must be an Administrator You might also need Administrator
  • 17.
  • 19.
  • 20.
    • Talk Overview:What can you expect • AI/ML to Augment Chatbots capabilities • Methods & Human to improve bot knowledge base • Perspectives: Making an Inclusive Chatbot • Wrapping Up
  • 21.
    Humans’ contribution • Whatis it? • Why? • Our hypothesis • Which technologies?
  • 22.
  • 23.
    Humans’ contribution Qi Qj Qk Ai Aj Ak Qi Ai Qi1 Qi1 Qi1 Ai Qi1 Qi1 Ai Ai Ai QiAi Qi1 Qi1 Qi1 Ai Qi1 Qi1 Ai Ai Ai Qi Ai Qi1 Qi1 Ai Qi1 Ai Ai Augmenting Data 1 Cleaning Data 2 Feeding the bot 3 Ask for clarification Reformulating answers Adaptive answers
  • 24.
  • 27.
  • 28.
    Adaptive answers • Whatis it? • Why? • Our hypothesis • Which technologies?
  • 29.
  • 30.
    Adaptive answers Qi Qj Qk Full Text FullText Full Text Ai Aj Ak Ai Aj Ak Ai Aj Ak Ai Ak User send message 1 Bot answers 2 Training Data from user 3 Model predict which answers the user may like or not 4 Model training 5 Aj Al Ak New answers’ prediction True values Predcitons Memory to use in next conversations Al Ak 6
  • 31.
  • 33.
  • 34.
    Reformulating answers • Whatis it? • Why? • Our hypothesis • Which technologies?
  • 35.
  • 36.
    Reformulating answers Qj User sendmessage 1 Ai Ai1 Ai2 Ai3 Ai Ai1 Ai2 Ai3 Bot chose a response and send it to the user 2 The already given responses are removed from the ones the bot will chose from 3 Qj Ai Ai1 Ai2 Ai3 OR 4
  • 37.
  • 39.
  • 40.
    Ask for clarification •What is it? • Why? • Our hypothesis • Which technologies?
  • 41.
  • 42.
    Ask for clarification Qi 2questions are similar 0 Qi Qi1 Qi1 Qj Qj1 Qj1 Qi Qj Qf Qe Qf User ask one of them 1 Key_word_i1 Key_word_i2 Key_word_k1 Key_word_k2 Key_word_i1 Key_word_i2 Key_word_k1 Key_word_k2 Keys words are extracted from each of those questions 2 Keys words are used to ask clarification to the user 3 Do you mean Key_word_i1 Key_word_i2 Key_word_k1 Key_word_k2 or Key_word_i1 Key_word_i2 Key_word_k1 Key_word_k2 ?
  • 43.
  • 45.
    Ask for clarification •Future improvements
  • 46.
    • Talk Overview:What can you expect • AI/ML to Augment Chatbots capabilities • Methods & Human to improve bot knowledge base • Perspectives: Making an Inclusive Chatbot • Wrapping Up
  • 47.
    Inclusive Chatbots Because textingor talking may prove to be temporarily or permanently challenging because of morphology specifics, trauma, mobility deficiency, stroke, diseases or any deficiency.
  • 48.
    • Talk Overview:What can you expect • AI/ML to Augment Chatbots capabilities • Methods & Human to improve bot knowledge base • Perspectives: Making an Inclusive Chatbot • Wrapping Up
  • 49.
    Feedback / Questions Doyou have any questions? Your feedback is important to us. Don’t forget to rate and review the sessions.