Successfully reported this slideshow.
Your SlideShare is downloading. ×

The Software Challenges of Building Smart Chatbots - ICSE'21

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 44 Ad

The Software Challenges of Building Smart Chatbots - ICSE'21

Download to read offline

Chatbots are popular solutions assisting humans in multiple fields, such as customer support or e-learning. However, building such applications has become a complex task requiring a high-level of expertise in a variety of technical domains. Chatbots need to integrate (AI-based) NLU components, but also connect to internal/external services, deploy on various platforms, etc.

The briefing will first cover the current landscape of chatbot frameworks. Then, we’ll get our hands dirty and create a few bots of increasing difficulty playing with aspects like entity recognition, sentiment analysis, event processing, or testing. By the end of the session, attendees will have all the keys to understand the main steps and obstacles to building a good chatbot.

Chatbots are popular solutions assisting humans in multiple fields, such as customer support or e-learning. However, building such applications has become a complex task requiring a high-level of expertise in a variety of technical domains. Chatbots need to integrate (AI-based) NLU components, but also connect to internal/external services, deploy on various platforms, etc.

The briefing will first cover the current landscape of chatbot frameworks. Then, we’ll get our hands dirty and create a few bots of increasing difficulty playing with aspects like entity recognition, sentiment analysis, event processing, or testing. By the end of the session, attendees will have all the keys to understand the main steps and obstacles to building a good chatbot.

Advertisement
Advertisement

More Related Content

Slideshows for you (20)

Similar to The Software Challenges of Building Smart Chatbots - ICSE'21 (20)

Advertisement

More from Jordi Cabot (20)

Advertisement

The Software Challenges of Building Smart Chatbots - ICSE'21

  1. 1. The Software Challenges of Building Smart Chatbots – ICSE’21 TB @xatkit – xatkit.com
  2. 2. Context
  3. 3. Jordi Cabot - @JordiCabot Gwendal Daniel - g__daniel
  4. 4. Can you help us create a chatbot to help newcomers to write good bug reports on GitHub? Sure! It’s just a set of questions & answers, this will be pretty simple! Narrator It wasn’t. Once upon a time in 2018 (we were young and naïve)
  5. 5. We were asked to build a bot. We ended up building a bot platform But at least we’ve learnt some things along the way that today we want to share with you 3 times so far!
  6. 6. The plan for today • THEORY (30’) • HANDS-ON PRACTICE (1H45’) • You can watch • Better if you watch and try (installing included!) • Ask questions on the chat at anytime, we’ll keep an eye on them
  7. 7. Why you all need to have some bot building notions
  8. 8. And more predictions: - Business Insider experts predict that by 2022, banks can automate up to 90% of their customer interaction using chatbots - By 2024, consumer retail spend via chatbots worldwide will reach $142 billion—up from just $2.8 billion in 2019. Chatbot market is worth over 1 Billion USD
  9. 9. Chatbots <-> Software engineers • Software Engineers -> Chatbot – Chatbots are a new type of software artefact that we’ll need to consider in our dev projects (Multi-experience dev) – Plenty of research opportunities to uncover • Chatbot -> Software Engineering – Bots are more and more playing a supportive role in many SE tasks (e.g. in GH) so better to know how they work and their limitations
  10. 10. Basic concepts
  11. 11. What is exactly a Chatbot? • Bot are software components • Chatbot = Bot + Chat interface • Voicebot = Bot + Voice Interface Voicebot Chatbot Bot
  12. 12. VoiceBot • Alexa, Google Assistant ,… • They hide a chatbot inside 1. Speech-to-text 2. Process the text as a chatbot 3. Text-to-speech to reply
  13. 13. Chatbot vs LiveChat • LiveChat: a person is behind • They can be combined – Bot  Person: when the bot doesn’t know how to answer (or the visitor is specially “interesting”) – Person  Bot: when no support person is available or the client is not worth
  14. 14. Bots and NLP/NLU • At the core, a NLU/NLP engine with two key missions – Classify the user utterance (i.e. input text) in one of the intents (set of possible questions/requests) defined in the bot – Identify the parameters (called « entities ») from the utterance
  15. 15. We ignore open-domain bots (e.g. LaMDA) • They can chat about everything but – A complete black-box – ``Useless’’ for a specific business – You could use them (or QA systems) as a fallback
  16. 16. Why chatbot development is so challenging?
  17. 17. Chatbots combine all the complexity of bots with the complexity of NL (including typos, trolls, sentiment analysis…)
  18. 18. Many pieces to assemble Input/Output Messaging channels External Platforms NLU Engine Events!
  19. 19. Chatbots are complex software systems Conversation Logic Text Processing External Services Messaging Platforms Deployment Evolution Maintenance Tests
  20. 20. Jordi Cabot¹, Loli Burgueño², Robert Clarisó², Gwendal Daniel², Jorge Perianez-Pascual³,and Roberto Rodriguez-Echeverria³ ¹ICREA, Barcelona, Spain ²Open University of Catalonia, Barcelona, Spain ³University of Extremadura, Cáceres, Spain Testing challenges for NLP-intensive bots BotSE ‘21
  21. 21. NLP for chatbots in 5 min
  22. 22. NLP in a chatbot implies Parameters (entities) I would like to order a medium pepperoni pizza Intent Input text (utterance) • Understand what the user is trying to ask the bot • Identify and gather the data in the request that the bot needs to construct the response
  23. 23. NLU approaches • Regular expressions • Require an almost exact match. Not used • Grammar-based approaches • Useful for small domains with a very specific vocabulary and constructs • E.g. Stanford Parser • Neural Networks - Multiclass classifiers 1. You provide some training sentences for each intent (the "classes" ). 2. The classifier learns from these sentences 3. It assigns any new input text to one of the classes (with %confidence)
  24. 24. Example: bot that greets you, informs about the schedule and lets you buy a pizza Greetings Hi Good morning Good afternoon Hello Morning! I’d like to buy a pizza I want a pizza Can I order a pizza? Buy Are you open? Is it open now? What are your opening hours? At what time do you start? Schedule If a user says, “I would like to order a pizza” what is the closest intent? Not that easy when intents start to overlap (e.g. offering a job vs looking for a job: “can I offer you a job?” vs “do you have any job offer”)
  25. 25. Entity recognition • We need to understand the intent but also identify the entities • Much easier if PizzaType is an enumeration ({ Pepperoni, Veggie, Hawaian ,…}) instead of the any string ChooseYourPizza I’d like a PizzaType pizza I’m taking a PizzaType pizza I think I’m going for a PizzaType
  26. 26. Common tricks in NLU Engines • Tokenizers split the sentences into words • Stemmers identify the lemma of the word (e.g. to focus on the verb, not on the tense of the verb for matching purposes) • Synonyms and paraphrasing are an automatic way to expand the training sentences • Entity augmentation also generalize training sentences • E.g. What’s the weather in Barcelona -> What’s the weather in @city (the intent will match no matter what city the user asks for, no need to train the classifier with all the cities of the world as it will already know the list of cities itself). • Augmentation is possible for a large number of data types and common concepts (measures, countries, dates,…) https://medium.com/@jseijas/how-to-build-your-own-nlp-for-chatbots-9b4c08eb03a9
  27. 27. Beyond intent classification • Great bots go beyond providing the right answer, e.g. they could show some empathy • Several language models (for toxicity detection, for sentiment analysis, language identification, emoji support…) can be easily integrated in bots
  28. 28. PRO tip: Monitor your bots • You will fail to predict: • What users want to know • How they express their intentions • Careful with self-learning • Filter out what the bot needs to add as training sentences to avoid strange derivations
  29. 29. What bot building tool should I use?
  30. 30. >100 bot platforms • Rasa, BotPress, Chatfuel, Inbenta, Botsify, Flow XO, Xatkit… • And plenty NLP engines: DialogFlow, Lex, LUIS, Watson, NLP.js
  31. 31. It all depends • What are your priorities – Open source? – Free? – Programming language of the bot? – Natural languages (e.g. Catalan) – Availability of connectors – On-premises option? – Plain textual bots or fully functional ones? – Low-code/ no-code or tech skills? – Possibility to migrate from one NLP engine to another? – Monitoring dashboard / reports ? – …
  32. 32. Building bots in Xatkit
  33. 33. Xatkit’s philosophy: Do not reinvent the wheel Other solutions Plenty of great NLU engines and libraries, let’s not waste our time building a new one but take the best of breed and focus on features to help you create better bots Xatkit is a chatbot orchestration platform
  34. 34. Another reason to avoid strong coupling to specific NLU/NLP libraries
  35. 35. Xatkit architecture Chatbot Designer Intent Recognition NLU Providers (platform-specific) Platform DSL / Libraries Intent DSL Xatkit DSL Chatbot User Instant Messaging Platforms Xatkit Runtime Execution DSL uses uses Platform-independent chatbot definition External Services Deployment Configuration Platform Designer
  36. 36. Xatkit DSL • Bots are created with the Xatkit DSL offering a chatbot- specific syntax for creating: – Intents the bot needs to match – The behavior to execute in response to the matched intents • Our DSL aims to combine the benefits of: – Having a chatbot-specific DSL that facilitates the creation of chatbot applications by providing higher-level chatbot modeling primitives – The full power of a general programming language In this scenario, Internal DSLs are the way to go (Martin Fowler)
  37. 37. Our DSL is implemented as a Java Fluent API • Create bots using your preferred Java editor – Benefit from all existing Java tooling when developing bots (e.g. debuggers) – Reuse any Java library for complex bot behaviors – Intuitive Fluent Interfaces to help you create advanced conversations – Based on state machine semantics to build any type of bot (there is also a no-code interface, e.g. using excel, for non-tech people)
  38. 38. I18n: Xatkit helps you build bots ready to be localized
  39. 39. Easy deployments A Xatkit bot can be packaged as a single jar with no external dependencies. Just execute the jar to get the bot up and running Via a simple configuration file you can connect your bot with external services, e.g. NLP Engines and communication channels
  40. 40. Xatkit is available on GitHub
  41. 41. To know more: xatkit.com / info@xatkit.com Xatkit is your best bet to sell more doing less @xatkit – xatkit.com
  42. 42. Let’s play!!!

Editor's Notes

  • We’re still young 
  • We believe we had good arguments to decide that the world needed another chatbot development platform
  • This is really designed to be a hands-on tutorial. We’ll start by installing Xatkit
  • AI and Virtual assistants are everywhere
  • And the chatbot market is growing non-stop
  • Typically, this voice interface ends up being a speech-to-text component so a voicebot is a chatbot plus this speech-to-text and text-to-speech layer on top


  • Google on Tuesday unveiled its latest language breakthrough, the development of a conversational language model called LaMDA (Language Model for Dialogue Applications). Google discussed the new model during the keynote address at its I/O conference. 
    Like other recently-developed language models, including BERT and GPT-3, LaMDA is built on Transformer, the neural network architecture that Google Research invented and open-sourced in 2017. 
    However, unlike other language models, Google's LaMDA was trained on dialogue, teaching it how to engage in free-flowing conversations. This training taught LaMDA to deliver responses that not only make sense given the context but are specific. 
  • By complexity of NL, I mean that beyond intent classification, you’ve plenty of additional issues to deal with in NL: trolls, typos, sentiment analysis
  • When you start looking at what you need to build this bot you realize there are many more components involved

    Event webhooks
  • We´ll assume you’re familiar with other complications such as APIs, webhooks,…
  • The network uses synonyms and stemming and lemmatization to classify the intent

    Sometimes we may need to help the Engine by putting close questions in different paths
  • Hay mucho “graciosillo” suelto, no queremos que el bot aprenda palaborats
  • La diferencia entre un motor NLP y una plataforma es que el motor NLP está especializado en clasificar el texto del usuario, no tanto en lo qué el bot deber hacer con ella.

×