Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Serverless and Chatbots: A Match Made in the Cloud

146 views

Published on

Video and slides synchronized, mp3 and slide download available at URL https://bit.ly/2FeA40M.

Gillian Armstrong uses practical examples from their chatbot and shares some of the lessons they learned which help starting a chatbot. Filmed at qconsf.com.

Gillian Armstrong works as a Technical Lead in the Cognitive Technologies team in Liberty IT. Her team is focused on thinking about big problems, and working out how to solve them using innovative technology in interesting new ways. At the moment she is working on Artificial Intelligence, with a particular focus on Conversational AI design and development.

Published in: Technology
  • Be the first to comment

Serverless and Chatbots: A Match Made in the Cloud

  1. 1. Chatbots and Serverless: A match made in the Cloud Gillian Armstrong @virtualgill QConSF
  2. 2. InfoQ.com: News & Community Site • Over 1,000,000 software developers, architects and CTOs read the site world- wide every month • 250,000 senior developers subscribe to our weekly newsletter • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • 2 dedicated podcast channels: The InfoQ Podcast, with a focus on Architecture and The Engineering Culture Podcast, with a focus on building • 96 deep dives on innovative topics packed as downloadable emags and minibooks • Over 40 new content items per week Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ serverless-chatbots
  3. 3. Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide Presented at QCon San Francisco www.qconsf.com
  4. 4. Gillian Armstrong @virtualgill Technologist Cognitive Technologies Liberty IT Gillian Armstrong @virtualgill
  5. 5. 2016 2020 7% SOURCE: US Census, Gartner, Edison Research, Voicebot.ai 75% Smart Speaker US Household Ownership Forecast
  6. 6. 2016 2020 7% SOURCE: US Census, Gartner, Edison Research, Voicebot.ai 75% Smart Speaker US Household Ownership Forecast The world is changing fast
  7. 7. So we need to be able to learn fastThe world is changing fast Gillian Armstrong @virtualgill
  8. 8. Change Fast Learn Fast use what we learn to so we can keep on learning Gillian Armstrong @virtualgill
  9. 9. Change Fast Learn Fast Both Chatbots and Serverless let us and Gillian Armstrong @virtualgill
  10. 10. What is a Chatbot??? What do you mean byServerless??? ??? ??? ??? Gillian Armstrong @virtualgill
  11. 11. Gillian Armstrong @virtualgill
  12. 12. Gillian Armstrong @virtualgill
  13. 13. A Serverless solution is one that costs you nothing to run if nobody is using it (excluding data storage cost) For the purposes of this presentation... - Paul Johnston Gillian Armstrong @virtualgill
  14. 14. For the purposes of this presentation... A Chatbot is something you can interact with conversationally using natural language - me Gillian Armstrong @virtualgill
  15. 15. Gillian Armstrong @virtualgill
  16. 16. CHANNEL/DEVICE NLU NLP INTENTS ENTITIES I n t e r a c t i o n M o d e l Fulfillment MachineLearning ASR TTS CUX/VUX A very quick overview of a Chatbot using Conversational AI Gillian Armstrong @virtualgill
  17. 17. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment MachineLearning ASR TTS CUX/VUX Utterance (“what the user said”) Automatic Speech Recognition (Speech to Text) Text to Speech Mapping of Utterance to Intent (“what the user meant”) Extraction of Entities (variables like time, place, number, etc) Oh hey, there’s still UI design Insert cool serverless architecture here Conversational / Voice User Experience (“how we sound and the words we say”) Conversational User Interface ENTITIES Intent maps to Fulfillment (“what we need to do”) Gillian Armstrong @virtualgill
  18. 18. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment MachineLearning ASR TTS CUX/VUX Design it so a Human can understand it Train your Bot to understand Humans Write code to make it do something! 1 2 3 What we need to do… ENTITIES Gillian Armstrong @virtualgill
  19. 19. CHANNEL/DEVICE NLU NLP INTENTS I n t e r a c t i o n M o d e l Fulfillment MachineLearning ASR TTS CUX/VUX Design it so a Human can understand it Train your Bot to understand Humans Write code to make it do something! 1 2 3 What we need to do… Often starting point and focus Hardcoded and then panic-written after that well-received demo Forgotten ENTITIES Gillian Armstrong @virtualgill
  20. 20. Design it so a Human can understand it 1 Gillian Armstrong @virtualgill
  21. 21. In a chatbot, Conversation is the Interface Gillian Armstrong @virtualgill
  22. 22. Source: http://hosanna1.com/ Not a CSS and HTML problem! Design Matters! Gillian Armstrong @virtualgill
  23. 23. Up to now we’ve kept designing based on what came before Gillian Armstrong @virtualgill
  24. 24. We’ve carried the concept of menus and commands into our graphical user interfaces Gillian Armstrong @virtualgill
  25. 25. We base our web forms on the paper forms that came before Gillian Armstrong @virtualgill
  26. 26. Gillian Armstrong @virtualgill
  27. 27. Model your design on a human, not a website People are the original conversational interface Gillian Armstrong @virtualgill
  28. 28. When you start with thinking about trying out a design you may discover that a chatbot isn’t even the right thing to build… Gillian Armstrong @virtualgill
  29. 29. But assuming it is… Gillian Armstrong @virtualgill
  30. 30. Be Clear Be Helpful Be Nice • Proactively offer help • Get people back on track • Repetition is annoying • Repetition is annoying • Take the blame – in a conversation the human is never wrong • Chatbots aren't smarter than a human • Be understanding if people don’t want to talk to you – offer alternatives. 1 minute Conversational Design overview • Keep your personality consistent • Keep answers short and to the point • Set expectations – what can you do • Reflect back key information and confirm before critical actions. Gillian Armstrong @virtualgill
  31. 31. The Web is about Telling the user what to do Conversational UI is about Listening to what the user wants to do Gillian Armstrong @virtualgill
  32. 32. Train your Bot to understand Humans 2 Gillian Armstrong @virtualgill
  33. 33. Amazon Lex Console What the Web Tutorials told me building a chatbot looked like… Gillian Armstrong @virtualgill
  34. 34. BOOK MEETING IntentUtterance SAY HELLO GET HELP Fulfillment Gillian Armstrong @virtualgill
  35. 35. IntentUtterance As simple providing a set of sample utterances! Book Meeting at <TIME> on <DATE> Set up Appointment Create event on <DATE> Set up Meeting Book me a meeting at <TIME> … Hello Hi there Hey Good Morning … Help What can you do Help me I need help … BOOKMEETINGSAYHELLOGETHELP Gillian Armstrong @virtualgill
  36. 36. Amazon Lex Console SET OF INTENTS Represents the functionality the bot has SAMPLE UTTERANCES Examples of what a user might say to trigger this Intent. This is training data - not a regex match. Gillian Armstrong @virtualgill
  37. 37. Amazon Lex Console SLOTS (ENTITIES) are defined showing type (place, name, time, custom, etc.) If the user does not supply this data the bot knows to prompt for it. The sample utterances show where to expect the data to be supplied in an utterance Gillian Armstrong @virtualgill
  38. 38. FULFILLMENT location defined. Intent and Slot matches will be sent to this lambda or returned directly to client. Amazon Lex Console Gillian Armstrong @virtualgill
  39. 39. Amazon Lex Console Publish to make the functionality live to users. Build to create the Interaction Model Gillian Armstrong @virtualgill
  40. 40. Amazon Lex Console Built-in Test Console allows you to test directly here after building, but before publishing. Gillian Armstrong @virtualgill
  41. 41. Amazon Lex Console Easy to connect your bot to popular channels through console. Gillian Armstrong @virtualgill
  42. 42. Amazon Lex Console Chatbot Done…? Gillian Armstrong @virtualgill
  43. 43. Sorry, I’m just a baby bot. I’m still learning. Gillian Armstrong @virtualgill
  44. 44. Sorry, I’m just a baby bot. I’m still learning. are you? Gillian Armstrong @virtualgill
  45. 45. IntentUtterance As simple providing a set of sample utterances! Book Meeting at <TIME> on <DATE> Set up Appointment Create event on <DATE> Set up Meeting Book me a meeting at <TIME> … Hello Hi there Hey Good Morning … Help What can you do Help me I need help … BOOKMEETINGSAYHELLOGETHELP Gillian Armstrong @virtualgill
  46. 46. IntentUtterance As complex as providing a set of sample utterances! Book Meeting at <TIME> on <DATE> Set up Appointment Create event on <DATE> Set up Meeting Book me a meeting at <TIME> … Hello Hi there Hey Good Morning … Help What can you do Help me I need help … BOOKMEETINGSAYHELLOGETHELP Model your design on a human Get your sample utterances from your users… you wouldn’t make up other machine learning training data! Gillian Armstrong @virtualgill
  47. 47. Track metrics and review conversation to improve your Bot Interaction Model. Learn Fast Sorry, I’m just a baby bot. I’m still learning. Change Fast Gillian Armstrong @virtualgill
  48. 48. Look for Vocabulary used Should we add new synonyms Look for the Utterance Structure Should we add new training data Look for Interaction Patterns Can we improve the conversation flow Learn Fast Change Fast Reviewing Conversation Transcripts… Gillian Armstrong @virtualgill
  49. 49. Chatbots aren’t magic (although I wish they were) Gillian Armstrong @virtualgill
  50. 50. Write code to make it do something! 3 Gillian Armstrong @virtualgill
  51. 51. Pop a lambda here – and you’re done! Gillian Armstrong @virtualgill
  52. 52. Gillian Armstrong @virtualgill
  53. 53. Gillian Armstrong @virtualgill
  54. 54. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING FULFILLMENT… Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
  55. 55. Number of conversations Length of conversations Functionality being used Number of Users What people want us to add Where we are getting it right Where we are getting it wrong Examples of our Conversational Analytics Gillian Armstrong @virtualgill
  56. 56. Observability is critical This lets you quickly learn where there might be problems with your system. Gillian Armstrong @virtualgill
  57. 57. Why Serverless?? Gillian Armstrong @virtualgill
  58. 58. Lower Cost lets us Change Fast Learn Fast Off-loading Maintenance lets us Gillian Armstrong @virtualgill
  59. 59. Instead of custom building and managing your own servers, db, etc… Gillian Armstrong @virtualgill
  60. 60. Use standard pieces – that someone else has built and manages - and put them together to get the system you want Gillian Armstrong @virtualgill
  61. 61. Serverless Services Gillian Armstrong @virtualgill
  62. 62. Lambdas are quick to develop, isolate small pieces of functionality and allow you to scale different parts of the conversation independently. Gillian Armstrong @virtualgill
  63. 63. Ensure that everything scales - or at least know your limits. API Lambda functions Amazon DynamoDB Gillian Armstrong @virtualgill
  64. 64. Serverless means you don’t pay if no-one talks to your bot… …and lets you scale seamlessly if everyone wants to talk to your bot! Gillian Armstrong @virtualgill
  65. 65. Cost Usage No usage, no cost Higher usage, higher cost Gillian Armstrong @virtualgill
  66. 66. Cost Usage Watch the free tier… Cost may not be linear Gillian Armstrong @virtualgill
  67. 67. Monolith Some notes for those with existing systems Gillian Armstrong @virtualgill
  68. 68. Monolith Microservices async! event driven! Gillian Armstrong @virtualgill
  69. 69. Monolith Microservices Function as a Service Gillian Armstrong @virtualgill
  70. 70. REWRITE is not a four-letter word Gillian Armstrong @virtualgill
  71. 71. Model your design on a human, not a website People are the original conversational interface Gillian Armstrong @virtualgill
  72. 72. A good human conversation is synchronous You wait for a response You expect a response immediately Gillian Armstrong @virtualgill
  73. 73. A Conversational Architecture needs to - Be fast! - Appear synchronous to the user (there should be a response) This is particularly critical for a voice-based bot Gillian Armstrong @virtualgill
  74. 74. Monolith Microservices Function as a Service Gillian Armstrong @virtualgill
  75. 75. Do not just lift and shift your microservices into FaaS Gillian Armstrong @virtualgill
  76. 76. invoke invoke invoke Combining the time to invoke and the time to ‘spin up’ can hit performance and add complexity to your system. Choose to ‘keep warm’ only as a last resort. Do not end up with a collection of distributed microservices Gillian Armstrong @virtualgill
  77. 77. invoke Shhhhhh….. It’s ok to have more than one function in your Function as a Service… Look for where you can pull functionality into reusable libraries instead Gillian Armstrong @virtualgill
  78. 78. Having said that… don’t just lift and shift your microservices into FaaS Gillian Armstrong @virtualgill
  79. 79. API API API APIs designed for webforms tend to be set up to collect sets of data page by page Gillian Armstrong @virtualgill
  80. 80. API A Conversation is non-linear – the user can give partial information, or information that would have been collected later on your webform. Gillian Armstrong @virtualgill
  81. 81. REWRITE is not a four-letter word Gillian Armstrong @virtualgill
  82. 82. Both Serverless and Chatbots require new ways of thinking about your architecture. Gillian Armstrong @virtualgill
  83. 83. CHANGE FAST LEARN FAST We get there iteratively by being able to and Gillian Armstrong @virtualgill
  84. 84. CHANGE FAST LEARN FAST Both Chatbots and Serverless let us and Gillian Armstrong @virtualgill
  85. 85. CHANGE FAST LEARN FAST Both Chatbots and Serverless let us and but Good Supporting Engineering Practices are needed to do both with confidence and at scale Gillian Armstrong @virtualgill
  86. 86. Amazon Lex Console What the Web Tutorials told me building a chatbot looked like… Gillian Armstrong @virtualgill
  87. 87. CODE HERE Fulfillment in Code Gillian Armstrong @virtualgill
  88. 88. CODE HERE CODE HERE Bot Interaction Model in Code Gillian Armstrong @virtualgill
  89. 89. CODE HERE CODE HERE CODE HERE Design in Code Gillian Armstrong @virtualgill
  90. 90. CODE HERE CODE HERE CODE HERE CODE HERE CODE HERE CODE HERE CODE HERE Infrastructure in Code Gillian Armstrong @virtualgill
  91. 91. CODE HERE EVERYTHING in Code CODE HERE CODE HERE CODE HERE CODE HERE CODE HERE CODE HERE Gillian Armstrong @virtualgill
  92. 92. EVERYTHING in Code • Testable • Traceable • Repeatable • Observable Gillian Armstrong @virtualgill
  93. 93. Gillian Armstrong @virtualgill
  94. 94. CONTEXT CONVERSATION SESSION CHAT STREAM ANALYTICS NLP + INTENT MAPPING FULFILLMENT… Amazon Lex Amazon VPC AWS Direct Connect AWS Lambda AWS Lambda Amazon DynamoDB Amazon DynamoDB AWS Lambda Amazon CloudWatch Amazon ES IAM Amazon Polly Amazon S3 AWS X-Ray Amazon API Gateway Amazon Athena Amazon S3 Amazon API Gateway THE DIGITAL ASSISTANT BOT PLATFORM ARCHITECTURE
  95. 95. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Gillian Armstrong @virtualgill
  96. 96. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Local and Pre-Commit Hook Tests • Unit Test • Lambdas • Libraries • Static analysis • Cloud Formation • Code • JSON for API calls Gillian Armstrong @virtualgill
  97. 97. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments Gillian Armstrong @virtualgill
  98. 98. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments Build Tests • Unit Test • Lambdas • Libraries • Static analysis • Cloud Formation • Code • JSON for API calls Gillian Armstrong @virtualgill
  99. 99. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Gillian Armstrong @virtualgill
  100. 100. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Gillian Armstrong @virtualgill
  101. 101. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments AWS CloudFormation Templates Amazon S3 deployment bucket Amazon Lex Model Building Service Botbuilder Lambda function Packaged Fulfillment Lambdas Bot JSON files Gillian Armstrong @virtualgill
  102. 102. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Deploy Tests • Smoke Tests • Component Tests (Lex) Non-Prod Only Gillian Armstrong @virtualgill
  103. 103. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Fully tested new functionality available in minutes Gillian Armstrong @virtualgill
  104. 104. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Live Analytics Fully tested new functionality available in minutes Gillian Armstrong @virtualgill
  105. 105. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments All tests, builds, deploys and alarms sent to Slack channels to alert developers immediately Live Analytics ‘Live’ Tests - APM • Alarms • Cloudwatch metrics and Dashboards • Centralized Logging • Tracing Gillian Armstrong @virtualgill
  106. 106. Conversation as Code - Amazon Lex Fully Validated and Repeatable Bot Build and Deploy to any AWS Account Everything written in CFN, with JSON for API calls to Amazon Lex Model Building API as part of deploy Code push triggers tests, and automatic build and deploy across multiple environments Live Analytics ‘Live’ Tests - APM • Alarms • Cloudwatch metrics and Dashboards • Centralized Logging • Tracing Gillian Armstrong @virtualgill
  107. 107. If your chatbot stopped working, would you know? Don’t just wait for user complaints! Know what isn’t working and fix it, and keep adding more of what is! Monitor, no, seriously, monitor Gillian Armstrong @virtualgill
  108. 108. Conversational Analytics are vital Work out what metrics you need to track to know how your chatbot is performing. Know what isn’t working and fix it, and keep adding more of what is! Gillian Armstrong @virtualgill
  109. 109. Getting Feedback is vital You won’t get it right first time – listen to your users. Know what isn’t working and fix it, and keep adding more of what is! Gillian Armstrong @virtualgill
  110. 110. Change Fast Learn Fast lets us Being able to Gillian Armstrong @virtualgill
  111. 111. Conversational UI is about Listening to what the user wants to do… CHANGE FAST LEARN FAST …and that’s the secret to a great chatbot! Gillian Armstrong @virtualgill
  112. 112. Got questions? Let’s chat! @virtualgill KEY TAKEAW AYS • Model your Chatbot UI and Architecture based on observing Humans and real Conversation. Break out of old mindsets! • New tech still needs the good engineering practices you already know Gillian Armstrong @virtualgill
  113. 113. Appendix – Additional Information on Conversational Design Starting out: https://hackernoon.com/new-to-conversational-design-start-here-7f2f3a1b81bb Error Handling in Chatbot Series: Part 1 — Voice Recognition Errors: The one where we end up shouting at a computer https://chatbotsmagazine.com/difficult-conversations-1-voice-recognition-error-handling-74d93056ddce Part 2 – Conversational Errors: The one where it gets really interesting https://chatbotsmagazine.com/helping-your-baby-bot-learn-to-chat-like-a-grown-up-bot-99f5170f1c55 Part 3 – Technical Faults: The one we want to pretend will never happen (or ‘Help! My chatbot has fallen over and can’t get up’) https://chatbotsmagazine.com/good-ux-when-your-chatbot-is-having-a-very-bad-day-e4f22885d7fb You can find my other Blogs at https://medium.com/@virtualgill Come talk to me on Twitter – I hang out there a lot @virtualgill For my other talks and ways to get in contact with me head to http://virtualgill.io
  114. 114. Watch the video with slide synchronization on InfoQ.com! https://www.infoq.com/presentations/ serverless-chatbots

×