Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How We Built A Bot For FITC

161 views

Published on

Presented at FITC Toronto 2017
More info at http://fitc.ca/event/to17/

Presented by Rami Sayar, Microsoft

Overview

Imagine a platform where language is the new UI. When we build chat bots that augment human abilities and experiences, it’s not about man versus machine. It is about man with machines, and each excels at different things. Machines are incredibly persistent if we instruct them to be; even if all we want them to do is provide us with an endless series of cat GIFs. And what better way to please the human spirit than by randomly sending you a cat GIF and being able to answer your questions!

This talk will introduce Microsoft’s Conversation as a Platform and dive into the Bot Framework and Microsoft Cognitive Services. Rami will show you how we built the FITC bot, the design considerations and conversation flow followed by the reasoning behind our insistence on cat GIFs.

Target Audience

Creative JavaScript Developers

Assumed Audience Knowledge

JavaScript

Five Things Audience Members Will Learn

What are chat bots?
Designing a conversation tree for the FITC Bot.
How do you build a conversation tree with the bot framework?
Understanding how to use natural language processing APIs in your bots
Learning about the fabulous Cat API.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

How We Built A Bot For FITC

  1. 1. HOW WE BUILT A BOT FOR FITC Rami Sayar - @ramisayar Senior Technical Evangelist Microsoft Canada @RAMISAYAR
  2. 2. @ramisayar @RAMISAYAR
  3. 3. What You Will Learn Today • What are Bots? Why Now? • Conversations as a Platform? • Building the FITC Bot • Introduction to the Bot Framework • Introduction to Cognitive Services
  4. 4. What are Bots? Why Now? @RAMISAYAR
  5. 5. @RAMISAYAR Image from Chatbots Magazine
  6. 6. Microsoft Xiaoice now Microsoft Zo • Natural Language Chat Bot • Over 20 million registered users • The average user interacts with the service 60 times a month • 850 thousand followers on Weibo • Ranked as Weibo’s top influencer
  7. 7. Are Bots Just Hype? Why Now?
  8. 8. Messaging and Mobile Platforms • Mobile is the new scale. • Every app can access your address book, getting an instant social graph. The phone number in particular acts as a unique social identifier. • They can access the photo library and camera directly (and location), making sharing easy. • Push notifications mean you don’t need people to keep checking your site (or open emails).
  9. 9. Once in a Decade Opportunity • Users spending more time on messaging apps • Opportunity for more natural experiences • High demand from companies of all sizes • Simpler deployment/update model than apps • App ecosystem overly crowded • Everybody wants to automate something @RAMISAYAR
  10. 10. Hype Cycle
  11. 11. Is It Here to Stay? • Messaging & Social are the top experiences across all platforms. • All the large players are bought in. • Language understanding has progressed considerably. • Conversational UI is familiar. • AI & Machine Learning are now mainstream. • Cognitive services are accessible.
  12. 12. Conversational UI + Intelligence are the differentiators. Don’t be a Clippy.
  13. 13. Building a Bot for FITC And also finding a way to troll Shawn @RAMISAYAR
  14. 14. Fun Functional
  15. 15. @RAMISAYAR
  16. 16. @RAMISAYAR
  17. 17. Introducing the Microsoft Bot Framework
  18. 18. https://github.com/Microsoft/BotBuilder
  19. 19. An X-ray of a Typical Bot Your bot code Entity Extraction Speech Vision / Face Natural Language Translation + Microsoft Cognitive Services Search Emotion Knowledge API … Message Input <> Output State Management Bot Connector Service Conversation Canvas / …… … Other services, APIs, Databases, Azure Machine Learning, Azure Search, etc… Bot Builder SDK Web Chat Direct Line… Email Facebook GroupMe Kik Skype Slack Telegram Twilio (SMS) Bot Builder SDK Your code goes here
  20. 20. Your conversation logic Logic Web Service Your Bot LUIS Your Bot Code • Build with C# or Node.js • You host your bot wherever you want • Dialogs to model a conversation
  21. 21. Bot Framework – Connector Service Interact naturally wherever your users are talking
  22. 22. Bot Framework – Connector Service Emulator
  23. 23. Bot Builder • Dialogs to Model a Conversation • Dialogs are reusable • Conversations are scalable to multiple machines • Dialog state is persisted in Bot Connector (per-user, per-session, per- user+session) • Types of Dialogs: • Built-in prompts • Yes/No, String, Number, Choices • Natural language understanding (LUIS) • Form slot filling (branching, disambiguation, multi-turn) • Profile (e.g., home address)
  24. 24. Bot Builder – Attachments • Attachments property • Array of attachment objects • Send / receive images • Rich Cards Name Description Example ContentType The contentType of the ContentUrl property image/png ContentUrl A link to content of type ContentType http://somedomain.com/cat.jpg Content An embedded object of type contentType If contentType = "application/vnd.microsoft.hero" then Content would be a JSON object for the HeroCard
  25. 25. Bot Builder – Rich Cards • Hero and Thumbnail cards • multipurpose card; • a single large or small image • One or more button • and a "tap action“ • along with text content to display on the card. • Receipt card • a single small image • a button • and a "tap action“ • text content • Carousel card • Multiple cards • Navigation between cards • Each has ImageButtonsText
  26. 26. Getting your Bot Project Started Step 1: Step 2: Step 3:
  27. 27. Creating your Bot • Create the connector • We'll start with the console connector • Create the bot • Pass in the connector • Add logic • Done through dialog • Communicate with the user • Done through the session
  28. 28. Your First Bot
  29. 29. Diving into the FITC Bot – Find Speaker github.com/sayar/fitcbot
  30. 30. Developer Portal • Register the bot • Configure desired channels • Publish in the Bot Directory. • All bots registered with Bot Framework are auto-configured to work with Skype and the Web.
  31. 31. Bot Directory • Public Directory of Bot Framework Bots • Users can discover, try, and add bots to the conversation experiences on which the bot is configured (no messenging app required) • Bots are public at developer discretion; bots must be submitted for review and approved in order to appear in the directory • Searchable https://bots.botframework.com/
  32. 32. • Lets you understand what your users are saying (e.g. “Read me the headlines” or “Pause for 5 minutes”) • Determines intent • Detects entities • Seamless integration with Speech Recognition • LUIS learns over time • Use pre-built, world class models from Bing and Cortana • Models work across Internet connected devices Language Understanding Intelligent Service Intent: TextToSpeech Content: NewsHeadlines Read me the headlines Play yesterday’s Daily Show Intent: PauseDevice DateTime.duration: 5 Minutes Pause for 5 minutes Intent: PlayEpisode Content: Daily Show DateTime.date: T-1 Day
  33. 33. • Analyze an Image – Understand content and features within an image for a growing set of categorization • Wide range of concepts from cat to fireworks • OCR – Detect and recognize words within photos or scanned documents • Generate Thumbnail – Scale and crop an image, while retaining key content • Powers Bing Image search and OneDrive.com Photos Vision APIs Recognized text: A GOAL WITHOUT A PLAN IS JUST A WISH Analyze image OCR Smart thumbnail Category People; 1 face found Adult / Racy? False / True Black & White? No Dominant colors Accent color
  34. 34. • Face Detection – Detects faces and their attributes within an image • Face Verification – Check if two faces belong to the same person • Similar Face Searching – Find similar faces within a set of images • Face Grouping – Organize many faces into groups • Face Identification – Search which person a face belongs to Face APIs Face Rectangle – width; height; left; top Attributes: • Age – 31 • Gender – Male Face Landmarks –pupil, nose, mouth Detection Result Grouping He is Chao Wang… Identity
  35. 35. • Identifies emotions based on expressions that psychologic research identifies as universal: Happiness, Surprise, Sadness, Anger, Fear, and Disgust • Has been trained to identify emotions in-the-wild. • Get insights into how your users react to content and experiences • Track facial expressions for multiple users over time. Emotion APIs
  36. 36. Diving into the FITC Bot – QnA Maker github.com/sayar/fitcbot
  37. 37. What Makes a Bot Great? • It is not how “smart” they are • It is not how much natural language they offer • It is not whether they use voice or not • It’s whether it solves the user’s needs in the quickest/easiest way compared to any other option • It’s all about user experience... Just like an app, or a website
  38. 38. How Does It React in Crisis? • Does it provide specific responses that lead? • Does it confirm actions? • Is it accessible & inclusive? • Does it allow human interruption? • Design for humans, not yourself. • Did you map out all the paths? You’re going to need a bigger whiteboard.
  39. 39. The "stubborn bot" The "clueless bot" The "mysterious bot" The "bot that can't forget"
  40. 40. Satya’s Artificial Intelligence Bots Design Principles • Bots must be designed to assist humanity. • Bots must be transparent. • Bots must maximize efficiencies without destroying the dignity of people. • Bots must be designed for intelligent privacy. • Bots must have algorithmic accountability so that humans can undo unintended harm. • Bots must guard against bias, ensuring proper, and representative research so that the wrong heuristics cannot be used to discriminate.
  41. 41. The Best Bots Make You Type Less, Not More
  42. 42. Thank You! Questions? tw: @ramisayar | gh: @sayar @RAMISAYAR
  43. 43. Resources, References, Links • Bot Framework Home Page • https://dev.botframework.com/ • Bot Framework downloads • https://docs.botframework.com/en-us/downloads/ • Bot Builder SDK on GitHub • https://github.com/Microsoft/BotBuilder • Bot Framework Blog • https://blog.botframework.com/ @RAMISAYAR
  44. 44. ©2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Office, Azure, System Center, Dynamics and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

×