Using voice commands has been pretty ubiquitous nowadays, as more mobile phone users use voice assistants like Siri and Cortana, as well as devices like Amazon Echo and Google Home have been invading our living rooms. You can interact with the intelligent assistant without leaving your couch. At the same time, chatbots have been insanely popular, and services like Slack and Facebook Messenger let you achieve multiple tasks without leaving the client- you can schedule a meeting, order some pizza, call a taxi, etc. Historically in web development, we have been relying on various UI elements to interact with your users. Now with the new technologies, you can develop rich applications with natural user interactions with a minimal visual interface. This enables countless use cases for richer and more accessible web applications. In this talk, Tomomi Imura will talk about the examples of the conversational interface, and what and how you can build with JavaScript in a browser using the Speech API, the open web standard, also with Node.js to work with the 3rd party platforms!

1. 1
Building Bots for Human with
Conversational Interface
Tomomi Imura (@girlie_mac)
#TechWorldSummit

2. 2
@girlie_mac
2
HELLO!
I am Tomomi Imura
from Slack

3. 3
...

4. 4
@girlie_mac
4
What are Bots?
a software application or script that
runs automated tasks
Typically, bots perform tasks that are both simple and
structurally repetitive, at a much higher rate than would be
possible for a human alone.
https://en.wikipedia.org/wiki/Internet_bot

5. 5
@girlie_mac
5
What are Bots?
Also are a user experience that can expose
services to users via conversational
engagement and rich interactions.

6. 66
"Bots are like new applications
that you can converse with."
-- Satya Nadella, Microsoft (2016)

7. 77
Future of computing revolves around
three principal factors:
● Hunams
● Digital assistants
● Bots

8. 8
@girlie_mac
8
User Interactions
Then & Now

9. 9
@girlie_mac
9
Traditional Web & App Interactions

10. 10
@girlie_mac
10
Interface with Social Interactions

11. 11
@girlie_mac
11
Conversational User Interactions:
with Voice Assistants
Alexa, how is the
weather?
Hey Siri
In various form-factors

12. 12
@girlie_mac
12
Conversational User Interactions:
Google Assistant (Voice & Text)
Hi, how can I help?

13. 13
Google Assistant
Web Search to Assistant (Voice)
★ Assistant queries are 20% longer than the similar
search
★ Assistant queries are 200x conversational than
search

14. 14
Web Search vs.
“Weather Stockholm”
“Time San Francisco”
“How’s the weather in Stockholm?”
“What time is it now in San Francisco?”
Voice Assistant

15. 15
Google Assistant
Web Search to Assistant
★ 40% as likely to be an action
“Set a timer”
“Call home”
“What is the song?”
“Open YouTube”
“Turn light off”
OK Google, walk my dog!

16. 16
@girlie_mac
16
Conversational User Interactions:
Slack Apps (Text-based)
!

17. 17
@girlie_mac
17
Conversational Interface
achieves:
Natural user interactions with
a minimal visual interface.

18. 18
@girlie_mac
18
No UI Clutter

19. 19
@girlie_mac
One large margherita pizza!
What kind of crust do
you want?
1. Regular crust
2. Thin crust
3. Gluten free crust
2
cha-ching!

20. 20
@girlie_mac
20

21. 21
@girlie_mac
21

22. 22
@girlie_mac
22
Minimal visual interface,
but not zero UI

23. 23
@girlie_mac

24. 24
@girlie_mac
24
Alexa UX
“Voice Chrome” Examples
https://developer.amazon.com/docs/alexa-voice-service/ux-design-overview.html
Listening
Speaking

25. 25
@girlie_mac
25
Hand-free interface is
safe during driving

26. 26
@girlie_mac
26
https://voicebot.ai/wp-content/uploads/2019/01/
in-car_voice_assistant_consumer_adoption_report
_2019_voicebot.pdf
In-Car Voice Assistant Use Cases
https://voicebot.ai/wp-content/uploads/2019/01/in-car
_voice_assistant_consumer_adoption_report_2019_vo
icebot.pdf

27. 27
@girlie_mac
27
Conversational
Interface should be:

28. 28
@girlie_mac
28
● Intuitive
● Accessible
● Productive

29. 29
@girlie_mac
29
Text-based Messaging Platforms
● Slack
● Facebook Messenger
● Telegram
● WeChat
● Kik
● Viver
● LINE etc.

30. 30
@girlie_mac
30
Messaging + Bots
for Interactive Communications & Productivity

31. 31
@girlie_mac
31
Slack PlatformSlack Client
Bots on Slack
Using Slack APIs to customize your workspace
Your App Server
(or go serverless e.g.
Google Cloud Functions)
Web API (HTTP)
RTM API
(WebSocket)
or

32. 32
Slack Interactions:
Slash Commands
1. A user sends a command
2. A payload sent to the configured server
URL via HTTP POST
3. The server sends back a JSON response
with HTTP 200

33. 33
Slack Interactions:
Conversational
1. A user sends a message
2. Events API “pings” to the
configures URL with the message
details
3. The bot sends back a message
via Web API method
chat.postMessage

34. 34
@girlie_mac
34
Slack Bots at Slack

35. 35
@girlie_mac
35
Slack Bots at Slack

36. 36
@girlie_mac
36
Slack Bots at Slack
#cats

37. 37
@girlie_mac
37
Taco Bot for Slack by Taco Bell
https://www.tacobell.com/feed/tacobot

38. 38
@girlie_mac
38
Conversational Bots + NLP
What is Natural Language Processing (NLP)?
Process and analyze natural language data to make the
interactions between computers & human, artificial & natural
languages possible

39. 39
@girlie_mac
39
Rise of Chat Bot =
Rise of Conversational NLP APIs

40. 40
@girlie_mac
40
APIs with No-Code Tools
● DialogFlow (Google)
● Wit.ai (Facebook)
● Recast.ai (SAP)
● Motion.ai (HubSpot)
● ChatBot
● Converse AI
● Landbot.io
● ManyChat
● Pandorabots
etc.

41. 41
@girlie_mac
41
User-Friendly “AI-as-a-Service”
DialogFlow
Motion.ai

42. 42
@girlie_mac
42
NLP w/ Dialogflow
Dialogflow
Unstructured Text
(Human Understandable)
User Intent
(Machine Understandable)
Remind me to
water plants at
7am everyday
Intent: Agenda / Create event
Task: Water plants
Time: 7am
Date: Everyday
User
{
“id”: “7098-293d-343e-b4556-6e3df36”
“timestamp”:”2019-06-04T10:45.4382”,
“lang”: “en”,
“result”: {
“source”: “agent”,
“fulfillment”: {
“speech”: “Reminds me to water plants at 7am
every day”,
“messages”: [
{
“type”: 0,
“speech”: “Reminds me to water plants at 7am
every day”,
}
]
},
“score”: 0.34
}
}

43. 43
@girlie_mac
43
Powerful NLP Platforms
Natural Language Processing & Cognitive platforms with variety of
APIs (e.g. text analysis, spell-check, translations, etc.)
● IBM Watson
● Google Cloud Natural Language API
● Microsoft Azure Bot Services & LUIS
● Amazon Lex
● Baidu UNIT

44. 44
@girlie_mac
44
Example: IBM Watson Sentiment Analysis API used with Slack
https://api.slack.com/tutorials/watson-sentiment

45. 45
@girlie_mac
Hi, I want to book a flight!
Yes, from SFO.
Where are you flying to?
Stockholm
Hi Linda, welcome back! Are you
flying from San Francisco, as
usual?
Okey-dokey, hold on.

46. 46
@girlie_mac
An airline customer
service bot
Linda thinks the
rep is a nice human
Bots and Ethics
She’s been very helpful
Hmm, hold on for a sec!

47. 47
@girlie_mac
47
KLM Royal Dutch Airlines
BlueBot (BB)
On Facebook Messenger

48. 48
@girlie_mac
48
UPS Chatbot
On:
● Facebook Messenger
● Skype
● Google Assistant
● Amazon Alexa

49. 49
@girlie_mac
49
Hmmm…
Did you miss me?

50. 50
Voice on Web Browsers
Yes, your browsers can talk too!

51. 51
@girlie_mac
51
Web Speech APIs
1. Speech recognition API
2. Speech synthesis API
http://caniuse.com/#feat=speech-recognition
http://caniuse.com/#feat=speech-synthesis

52. 52
@girlie_mac
52
Accessibility & Voice APIs
Who depends on the features?
● People with physical disabilities who cannot use the keyboard or mouse
● People with chronic conditions, such as repetitive stress injuries (RSI)
● People with cognitive and learning disabilities who need to use voice rather
than to type
● People who are blind / partial sight and cannot see what is on the screen
● People with dyslexia and other cognitive and learning disabilities who need to
hear and see the text to better understand it
https://www.w3.org/WAI

53. 53
@girlie_mac
53
const SpeechRecognition =
window.SpeechRecognition || window.webkitSpeechRecognition;
const recognition = new SpeechRecognition();
Speech Recognition API

54. 54
@girlie_mac
54
recognition.lang = 'en-US';
recognition.interimResults = false;
recognition.start();
recognition.addEventListener('result', (e) => {
let last = e.results.length - 1;
let text = e.results[last][0].transcript;
});

55. 55
Demo on CodePen: https://codepen.io/girliemac/pen/qKdgze

56. 56
@girlie_mac
56
Speech Synthesis API
const synth = window.speechSynthesis;
const utterance = new SpeechSynthesisUtterance();
utterance.text = 'わたしはロボット';
utterance.pitch = 1.5;
utterance.lang = 'ja-JP';
synth.speak(utterance);

57. 57
@girlie_mac
57
Demo:
Web AI Bot
https://webspeech.herokuapp.com/

58. 58
@girlie_mac
58
JavaScript on browser
Node.js server.js
Text data
socket.io
DialogFlow
Voice-to-text
(Web Speech Recognition API)
Text-to-voice
(Web Speech
Synthesis API)

59. 59
@girlie_mac
59
Article: smashingmagazine.com/
2017/08/ai-chatbot-web-
speech-api-node-js/
github.com/girliemac/
web-speech-ai

60. 60
@girlie_mac
60
Conversational
Interface is a
human interface

61. 61
@girlie_mac
61
Think human users first

62. 62
@girlie_mac
62
What I’ve mentioned:
● Give voice assistant some UI / visual cues
● Don’t trick users - let them know they chat with a bot
● Pick right APIs - Need no-code tools? Need translations?
● Think accessibility
Think Humans First

63. 63
@girlie_mac
63
Also,
● Avoid the text-only UI. Combine with rich interactions like links
& buttons, if the platform allows
● Give users a proper intro & onboarding guide
● Support keywords like help
● Handle errors gracefully
Think Humans First

64. 64
@girlie_mac
64
Write code for machine,
build bots for human.

65. 65
@girlie_mac
65
Lastly...

66. 66
@girlie_mac
66
AI or cats,
which will take us over
first?

67. Tack så mycket!
Tomomi Imura
@girlie_mac
girliemac
slideshare.net/tomomi
67

68. “Imitation is the sincerest form of flattery that mediocrity can
pay to greatness.” - Oscar Wilde
There are some near-exact copy of my talk floating around and I’d like to claim my originality. I created the
content from my own research and experiences, also wrote sample code from a scratch.
I have modified my content slightly since I discovered, yet, the copy talks are still way too similar to mine. The
side-by-side comparison proved the plagiarism. Yes, I am flattered, but this is definitely not cool.
However, I still like to keep my content under CC BY-SA 4.0 for educational purpose, so if you’d like to use the
materials for meetups, brown bag sessions, etc. please do state you borrowed content from me.
Thank you ❤
Tomomi