Voxxed Athens 2018 - Let’s Get Chatty with Conversational Interface with JavaScript by Tomomi Imura

1. Let'sGetChattywithConversational
InterfaceinJavaScript
BOTS,AI,ANDJAVASCRIPT
TomomiImura(@girlie_mac)
#vdAthens

2. @girlie_mac
●anopenweb&techadvocate
●codeJavaScript/Node.js
●workatSlackinSanFrancisco
●acatladyoftheInterWeb(created
HTTPStatusCats.https://http.cat)
tomomiimura
@girlie_mac

4. "Botsarelikenew
applicationsthat
youcanconverse
with."
--SatyaNadella,Microsoft

5. @girlie_mac
“Wewillevolve
incomputing
fromamobile
firsttoanAIfirst
world.”
--SundarPichai,Google

6. @girlie_macCCBY-SA:nextdayblinds.com

7. @girlie_mac
TraditionalWeb&AppUserInteractions

8. @girlie_mac
ModernWeb&AppswithSocialInteractions

9. @girlie_mac
ConversationalUserInteractions:SiriandAlexa(VoiceAssistants)
Alexa,howis
theweather?
HeySiri
Invariousform-factors

10. @girlie_mac
ConversationalUserInteractionsforKids-withVoice
CogniToysDino-https://cognitoys.com

11. @girlie_mac
ConversationalUserInteractionsinarobotshape-withVoice

12. @girlie_mac
ophiatheAIRobot

13. @girlie_mac
ConversationalUserInteractions:GoogleAssistant(Voice&Text)
(GIFanimation)

14. @girlie_mac
ConversationalUserInteractions:SlackBots(TextwithGUI)
!

15. Graphic
Interfaceto
Conversational
Interface

16. @girlie_mac
Delivermealargemargheritapizza!
Whatkindofcrustdoyou
want?
1.Regularcrust
2.Thincrust
3.Glutenfreecrust
Thincrust
chaching!

17. Conversational
Interfaceachieves:
Naturaluser
interactionswitha
minimalvisual
interface.

18. NoUIClutter.
LessTimeSpent.

19. @girlie_mac
Whereistheaddress
ofyourhome?
It’s325RubyStreet.
Requestaride
Comeovertomyplace!

20. @girlie_mac
Yes,getmearidenow
Yourdriver,Samwill
pickyouupin6minutes.
LookfortheredToyota
Prius!

21. @girlie_mac
AlexaUX(“VoiceChrome”Examples)
https://developer.amazon.com/docs/alexa-voice-service/ux-design-overview.html
Listening
Speaking
(GIFanimation)

22. @girlie_mac

23. Conversational
Interfaceis:

24. ●Intuitive
●Accessible
●Productive

25. @girlie_mac
MessagingPlatforms
●Slack
●FacebookMessenger
●Telegram
●WeChat
●Kik
●Viver
●LINEetc.

26. Messaging+BotsforMoreInteractiveCommunications

27. Slackbotsat

28. Slackbotsat

29. Slack
Messageactions

30. TacoBotbyTacoBell
https://www.tacobell.com/feed/tacobot

31. @girlie_mac
NaturalConversationAPIs
●DialogFlow(API.ai
/Google)
●Wit.ai(Facebook)
●MicrosoftBot
Framework
●Motion.ai
●Chatbots.io
●ConverseAI
●Landbot.io
●Recast.ai
●ManyChat
●Fluent.ai(VoiceUI)
etc.

32. @girlie_mac

33. AIaaS
Motion.ai
DialogFlow(API.ai)
Artificial
Intelligenceas
aService

34. @girlie_mac
ManyChat

35. @girlie_mac
Hi,
Iwanttobookaflight!
Yes,fromSFO.
Whereareyouflyingto?
London
HiLinda!Areyouflyingfrom
SanFrancisco,asusual?
Anairlinecustomer
servicebot
Lindamaynot
knowsheistalking
toabot!

36. @girlie_mac
Using
DialogFlow

37. @girlie_mac
MorePowerfulNLPPlatforms&APIs
NaturalLanguageProcessing&Cognitiveplatforms:
●IBMWatson
●GoogleCloudNaturalLanguageAPI
●MicrosoftLUIS
●AmazonLex
●BaiduUNIT

38. @girlie_mac
https://angel.co/newsletters/the-return-of-clippy-050418
“Clippyismaking
acomeback!”

39. Conversational
Interface
WithJavaScript

40. @girlie_mac
{REST}
JSSDK
TheAPIsare
mostly
accessiblewith
JS

41. @girlie_mac
Example:DialogFlow(API.ai)+FBMessenger
constapp=require('apiai')(CLIENT_ACCESS_TOKEN);
functionsendMessage(event){
letsender=event.sender.id;
lettext=event.message.text;
letai=app.textRequest(text,{
sessionId:SESSION_STRING});
ai.on('response',(response)=>{//Gotaresponsefromapi.ai
letaiText=response.result.fulfillment.speech;
//ThenPOSTtohttps://graph.facebook.com/v2.6/me/messages
...
});
ai.end();
}
API.aiNode.jsSDK

42. @girlie_mac
Example:DialogFlow(API.ai)+FBMessenger

43. @girlie_mac
Example:SlackMessageSentimentAnalysiswithIBMWatson
(GIFanimation)

44. @girlie_mac
Example:IBMWatson+Slack
Thebotworkflow:
1.ReadeachmessageonaSlackchannel
2.SendthemessagetoIBMWatsonfor
examination
3.Ifthelikelihoodofanemotionisabovethegiven
confidencethresholdpostthemostprominent
emotion

45. @girlie_mac
Example:IBMWatson+Slack
app.post('/events',(req,res)=>{
letq=req.body;
if(q.type==='event_callback'){
if(!q.event.text)return;
analyzeTone(q.event);
}
});
UseSlackEventsAPItograbthe
textwhenauserpostamessage
PassthetextdatatoWatsontoanalyze
HTTPPOSTusingExpressJS

46. @girlie_mac
Example:IBMWatson+Slack
constwatson=require('watson-developer-cloud');
lettone_analyzer=watson.tone_analyzer({
username:process.env.WATSON_USERNAME,
password:process.env.WATSON_PASSWORD,
});
constconfidencethreshold=0.55;
tone_analyzer.tone({text:text},(err,tone)=>{
tone.document_tone.tone_categories.forEach((tonecategory)=>{
if(tonecategory.category_id==='emotion_tone'){
tonecategory.tones.forEach((emotion)=>{
if(emotion.score>=confidencethreshold){
postEmotionOnSlackChannel(emotion);
}});
}});
});
Returns
emotionsscorein
0to1
Justinitializingitw/
yourAPIcredentials
PosttheresultonSlack

47. @girlie_mac
Example:IBMWatson+Slack+RaspberryPi(forfun)
functioncolorEmotion(emotion){
if(emotion.tone_id==='anger'){
setLED(red);
}elseif(emotion.tone_id==='joy'){
setLED(yellow);
}elseif(emotion.tone_id==='fear'){
setLED(purple);
}elseif(emotion.tone_id==='disgust'){
setLED(green);
}elseif(emotion.tone_id==='sadness'){
setLED(blue);
}
}ChangetheLEDcolorto
matchtheemotion

48. @girlie_mac

49. @girlie_mac
Ack,thissucks!Iwantmy
moneyback!
Anangrycustomerdetected.Connect
thecustomerwithahuman!

50. Conversational
InterfacewithVoice
inBrowser?

51. @girlie_mac
Project:ArtificialVoiceChat
Hello!
1
2
Howdy
2.GenerateArtificialreply
Usingthe3rdpartyNLPAPI
1.Usertalktobrowser
Voicecommand:
Voice->Text
33.Browserspeaksback
Text->SyntheticVoice
Text

52. @girlie_mac
WebSpeechAPI
SpeechRecognition&SpeechSynthesis
http://caniuse.com/#feat=speech-recognition
http://caniuse.com/#feat=speech-synthesis
13
4

53. @girlie_mac
WebSpeechAPI:Speech
Recognition
constSpeechRecognition=
window.SpeechRecognition||window.webkitSpeechRecognition;
constrecognition=newSpeechRecognition();
Getaninstanceofthe
SpeechRecognition,thecontroller
interface
InthecurrentChromium,itis
stillprefixed

54. @girlie_mac
WebSpeechAPI:SpeechRecognition(Cont’d)
recognition.lang='en-US';
recognition.interimResults=false;
recognition.start();
recognition.addEventListener('result',(e)=>{
letlast=e.results.length-1;
lettext=e.results[last][0].transcript;
});
Someproperties
Methods:start(),stop(),
abort()
Events:onresult,
onerror,
onaudiostarted,
onaudioend,etc.

55. @girlie_mac
WebSpeechAPI:SpeechRecognition-MicAccessPermission

56. @girlie_mac
WebSpeechAPI:SpeechRecognition
Hello!
VOICE
fromauser
TEXT
“hello”
SpeechRecognitiondemoonCodePen
https://codepen.io/girliemac/pen/dmpxgv

57. @girlie_mac
WebSpeechAPI:SpeechSynthesis
constsynth=window.speechSynthesis;
constutterance=newSpeechSynthesisUtterance();
utterance.text='Iamarobot';
utterance.pitch=1.5;
utterance.lang='ja-JP';
synth.speak(utterance);
Novendorprefix
Propertiesofthe
SpeechSynthesisUtteranceinterface

58. @girlie_mac
WebSpeechAPI-Multi-languageSupports
QUICKDEMO:
https://codepen.io/girliemac/pen/qKdgze
どう、元気にしてる？

59. @girlie_mac
WebSpeechAPI:SpeechSynthesis
(voicebyAlexor
whoeveritis)
TEXTtoVOICE
“howdy”
Howdy
SpeechRecognitiondemoonCodePen
https://codepen.io/girliemac/pen/dmpxgv

60. @girlie_mac
Project:ArtificialVoiceChat

61. Demotime!
https://webspeech.herokuapp.com/

62. @girlie_mac
Article:
smashingmagazine.com/
2017/08/ai-chatbot-web-
speech-api-node-js/
github.com/girliemac/
web-speech-ai

63. Conversational
Interfaceis
forhuman.

64. Abotsinterfacereallyis
ahumaninterface.

65. Writecodeformachine,
Buildbotsforhuman.

66. @girlie_mac
Thanky...
Ohhhh,wait!.There’sonemoreslide...

67. @girlie_mac
AIorcats,
whichwilltakeus
overfirst?

68. @girlie_mac
ευχαριστώ!
@girlie_mac
girliemac.com
github.com/girliemac
speakerdeck.com/girlie_mac

69. @girlie_mac
Attribution:
OpenEmojibyEmoji-One(CC-BY4.0)