Conversational User Interfaces, Past and Future

1. Conversational User Interfaces: Past and Future Crispin Reedy @crispinTX

2. 2017 goals 1. Provide more UX resources to UX professionals. 2. Improve brand consistency. 3. Improve “International” UXPA. User Experience Professionals Association (UXPA) Monthly webinars User Experience Magazine The Journal of Usability Studies Visit www.uxpa.org for more info about the conference, webinars, and other UX resources! Annual Conference in Toronto, Canada June 5 – 8, 2017 uxpa2017.org

3. Conversational User Interfaces: Past and Future Crispin Reedy @crispinTX

4. 4© 2017 Versay Solutions LLC • Voice User Interface Designer • 15+ years in the field • Former coder; got interested in UX • President of the Association for Voice Interaction Design • Consultant for Versay Solutions @crispinTX crispinreedy.com

5. CUI: Past and Future • How can chatbots learn from existing VUI design? What makes these new interfaces different, and how are they similar? Where do the Alexas and Siris come into the mix? We’ll discuss text-based vs. voice-based conversational user interfaces, and the landscape of Conversational User Interfaces, now and into the future. 5© 2017 Versay Solutions LLC

6. Disclaimers • What, exactly, is a Conversational User Interface? – For this discussion we are going broad and wide. • Conversation: – Interactive communication between two or more people • Spoken vs. written (chatbots) • Turntaking vs. one shot 6© 2017 Versay Solutions LLC

8. “I’m really excited about chat and natural language because there’s no user interface you have to deal with.” - Chatbot writer on support board

9. “Just this week, Facebook said it was “refocusing” its use of AI after its bots hit a failure rate of 70 percent, meaning bots could only get to 30 percent of requests without some sort of human intervention.” “I would call it overpromising,” said CP+B executive creative tech director Joe Corr. “Brands that created bots with a structured request or utility like Domino’s or in retail were easy. But bots that tried to break out of the utility and be chatbots became the problem.”

11. 23© 2017 Versay Solutions LLC Chatbots Voice Systems Taxonomy of CUIs Input Modality Domain Domain •Personal Assistants •Agents •General Search •Order Products •Troubleshooting •Manage Accounts •Alexa •Google Home •Wildfire •IVR •Billing information •Control-based

12. 24© 2017 Versay Solutions LLC Voice Systems Taxonomy of CUIs Domain •Alexa •Google Home •Wildfire •IVR •Troubleshooting •Manage Accounts Technology •What “thing” are you talking to? •What else are you connected to? (Network, data) •What kind of speech technology are you using? •What other input and output modalities are you coordinating with?

14. Voice Conversation vs. Chat Conversation • Differences – Written convention vs. spoken convention – Speech recognition vs. textual understanding • ASR vs. NLU • DNN, Machine Learning – Chatbot implies keyboard; voice interface could be to almost anything • This becomes really important: – Discoverability – Onboarding – Coordinating voice with other modalities

15. Voice Conversation vs. Chat Conversation • Commonalities – Logical structure of a conversation (flow) – Error recovery • If caller / user goes off topic – Prompts elicit expected responses • Sometimes! – Conversational maxims • English is still English! Well, sort of.

16. UX Still Applies! • Typical Tools: – Tasks – Personas – “Happy Path” High Level Conversations (Script) – Card sorts? – Read Aloud – Usability Testing • Wizard of Oz – A/B Testing 28© 2017 Versay Solutions LLC

17. Commonalities

18. Voice Conversational Model: Beginning NL Level 1: Open Ended Question • Hi, how may I help you today? • I’m calling about a problem with my phone. • Sure, what kind of problem are you having? • I don’t seem to have any internet. • I can help you with that. Are you calling about the Apple iPhone 6 that you have on your plan? • Yes NL Level 2: Identify Problem Directed Dialog: ID Phone Implicit: ID Caller

19. Chat Conversational Model: Beginning NL Level 1: Open Ended Question • Hi, how may I help you today? • I have a prob w/ phone • Sure, what kind of problem? • No internet connectin • OK. Is that about the Apple iPhone 6 you have on your plan? • Yep NL Level 2: Identify Problem Directed Dialog: ID Phone Implicit: ID Caller

20. Voice Conversational Model: Troubleshooting • So, first let’s check your settings. What about Airplane Mode… do you have that turned off? If you need help with that, say “tell me more.” • Tell me more • Sure. You can find Airplane Mode under “Settings.” (call continues) Step 1 + Fallback

21. Chat Conversational Model: Troubleshooting • Let’s check settings. Do you have Airplane Mode turned off? • IDK • OK to check Airplane Mode, go to “Settings.” Step 1 + Fallback

22. Error Recovery - Voice • If the recognizer doesn’t get something, you may have to reprompt. – How crucial is the information you needed to collect in order to move the conversation forward? • Eg: Alexa doesn’t always reprompt – Systems can be more directive if failure continues 34© 2017 Versay Solutions LLC “What color is that light?” It’s… <noise> “Sorry, what color?” “Sorry, what color?” <silence> “You can say ‘red’ or press 1, ‘green’ or press 2, …. (etc)

23. Error Recovery - Chatbot • Chatbots do need it! – Do you need to do a local recovery vs. a global recovery? – Specific point – Total reset 35Source: Chatbot.fail

25. 37

26. Grice’s Maxims Maxim Description Quantity Be as informative as is required, but not more informative. Quality Do not make statements that you believe are false or for which you lack evidence. Relation Be relevant. Manner Avoid obscure expressions Avoid ambiguity Be brief Be orderly

27. Paul Grice • British philosopher of language • “Logic and Conversation,” 1975 • The maxims describe the “default” assumptions that listeners make about how people talk. • When the assumptions are broken: – People look for underlying implications – People may assume that there is a problem with the conversation or speaker Source: Amazon

28. Grice’s Maxims: Applied • Be as informative as is required, but not more informative. – What is your user really wanting to do? • Goals and tasks – Am I telling the user too much during this conversation? • Unhelpful help. • Overly chatty bots

29. Source: Taco Bell Bot (That’s Nerdalicious)

30. Grice’s Maxims: Applied • Be relevant. – Users want to do what they want, not what you want them to do. • Upselling • Intrusive apps / bots – Maintain state

31. 43Source: chatbots.fail

32. Grice’s Maxims: Applied • Avoid obscure expressions – No jargon • Avoid ambiguity – Ex: “This” and “that” • Be brief • Be orderly – Organize information cleanly – ensure that IA (if applicable) is understandable and relevant.

33. Differences

34. 46© 2017 Versay Solutions LLC Voice Systems Taxonomy of CUIs Domain •Alexa •Google Home •Wildfire •IVR •Troubleshooting •Manage Accounts Technology •Are you doing speech recognition? •What kind? •What “thing” are you talking to? •What else are you connected to? •Network •Data •What other input modalities are you coordinating with?

35. Possibilities 47© 2017 Versay Solutions LLC Write an app (skill) for an agent such as Cortana / Alexa / Jibo Use cloud APIs to add ASR to your app / device / page / gadget Download an ASR engine and use full-featured capabilities for onboard robust recognition Build your own ASR / DNN

38. Today’s Cloud-Based Speech APIs • Distributed speech recognition – Collection and compression of speech is on the device – The language models are typically on the network – Can be speaker-dependent • Trains itself on your voice and on the acoustic environments you are in most often – Many companies are providing APIs to use their speech recognition 50© 2016 Versay Solutions LLC

40. Possible Approaches Cloud-Based APIs • Easy to get started • Lightweight • Not much specialized knowledge • Limited 52© 2016 Versay Solutions LLC Download / Embed / Roll Your Own • Customizable • Can be device-specific • More features • Higher powered • Will require specialized knowledge – Speech scientist • Can use open source

41. Should I Speech-Enable X? 53© 2016 Versay Solutions LLC Desktop App / Website • Easy to get started with API-based ASR • But the use case may not be as powerful Tablet / Mobile • Stronger use case • But will the network be available for APIs? Industrial Device • Great use case esp. with multimodal • But this is harder to do and probably will be custom Gadget • Decent use case • APIs are tailored for this • Will they do everything you need? • Will the extra modality be a plus or just a “silly add-on?” • Different modalities? Car • Safety considerations are high here • Need better user interfaces & more robust IVR • Touchtone can still be good for a lot of applications • Speech is good for complex call routing and input

42. Should I Speech-Enable X? 54

43. CUI: Past and Future • How can chatbots learn from existing VUI design? What makes these new interfaces different, and how are they similar? Where do the Alexas and Siris come into the mix? We’ll discuss text-based vs. voice-based conversational user interfaces, and the landscape of Conversational User Interfaces, now and into the future. 55© 2017 Versay Solutions LLC

44. Iron Man 2: Marvel Studios, Paramount Pictures

45. Resources • The Voice in the Machine: Building Computers that Understand Speech – Roberto Pieraccini – YouTube video: “Open the Pod Bay Doors, Siri” • Conversational design from Google: – https://developers.google.com/actions/design/ • Alexa VUI design – Alexa Skills Kit Voice Design Handbook” • Association for Voice Interaction Design - AVIxD.org – Wiki, Brown Bags, Journal – Looking for members and volunteers! 57© 2017 Versay Solutions LLC