Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Applying Science to Conversational UX Design

1,778 views

Published on

Virtual agents are back, and they're everywhere! Their user interfaces tend to be simply those of instant messaging... or none at all. Thus the user experience resides more in the sequencing of bits of natural language than in that of menus or screens. Although everyone knows how to engage in human conversation, creating an app that behaves like one requires a technical knowledge of the mechanics of human conversation. While Conversational UX Design is still a nascent discipline, formal models from Conversation Analysis offer a scientific foundation for design. This session from SXSW 2017 provided design principles and models for creating conversational UX.

Published in: Design
  • Hey guys! Who wants to chat with me? More photos with me here 👉 http://www.bit.ly/katekoxx
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Applying Science to Conversational UX Design

  1. 1. Applying Science to Conversational UX Design Bob Moore, Raphael Arar IBM Research
  2. 2. Interfaces have come a long way.
  3. 3. Applying Science to Conversational UX Design © 2017 IBM Research
  4. 4. Conversational Agents
  5. 5. “ Jean Baudrillard Sociologist The sad thing about artificial intelligence is that it lacks artifice and therefore intelligence.
  6. 6. Human Conversation APIs Conversational Systems Speech to Text Text to SpeechNatural Language
 Understanding Dialog Management
  7. 7. Human Conversation APIs Conversational Systems Text to SpeechSpeech to Text Natural Language
 Understanding Dialog Management
  8. 8. USER EXPERIENCE DESIGN ARCHITECTURE INFORMATION ARCHITECTURE HUMAN-COMPUTER INTERACTION INTERACTION DESIGN VISUAL DESIGN MECHANICAL ENGINEERING ELECTRICAL ENGINEERING CONTENT CREATION (Text, Data, Graphics) Signage Info Viz Navigation INTERFACE DESIGN Ubicomp Controls Interactive Environments USABILITY ENGINEERING INDUSTRIAL DESIGN Dan Saffer, UX Designer
  9. 9. USER EXPERIENCE DESIGN ARCHITECTURE INFORMATION ARCHITECTURE HUMAN-COMPUTER INTERACTION INTERACTION DESIGN CONVERSATION DESIGN MECHANICAL ENGINEERING ELECTRICAL ENGINEERING CONTENT CREATION (Text, Data, Graphics) Signage Ontology Management Navigation CONVERSATIONAL UI Ubicomp Controls Interactive Environments USABILITY ENGINEERING INDUSTRIAL DESIGN
  10. 10. What’s the state-of-the- art for conversational UI?
  11. 11. Web. 2017.
  12. 12. Applying Science to Conversational UX Design © 2017 IBM Research
  13. 13. Web. 1996.
  14. 14. Conversational UI. 2017.
  15. 15. Conversational UI. ????
  16. 16. Human Conversation APIs Conversational Systems Text to SpeechSpeech to Text Natural Language
 Understanding Dialog Management
  17. 17. Conversational Systems Dialog Management Conversation Analysis Human Conversation
  18. 18. What does natural conversation sound like?
  19. 19. 20 Des: What is the name? 21 Guy: Detweiler. D-e-t, 22 (1.2) 23 Guy: w-e, 24 (0.4) 25 Guy: i-l-e-r-. 26 (2.0) 27 Des: Foursome? 28 Guy: Yah. 29 (0.4) 30 Des: Electric carts? 31 (0.6) 32 Guy: Uh:::, n:no? I don’t 33 think so. 34 Des: Okay. We'll see yuh then, 35 Guy: Righto, 36 Des: Mm hm, Bye? 01 Des: G'morning. San Juan Hills 02 Country Club? 03 Guy: Guh morning. What’s-w-what 04 kind of a starting time 05 ken:: we get fer::hh 06 sometime this afternoon. 07 (0.7) 08 Guy: Any[time- 09 Des: [Oh:::, [let's see. 10 Guy: [Any time 11 tuhday. 12 Des: Two fordy. One, thirdy. 13 Guy: One thirty? 14 Des: Mm hm::? 15 Guy: One thirty. 16 (0.7) 17 Guy: .hh W'l at sounds like a 18 good time? 19 (0.4)
  20. 20. Not every voice or 
 text interaction is a conversation.
  21. 21. Speaker-change recurs, or at least occurs.1 Overwhelmingly, one party talks at a time.2 Occurrences of more than one speaker at a time are common, but brief.3 Transitions (from one turn to a next) with no gap and no overlap are common. Together with transitions characterized by slight gap
 or slight overlap, they make up the vast majority of transitions 4 Turn order is not fixed, but varies.5 Turn size is not fixed, but varies.6 Length of conversation is not specified in advance.7 Relative distribution of turns is not specified in advance.9 Number of parties can vary.10 Talk can be continuous or discontinuous.11 Turn-allocation techniques are obviously used. A current speaker may select a next speaker (as when he addresses a question to
 another party); or parties may self-select in starting to talk 12 Various 'turn-constructional units' are employed; e.g., turns can be projectedly 'one word long', or they can be sentential in length13 Repair mechanisms exist for dealing with turn-taking errors and violations; e.g., if two parties find themselves talking at the same time, one of them will stop prematurely, thus repairing the trouble 14 What parties say is not specified in advance8 — Harvey Sacks, Emanuel A. Schegloff, Gail Jefferson
  22. 22. Meet Alma Natural Conversation Framework
  23. 23. Natural Language != Natural Conversation
  24. 24. intent distance cuisine place Natural Language
  25. 25. Natural Conversation action pair granting request
  26. 26. dependency sequence closing hearing trouble dependency dependency understanding trouble base second part base first part
  27. 27. Conversational UX a working set of principles
  28. 28. Saying is doing
  29. 29. J: T's- tsuh beautiful day out isn't it? L: Yeh it's jus' gorgeous... CA A: God izn it dreary. (0.6) B: [Y'know I don't think- A: [.hh- It's warm though,
  30. 30. UX “A "signifier" is some sort of indicator, some signal in the physical or social world that can be interpreted meaningfully. Don Norman Cognitive Scientist
  31. 31. Recipient design
  32. 32. CA “By 'recipient design' we refer to a multitude of respects in which the talk by a party in a conversation is constructed or designed in ways which display an orientation and sensitivity to the particular other(s) who are the co-participants. Harvey Sacks, Emanuel A. Schegloff, Gail Jefferson Sociologists
  33. 33. B: Who's doing your remodel? A: Dave CA C: Who's doing your remodel? A: My neighbor across the street. 
 He's a contractor.
  34. 34. UX “[Human-centered design is] an approach that puts human needs, capabilities, and behavior first, then designs to accommodate those needs, capabilities, and ways of behaving. Don Norman Cognitive Scientist
  35. 35. Minimization
  36. 36. D: Who's doing your remodel? CA A: Dave D: Who? A: You know, my neighbor across the street. D: Oh! A: You had a beer with him? D: Right.
  37. 37. CA B: uh, yeah, I guess I'd like Mexican food A: Mañana's is on Fourth and Winchester. It's a great Mexican restaurant within walking distance. It gets five out of five stars. Would you like me to make a reservation for you at Mañana's? A: What kind of food would you like? Mexican Voice inputs are cheap, but voice outputs are expensive
  38. 38. UX Occam’s Razor1 Minimize Cognitive Load2 Eliminate Excise3
  39. 39. Understanding is interactional
  40. 40. CA
  41. 41. UX Mental Models1 Feedback2
  42. 42. Emotions describe actions
  43. 43. CA
  44. 44. CA
  45. 45. UX Visceral Behavioral Reflexive Norman’s 3 Levels of Emotional Design
  46. 46. The best input method 
 is situational
  47. 47. UX Context matters!
  48. 48. ibm.biz/conversational-ux
  49. 49. Bob Moore rjmoore@us.ibm.com Raphael Arar rarar@us.ibm.com Thank you.

×