Applying Science to Conversational UX Design

Raphael Arar
Raphael ArarDesigner & Researcher at IBM Research
Applying Science to Conversational UX Design
Bob Moore, Raphael Arar
IBM Research
Interfaces have come
a long way.
Applying Science to Conversational UX Design
© 2017 IBM Research
Applying Science to Conversational UX Design
Applying Science to Conversational UX Design
Applying Science to Conversational UX Design
Conversational Agents
Applying Science to Conversational UX Design
“
Jean Baudrillard
Sociologist
The sad thing about
artificial intelligence is
that it lacks artifice and
therefore intelligence.
Human
Conversation
APIs
Conversational Systems
Speech to Text Text to SpeechNatural Language

Understanding
Dialog Management
Human
Conversation
APIs
Conversational Systems
Text to SpeechSpeech to Text Natural Language

Understanding
Dialog Management
USER EXPERIENCE
DESIGN
ARCHITECTURE
INFORMATION
ARCHITECTURE
HUMAN-COMPUTER
INTERACTION
INTERACTION
DESIGN
VISUAL DESIGN
MECHANICAL
ENGINEERING
ELECTRICAL
ENGINEERING
CONTENT CREATION
(Text, Data, Graphics)
Signage
Info Viz
Navigation INTERFACE DESIGN
Ubicomp
Controls
Interactive
Environments
USABILITY
ENGINEERING
INDUSTRIAL
DESIGN
Dan Saffer, UX Designer
USER EXPERIENCE
DESIGN
ARCHITECTURE
INFORMATION
ARCHITECTURE
HUMAN-COMPUTER
INTERACTION
INTERACTION
DESIGN
CONVERSATION DESIGN
MECHANICAL
ENGINEERING
ELECTRICAL
ENGINEERING
CONTENT CREATION
(Text, Data, Graphics)
Signage
Ontology Management
Navigation CONVERSATIONAL UI
Ubicomp
Controls
Interactive
Environments
USABILITY
ENGINEERING
INDUSTRIAL
DESIGN
What’s the state-of-the-
art for conversational UI?
Web. 2017.
Applying Science to Conversational UX Design
© 2017 IBM Research
Web. 1996.
Applying Science to Conversational UX Design
Conversational UI. 2017.
Applying Science to Conversational UX Design
Conversational UI. ????
Human
Conversation
APIs
Conversational Systems
Text to SpeechSpeech to Text Natural Language

Understanding
Dialog Management
Conversational Systems
Dialog Management
Conversation
Analysis
Human
Conversation
What does natural
conversation sound like?
20 Des: What is the name?
21 Guy: Detweiler. D-e-t,
22 (1.2)
23 Guy: w-e,
24 (0.4)
25 Guy: i-l-e-r-.
26 (2.0)
27 Des: Foursome?
28 Guy: Yah.
29 (0.4)
30 Des: Electric carts?
31 (0.6)
32 Guy: Uh:::, n:no? I don’t
33 think so.
34 Des: Okay. We'll see yuh then,
35 Guy: Righto,
36 Des: Mm hm, Bye?
01 Des: G'morning. San Juan Hills
02 Country Club?
03 Guy: Guh morning. What’s-w-what
04 kind of a starting time
05 ken:: we get fer::hh
06 sometime this afternoon.
07 (0.7)
08 Guy: Any[time-
09 Des: [Oh:::, [let's see.
10 Guy: [Any time
11 tuhday.
12 Des: Two fordy. One, thirdy.
13 Guy: One thirty?
14 Des: Mm hm::?
15 Guy: One thirty.
16 (0.7)
17 Guy: .hh W'l at sounds like a
18 good time?
19 (0.4)
Not every voice or 

text interaction is a
conversation.
Speaker-change recurs, or at least occurs.1
Overwhelmingly, one party talks at a time.2
Occurrences of more than one speaker at a time are common, but brief.3
Transitions (from one turn to a next) with no gap and no overlap are common. Together with transitions characterized by slight gap

or slight overlap, they make up the vast majority of transitions
4
Turn order is not fixed, but varies.5
Turn size is not fixed, but varies.6
Length of conversation is not specified in advance.7
Relative distribution of turns is not specified in advance.9
Number of parties can vary.10
Talk can be continuous or discontinuous.11
Turn-allocation techniques are obviously used. A current speaker may select a next speaker (as when he addresses a question to

another party); or parties may self-select in starting to talk
12
Various 'turn-constructional units' are employed; e.g., turns can be projectedly 'one word long', or they can be sentential in length13
Repair mechanisms exist for dealing with turn-taking errors and violations; e.g., if two parties find themselves talking
at the same time, one of them will stop prematurely, thus repairing the trouble
14
What parties say is not specified in advance8
— Harvey Sacks, Emanuel A. Schegloff, Gail Jefferson
Meet Alma
Natural Conversation Framework
Natural Language !=
Natural Conversation
intent
distance
cuisine place
Natural Language
Natural Conversation
action pair
granting
request
dependency
sequence closing
hearing trouble
dependency
dependency
understanding trouble
base second part
base first part
Applying Science to Conversational UX Design
Applying Science to Conversational UX Design
Conversational UX
a working set of principles
Saying is doing
J: T's- tsuh beautiful day out
isn't it?
L: Yeh it's jus' gorgeous...
CA
A: God izn it dreary.
(0.6)
B: [Y'know I don't think-
A: [.hh- It's warm though,
UX
“A "signifier" is some sort
of indicator, some signal
in the physical or social
world that can be
interpreted meaningfully.
Don Norman
Cognitive Scientist
Recipient design
CA
“By 'recipient design' we refer to a
multitude of respects in which the
talk by a party in a conversation is
constructed or designed in ways
which display an orientation and
sensitivity to the particular other(s)
who are the co-participants.
Harvey Sacks, Emanuel A. Schegloff, Gail Jefferson
Sociologists
B: Who's doing your remodel?
A: Dave
CA
C: Who's doing your remodel?
A: My neighbor across the street. 

He's a contractor.
UX
“[Human-centered design is] an
approach that puts human needs,
capabilities, and behavior first, then
designs to accommodate those
needs, capabilities, and ways of
behaving.
Don Norman
Cognitive Scientist
Minimization
D: Who's doing your remodel?
CA
A: Dave
D: Who?
A: You know, my neighbor across the street.
D: Oh!
A: You had a beer with him?
D: Right.
CA
B: uh, yeah, I guess I'd like Mexican food
A: Mañana's is on Fourth and Winchester.
It's a great Mexican restaurant within
walking distance. It gets five out of
five stars. Would you like me to make
a reservation for you at Mañana's?
A: What kind of food would you like?
Mexican
Voice inputs are cheap, but voice
outputs are expensive
UX
Occam’s Razor1
Minimize Cognitive Load2
Eliminate Excise3
Understanding is
interactional
CA
UX
Mental Models1
Feedback2
Emotions describe
actions
CA
CA
UX
Visceral Behavioral Reflexive
Norman’s 3 Levels of Emotional Design
The best input method 

is situational
Applying Science to Conversational UX Design
UX
Context matters!
ibm.biz/conversational-ux
Bob Moore
rjmoore@us.ibm.com
Raphael Arar
rarar@us.ibm.com
Thank you.
1 of 58

Recommended

Chatbot interaction design — IxDA London July 2017 by
Chatbot interaction design — IxDA London July 2017Chatbot interaction design — IxDA London July 2017
Chatbot interaction design — IxDA London July 2017Boon Yew Chew
1.4K views31 slides
Chatbots, the design agenda by
Chatbots, the design agendaChatbots, the design agenda
Chatbots, the design agendaBoon Yew Chew
1.3K views37 slides
Heidi Young, Ozlo VP of Engineering, Seattle Interactive 2016 by
Heidi Young, Ozlo VP of Engineering, Seattle Interactive 2016Heidi Young, Ozlo VP of Engineering, Seattle Interactive 2016
Heidi Young, Ozlo VP of Engineering, Seattle Interactive 2016teamozlo
399 views28 slides
Chatbot and AI Design Principles by
Chatbot and AI Design PrinciplesChatbot and AI Design Principles
Chatbot and AI Design PrinciplesMauricio Perez
1.8K views34 slides
Deus UX Machina by
Deus UX MachinaDeus UX Machina
Deus UX MachinaSarah Hoaglan
1K views51 slides
Several bots are typing - Talk given at Nashville UX by
Several bots are typing - Talk given at Nashville UXSeveral bots are typing - Talk given at Nashville UX
Several bots are typing - Talk given at Nashville UXJustin Threlkeld
665 views66 slides

More Related Content

Viewers also liked

Elvia Vasconcelos, Experience Design for Chatbots by
Elvia Vasconcelos, Experience Design for ChatbotsElvia Vasconcelos, Experience Design for Chatbots
Elvia Vasconcelos, Experience Design for ChatbotsNaveed R
1K views60 slides
Designing self-service experiences by
Designing self-service experiencesDesigning self-service experiences
Designing self-service experiencesPebbleRoad
2.5K views48 slides
designing conversations: Conversational interfaces, Bot Interactions, Chatb... by
designing conversations: Conversational interfaces, Bot Interactions, Chatb...designing conversations: Conversational interfaces, Bot Interactions, Chatb...
designing conversations: Conversational interfaces, Bot Interactions, Chatb...Billy Choi
1.9K views55 slides
Ultimate Messenger Bot UX Guide by Sanuker by
Ultimate Messenger Bot UX Guide by SanukerUltimate Messenger Bot UX Guide by Sanuker
Ultimate Messenger Bot UX Guide by SanukerVicky Lee
910 views29 slides
Chatbot Design Canvas by
Chatbot Design CanvasChatbot Design Canvas
Chatbot Design CanvasAdreson Vilson Vita Sá
1.1K views1 slide
Chatbot: more UX and API design than code by
Chatbot: more UX and API design than codeChatbot: more UX and API design than code
Chatbot: more UX and API design than codeKwork Innovaatiot
12.2K views23 slides

Viewers also liked(6)

Elvia Vasconcelos, Experience Design for Chatbots by Naveed R
Elvia Vasconcelos, Experience Design for ChatbotsElvia Vasconcelos, Experience Design for Chatbots
Elvia Vasconcelos, Experience Design for Chatbots
Naveed R1K views
Designing self-service experiences by PebbleRoad
Designing self-service experiencesDesigning self-service experiences
Designing self-service experiences
PebbleRoad2.5K views
designing conversations: Conversational interfaces, Bot Interactions, Chatb... by Billy Choi
designing conversations: Conversational interfaces, Bot Interactions, Chatb...designing conversations: Conversational interfaces, Bot Interactions, Chatb...
designing conversations: Conversational interfaces, Bot Interactions, Chatb...
Billy Choi1.9K views
Ultimate Messenger Bot UX Guide by Sanuker by Vicky Lee
Ultimate Messenger Bot UX Guide by SanukerUltimate Messenger Bot UX Guide by Sanuker
Ultimate Messenger Bot UX Guide by Sanuker
Vicky Lee910 views
Chatbot: more UX and API design than code by Kwork Innovaatiot
Chatbot: more UX and API design than codeChatbot: more UX and API design than code
Chatbot: more UX and API design than code
Kwork Innovaatiot12.2K views

Similar to Applying Science to Conversational UX Design

Lecture1.ppt by
Lecture1.pptLecture1.ppt
Lecture1.pptAnsaRasheed3
13 views43 slides
What Does Conversational Information Access Exactly Mean and How to Evaluate It? by
What Does Conversational Information Access Exactly Mean and How to Evaluate It?What Does Conversational Information Access Exactly Mean and How to Evaluate It?
What Does Conversational Information Access Exactly Mean and How to Evaluate It?krisztianbalog
248 views78 slides
PACT - Un framework per progettare l'interazione con le nuove tecnologie by
PACT - Un framework per progettare l'interazione con le nuove tecnologiePACT - Un framework per progettare l'interazione con le nuove tecnologie
PACT - Un framework per progettare l'interazione con le nuove tecnologieRiva Giuseppe
7.8K views62 slides
Midwest km pugh conversational ai and ai for conversation 190809 by
Midwest km pugh conversational ai and ai for conversation 190809Midwest km pugh conversational ai and ai for conversation 190809
Midwest km pugh conversational ai and ai for conversation 190809Katrina (Kate) Pugh
416 views31 slides
2015-04-29 research seminar by
2015-04-29 research seminar2015-04-29 research seminar
2015-04-29 research seminarifi8106tlu
197 views35 slides
Design Process | Tool 01" personae" by
Design Process | Tool 01" personae"Design Process | Tool 01" personae"
Design Process | Tool 01" personae"Gessica Puri
750 views20 slides

Similar to Applying Science to Conversational UX Design(20)

What Does Conversational Information Access Exactly Mean and How to Evaluate It? by krisztianbalog
What Does Conversational Information Access Exactly Mean and How to Evaluate It?What Does Conversational Information Access Exactly Mean and How to Evaluate It?
What Does Conversational Information Access Exactly Mean and How to Evaluate It?
krisztianbalog248 views
PACT - Un framework per progettare l'interazione con le nuove tecnologie by Riva Giuseppe
PACT - Un framework per progettare l'interazione con le nuove tecnologiePACT - Un framework per progettare l'interazione con le nuove tecnologie
PACT - Un framework per progettare l'interazione con le nuove tecnologie
Riva Giuseppe7.8K views
Midwest km pugh conversational ai and ai for conversation 190809 by Katrina (Kate) Pugh
Midwest km pugh conversational ai and ai for conversation 190809Midwest km pugh conversational ai and ai for conversation 190809
Midwest km pugh conversational ai and ai for conversation 190809
2015-04-29 research seminar by ifi8106tlu
2015-04-29 research seminar2015-04-29 research seminar
2015-04-29 research seminar
ifi8106tlu197 views
Design Process | Tool 01" personae" by Gessica Puri
Design Process | Tool 01" personae"Design Process | Tool 01" personae"
Design Process | Tool 01" personae"
Gessica Puri750 views
Voice usability testing with WOZ methodology - UX SCOT 2019 by Abi Reynolds
Voice usability testing with WOZ methodology - UX SCOT 2019Voice usability testing with WOZ methodology - UX SCOT 2019
Voice usability testing with WOZ methodology - UX SCOT 2019
Abi Reynolds144 views
Conversational AI from an Information Retrieval Perspective: Remaining Challe... by krisztianbalog
Conversational AI from an Information Retrieval Perspective: Remaining Challe...Conversational AI from an Information Retrieval Perspective: Remaining Challe...
Conversational AI from an Information Retrieval Perspective: Remaining Challe...
krisztianbalog266 views
Visual thinking for service design — CanUX November 2016 by Boon Yew Chew
Visual thinking for service design — CanUX November 2016Visual thinking for service design — CanUX November 2016
Visual thinking for service design — CanUX November 2016
Boon Yew Chew2K views
Superheroes SXSW 2013 by Yvonne So
Superheroes SXSW 2013Superheroes SXSW 2013
Superheroes SXSW 2013
Yvonne So4.1K views
Bringing the User Into the Process with Design Thinking by Jacqueline L. Frank
Bringing the User Into the Process with Design ThinkingBringing the User Into the Process with Design Thinking
Bringing the User Into the Process with Design Thinking
Design thinking for accessible user experiences - UX Scotland2014 by David Sloan
Design thinking for accessible user experiences - UX Scotland2014Design thinking for accessible user experiences - UX Scotland2014
Design thinking for accessible user experiences - UX Scotland2014
David Sloan3.4K views
Ux scot voice usability testing with woz - ar and sf - june 2019 by User Vision
Ux scot   voice usability testing with woz - ar and sf  - june 2019Ux scot   voice usability testing with woz - ar and sf  - june 2019
Ux scot voice usability testing with woz - ar and sf - june 2019
User Vision400 views
Communication-Collaboration (1).pptx by IanJamesCalanza
Communication-Collaboration (1).pptxCommunication-Collaboration (1).pptx
Communication-Collaboration (1).pptx
IanJamesCalanza32 views
@SCtechhub meetup #4 Design Thinking by sctechhub
@SCtechhub meetup #4 Design Thinking@SCtechhub meetup #4 Design Thinking
@SCtechhub meetup #4 Design Thinking
sctechhub1.6K views
Aidas pasilis. Oh my Agile! by Agile Lietuva
Aidas pasilis. Oh my Agile!Aidas pasilis. Oh my Agile!
Aidas pasilis. Oh my Agile!
Agile Lietuva473 views
Use Your Words: Content Strategy to Influence Behavior by Liz Danzico
Use Your Words: Content Strategy to Influence BehaviorUse Your Words: Content Strategy to Influence Behavior
Use Your Words: Content Strategy to Influence Behavior
Liz Danzico5.1K views

Recently uploaded

Design System.pdf by
Design System.pdfDesign System.pdf
Design System.pdfAtiqur Rahaman
19 views20 slides
StratPlanning Manual 220713.pdf by
StratPlanning Manual 220713.pdfStratPlanning Manual 220713.pdf
StratPlanning Manual 220713.pdfLakewalk Media
19 views43 slides
Oregon Ducks 4 Spencer Webb Hoodie by
Oregon Ducks 4 Spencer Webb HoodieOregon Ducks 4 Spencer Webb Hoodie
Oregon Ducks 4 Spencer Webb Hoodiebrandshop1
13 views1 slide
217 Drive - All on upper.pptx by
217 Drive - All on upper.pptx217 Drive - All on upper.pptx
217 Drive - All on upper.pptxvidstor282
15 views7 slides
Free World aids day Template from Best presentation design agency by
Free World aids day Template from Best presentation design agencyFree World aids day Template from Best presentation design agency
Free World aids day Template from Best presentation design agencyslideceotemplates
8 views10 slides
Benzodiazepines--Medicinal Chemistry by
Benzodiazepines--Medicinal ChemistryBenzodiazepines--Medicinal Chemistry
Benzodiazepines--Medicinal ChemistryNarminHamaaminHussen
7 views32 slides

Recently uploaded(20)

StratPlanning Manual 220713.pdf by Lakewalk Media
StratPlanning Manual 220713.pdfStratPlanning Manual 220713.pdf
StratPlanning Manual 220713.pdf
Lakewalk Media19 views
Oregon Ducks 4 Spencer Webb Hoodie by brandshop1
Oregon Ducks 4 Spencer Webb HoodieOregon Ducks 4 Spencer Webb Hoodie
Oregon Ducks 4 Spencer Webb Hoodie
brandshop113 views
217 Drive - All on upper.pptx by vidstor282
217 Drive - All on upper.pptx217 Drive - All on upper.pptx
217 Drive - All on upper.pptx
vidstor28215 views
Free World aids day Template from Best presentation design agency by slideceotemplates
Free World aids day Template from Best presentation design agencyFree World aids day Template from Best presentation design agency
Free World aids day Template from Best presentation design agency
IEC 600068-2-39 ENVIROMENT TESTING COMBINED TEMPERATURE LOW HUMIDTY.pdf by NirmalanGanapathy1
IEC 600068-2-39 ENVIROMENT TESTING COMBINED TEMPERATURE LOW HUMIDTY.pdfIEC 600068-2-39 ENVIROMENT TESTING COMBINED TEMPERATURE LOW HUMIDTY.pdf
IEC 600068-2-39 ENVIROMENT TESTING COMBINED TEMPERATURE LOW HUMIDTY.pdf
Sudden Deafness Design Document by wyfangherman
Sudden Deafness Design DocumentSudden Deafness Design Document
Sudden Deafness Design Document
wyfangherman49 views
The Report is Dead, Long Live the Report ! Communicating Usability Research F... by Centralis
The Report is Dead, Long Live the Report ! Communicating Usability Research F...The Report is Dead, Long Live the Report ! Communicating Usability Research F...
The Report is Dead, Long Live the Report ! Communicating Usability Research F...
Centralis5 views
Designing Effective AR Experiences by Kumar Ahir
Designing Effective AR ExperiencesDesigning Effective AR Experiences
Designing Effective AR Experiences
Kumar Ahir5 views
TISFLEET WEB DESIGN PROJECT by Rabius Sany
TISFLEET WEB DESIGN PROJECTTISFLEET WEB DESIGN PROJECT
TISFLEET WEB DESIGN PROJECT
Rabius Sany41 views

Applying Science to Conversational UX Design