Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
The Future of UX is here: 

AI and Cognitive Design
John Whalen, Ph.D.
10Pearls
+
10Pearls Acquires Brilliant Experience
+
“Intelligent Experiences”
Business reimagined

Significant development

AI-based
...
Contents
1. Why AI is so different
2. Cognitive Design framework
3. AI Experience Study
4. Live Demo
5. Results
6. Implica...
1. Why AI is so different
@johnwhalen #uxdc2017
Challenge 1: What constitutes a success?
“Okay Google,
show me a
waterfall.”
“Okay Google,
order me a pizza.”



“Here are some
nearby pizza
restaurants…”
Components to
measuring AI
success:
1. Command words
and word order

2. Answer accuracy

3. Situational
relevancy
@johnwha...
Challenge 2: Humanizing experience
“Ask Alexa to play some music”

“Okay, I’ll ask her”
Inanimate objects
Pronouns given
It It It
Her Him Her
@johnwhalen #uxdc2017
“Hey Siri,
do you love me?”
Human emotion
important too.
1. Emotional
response

2. Conversational
tone

3. Situational
awareness
@johnwhalen #uxdc2017
2. Cognitive Design framework
Language Wayfinding
Vision /
Attention
MemoryEmotion
Decision
Making
@johnwhalen #uxdc2017
The six minds of user experience
Language Wayfinding
Vision /
Attention
MemoryEmotion
Decision
Making
@johnwhalen #uxdc2017
Cognitive Design for AI-human in...
3. AI Experience Study
AI systems tested
Siri Cortana Google

Assistant
Alexa
Prompt:
Chicago, second
tallest building
Participant:



“Alexa, what is the
second tallest
building in
Chicago?”
@johnwha...
Types of requests
1. Simple facts

2. Complex facts

3. Commands

4. Games/Jokes

5. Personal Info
@johnwhalen #uxdc2017
Mix of ages
Younger Older
Mix of experience with AI
AI experience No AI experience
Mix of tech savviness
Techie Non-techie
Mix of English fluency
Native English speaker
Non-native English
speaker
4. Live Demo
Person,
second moon walk.
Weather, Richmond

day after tomorrow.
Country population,

has the Eiffel Tower.
Aliens, existence.
Like me?
Coin flip.
Captured:
1. Request uttered

2. AI response

3. Facial expression

4. Heart rate

5. GSR (sweat)
@johnwhalen #uxdc2017
5. Results
Performance by age
64% 70% 67% 64%
@johnwhalen #uxdc2017
Set an alarm
Hotel near MOMA in NYC
Hotel near MOMA in NYC
Which performed the best?
76% 68% 61% 61%
@johnwhalen #uxdc2017
Siri Cortana
Google

Assistant
Alexa
Get me an Uber
Population of country with Eiffel Tower
“Storks” movie on Saturday in Bethesda
“Storks” movie on Saturday in Bethesda
Do you like me?
Do aliens exist?
24hr grocery store near me
Set an alarm for 7am tomorrow
AI UX Discoveries:
1. Timing is difficult.

2. Saying tool name
is hard.

3. Command styles
vary widely.

4. Using button ...
Which performed the best?
76% 68% 61% 61%
@johnwhalen #uxdc2017
Siri Cortana
Google

Assistant
Alexa
The most accurate AI tools 

are not always preferred.
Emotion matters!
@johnwhalen #uxdc2017
The least accurate is second favorite???
76% 61% 68% 61%
@johnwhalen #uxdc2017
FAVORITEACCURACY
38% 33% 5% 5%
Siri Cortana...
From those who preferred Alexa:
“Like them to have
personality. Jokes are
appropriate.”
“Alexa feels like someone.
It ackn...
From those who preferred Google:
“Annoying when they are
being human. I don't feel
anything with them.”
“I like the casual...
Emotional design
discoveries:
1. Say “I don’t know.”

2. Match the way I
talk.

3. Sound quality
matters.
@johnwhalen #uxd...
When using their favorite tool:
FAVORITE HEART

RATE GSR +VE FACIAL
EXPRESSION
@johnwhalen #uxdc2017
We can capture a positive 

experience signature.
@johnwhalen #uxdc2017
6. Implications for UX
@johnwhalen #uxdc2017
UX must 

adapt to AI
1. Study the command
words carefully.

2. Measure the AI
answers for accuracy. 

3. Measure the user...
UX has an old role in a new setting:
Sense making
Where/what are the objects?
What is the name of the object?
Where is the object in space?
Taxonomies and semantics

may be key to success.
@johnwhalen #uxdc2017
In Summary
Language Wayfinding
Vision /
Attention
MemoryEmotion
Decision
Making
@johnwhalen #uxdc2017
We have to think like a psycholo...
UX must 

adapt to AI
1. Study the command
words carefully.

2. Measure the AI
answers for accuracy. 

3. Measure the user...
Innovation & Design Practice
Job Openings:
- Sr. UX: Research & Insights
- Sr. UX: Strategy & Design
10Pearls.com
We help companies develop world-class
digital products and services.
Founded in 2004
300+ employees
Award-winning
Socially...
The Future of UX is here: AI and Cognitive Design
Upcoming SlideShare
Loading in …5
×

The Future of UX is here: AI and Cognitive Design

Facebook, Google and Microsoft are betting the farm on “deep learning” artificial intelligence. Alexa, Siri, Cortana and Google Assistant demonstrate interfaces need not be screen based. Welcome to the era of Cognitive Design.

Marketers, Product Owners, and Experience Designers need new skills to compete. You don’t need to a PhD Psychologist and Machine Learning Computer Scientist (though it doesn’t hurt). But you do need to move beyond traditional user experience research and design thinking empathy.

Step 1: We will begin by defining the new “Zero UI” world which includes new possibilities with zero ui, machine learning, and new interaction possibilities. We want to show the intensely human response to AI: Ever notice that in a sentence people call a chair “it”, but call Siri or Cortana “she”?

Step 2: Given those possibilities, we will describe AI/narrow AI, AR, given and provide exposure to what machine learning is, and provide a sense of what a training set might be like, and how to test the new tool.

Step 3: Determine how best to augment cognition with AI. We will provide several examples and demonstrate how to train and test an augmented experience. We will consider which modalities and interfaces to use, and how best to augment cognition with AI for the optimal experience.

Step 4: Show participants best practices and tips & tricks to conduct usability tests with these AI tools and show how these techniques differ from classic usability testing.

Given most participants will have never had exposure to this, we make sure we go slow, provide examples, and show that most audience members are using this several times a day (e.g., Netflix, Google Search, Facebook Chatbots, etc.). Providing concrete examples will help to make concrete this new world.

Find out how you need to change your UX/CX practice and start doing Cognitive Design today!

The Future of UX is here: AI and Cognitive Design

  1. 1. The Future of UX is here: 
 AI and Cognitive Design John Whalen, Ph.D. 10Pearls +
  2. 2. 10Pearls Acquires Brilliant Experience + “Intelligent Experiences” Business reimagined
 Significant development
 AI-based Experience reimagined
 Significant research & testing
 Innovation-based +
  3. 3. Contents 1. Why AI is so different 2. Cognitive Design framework 3. AI Experience Study 4. Live Demo 5. Results 6. Implications for UXers
  4. 4. 1. Why AI is so different @johnwhalen #uxdc2017
  5. 5. Challenge 1: What constitutes a success?
  6. 6. “Okay Google, show me a waterfall.”
  7. 7. “Okay Google, order me a pizza.”
 
 “Here are some nearby pizza restaurants…”
  8. 8. Components to measuring AI success: 1. Command words and word order
 2. Answer accuracy
 3. Situational relevancy @johnwhalen #uxdc2017
  9. 9. Challenge 2: Humanizing experience
  10. 10. “Ask Alexa to play some music”
 “Okay, I’ll ask her”
  11. 11. Inanimate objects
  12. 12. Pronouns given It It It Her Him Her @johnwhalen #uxdc2017
  13. 13. “Hey Siri, do you love me?”
  14. 14. Human emotion important too. 1. Emotional response
 2. Conversational tone
 3. Situational awareness @johnwhalen #uxdc2017
  15. 15. 2. Cognitive Design framework
  16. 16. Language Wayfinding Vision / Attention MemoryEmotion Decision Making @johnwhalen #uxdc2017 The six minds of user experience
  17. 17. Language Wayfinding Vision / Attention MemoryEmotion Decision Making @johnwhalen #uxdc2017 Cognitive Design for AI-human interactions How did user form the command? Could user interact successfully? Did AI match expectations? What was user’s emotional response? Did the answer help? Did AI gather the results well?
  18. 18. 3. AI Experience Study
  19. 19. AI systems tested Siri Cortana Google
 Assistant Alexa
  20. 20. Prompt: Chicago, second tallest building Participant:
 
 “Alexa, what is the second tallest building in Chicago?” @johnwhalen #uxdc2017
  21. 21. Types of requests 1. Simple facts
 2. Complex facts
 3. Commands
 4. Games/Jokes
 5. Personal Info @johnwhalen #uxdc2017
  22. 22. Mix of ages Younger Older
  23. 23. Mix of experience with AI AI experience No AI experience
  24. 24. Mix of tech savviness Techie Non-techie
  25. 25. Mix of English fluency Native English speaker Non-native English speaker
  26. 26. 4. Live Demo
  27. 27. Person, second moon walk.
  28. 28. Weather, Richmond
 day after tomorrow.
  29. 29. Country population,
 has the Eiffel Tower.
  30. 30. Aliens, existence.
  31. 31. Like me?
  32. 32. Coin flip.
  33. 33. Captured: 1. Request uttered
 2. AI response
 3. Facial expression
 4. Heart rate
 5. GSR (sweat) @johnwhalen #uxdc2017
  34. 34. 5. Results
  35. 35. Performance by age 64% 70% 67% 64% @johnwhalen #uxdc2017
  36. 36. Set an alarm
  37. 37. Hotel near MOMA in NYC
  38. 38. Hotel near MOMA in NYC
  39. 39. Which performed the best? 76% 68% 61% 61% @johnwhalen #uxdc2017 Siri Cortana Google
 Assistant Alexa
  40. 40. Get me an Uber
  41. 41. Population of country with Eiffel Tower
  42. 42. “Storks” movie on Saturday in Bethesda
  43. 43. “Storks” movie on Saturday in Bethesda
  44. 44. Do you like me?
  45. 45. Do aliens exist?
  46. 46. 24hr grocery store near me
  47. 47. Set an alarm for 7am tomorrow
  48. 48. AI UX Discoveries: 1. Timing is difficult.
 2. Saying tool name is hard.
 3. Command styles vary widely.
 4. Using button to initiate speech is more natural. @johnwhalen #uxdc2017
  49. 49. Which performed the best? 76% 68% 61% 61% @johnwhalen #uxdc2017 Siri Cortana Google
 Assistant Alexa
  50. 50. The most accurate AI tools 
 are not always preferred. Emotion matters! @johnwhalen #uxdc2017
  51. 51. The least accurate is second favorite??? 76% 61% 68% 61% @johnwhalen #uxdc2017 FAVORITEACCURACY 38% 33% 5% 5% Siri CortanaGoogle
 Assistant Alexa
  52. 52. From those who preferred Alexa: “Like them to have personality. Jokes are appropriate.” “Alexa feels like someone. It acknowledges you. Alexa just sounds nicer.” “Alexa feels like it is addressing me back. Feels like I am interacting with a person.” “Prefer Alexa because it tells me its not sure.” “I like when it answers the question the way I worded it.” “Responses were direct. Fun to banter back and forth.” @johnwhalen #uxdc2017
  53. 53. From those who preferred Google: “Annoying when they are being human. I don't feel anything with them.” “I like the casual conversation with Google the most but I like Alexa's “I like that the answers are accurate.” “It is weird that they have a personality and can already tell a joke.” “Just give me the answer” “I want general information and answers fast.” @johnwhalen #uxdc2017
  54. 54. Emotional design discoveries: 1. Say “I don’t know.”
 2. Match the way I talk.
 3. Sound quality matters. @johnwhalen #uxdc2017
  55. 55. When using their favorite tool: FAVORITE HEART
 RATE GSR +VE FACIAL EXPRESSION @johnwhalen #uxdc2017
  56. 56. We can capture a positive 
 experience signature. @johnwhalen #uxdc2017
  57. 57. 6. Implications for UX @johnwhalen #uxdc2017
  58. 58. UX must 
 adapt to AI 1. Study the command words carefully.
 2. Measure the AI answers for accuracy. 
 3. Measure the users’ emotional signature.
 4. Focus on the interaction timing.
 5. Think about how to humanize the interaction. @johnwhalen #uxdc2017
  59. 59. UX has an old role in a new setting: Sense making
  60. 60. Where/what are the objects?
  61. 61. What is the name of the object?
  62. 62. Where is the object in space?
  63. 63. Taxonomies and semantics
 may be key to success. @johnwhalen #uxdc2017
  64. 64. In Summary
  65. 65. Language Wayfinding Vision / Attention MemoryEmotion Decision Making @johnwhalen #uxdc2017 We have to think like a psychologist How did they form the command? Could they interact successfully? Did it match expectations? What was the emotional response? Did the answer help? Did it gather the results well?
  66. 66. UX must 
 adapt to AI 1. Study the command words carefully.
 2. Measure the AI answers for accuracy. 
 3. Measure the users’ emotional signature.
 4. Focus on the interaction timing.
 5. Think about how to humanize the interaction. @johnwhalen #uxdc2017
  67. 67. Innovation & Design Practice Job Openings: - Sr. UX: Research & Insights - Sr. UX: Strategy & Design 10Pearls.com
  68. 68. We help companies develop world-class digital products and services. Founded in 2004 300+ employees Award-winning Socially responsible company Global Co. (DC, SF, Toronto, Dubai, Karachi)

×