Voice User Interface Design - Big Design 2017

Crispin Reedy
Crispin ReedyVoice User Experience Designer at Versay
© 2017 Versay Solutions
Voice User Interface
Design:
Skills, Actions, And The Future
Crispin Reedy, Versay Solutions
@crispinTX crispinreedy.com
#BigD17
© 2017 Versay Solutions
Voice User Interface
Design:
Skills, Actions, And The Future
Disclaimer: This session was
NOT sponsored by Dominos
© 2017 Versay Solutions
• Voice User Interface Designer
• 15+ years in the field
• Former coder; got interested in UX
• President of the Association for Voice
Interaction Design
• Consultant for Versay Solutions
@crispinTX
crispinreedy.com
© 2017 Versay Solutions
Session Description
• Amazon Skills for Alexa, Google Actions for Home
– Should your company build a conversational
voice interface for one of these systems, and if
so, how?
• What are the differences between a voice user
interface and other types of UIs?
• What types of skills does a VUI designer need?
• What are some best practices for these VUIs?
• You’ll walk away with answers to the questions
“If, Why, and How” you might choose to explore
this interesting new area of design.
© 2017 Versay Solutions
Easy Answer To #1
• If your company is involved in home
automation:
• Mostly likely Yes, and Yesterday
• Although how you do it will depend on your
platform
• More on that later!
• Everyone else
• Let’s keep talking!
© 2017 Versay Solutions
Basic Terms
© 2017 Versay Solutions
Terms & Technologies
•Speech Recognition
•Natural Language Understanding
•Voice Verification (Biometrics)
•Text to Speech
© 2017 Versay Solutions
Speech Recognition “ASR”
“See the cat.”
© 2017 Versay Solutions
Natural Language Understanding
•Extracting meaning from natural text
“Hello, yes,
I’d like to
pay my
water bill.
Can you
help me with
that?
Intent =
BillPay
Entity
(Bill Type) =
Water
© 2017 Versay Solutions
Voice Verification
“My voice is
my password.”
“Authenticated.
Welcome, Mr.
Smith.”
✓
Text To Speech
© 2017 Versay Solutions
Speech Recognition
• Hands-free command /
control
• Dictation
• Input text
• Small form factor
device, etc.
Text To Speech
• Output text dynamically
• Respond to input
• Useful when no
display is available
Natural Language
Understanding
• Necessary for all
language-based input
• Extract meaning
• Parse large volumes of
text
Voice Verification
• Security
ASR
Application
Data
• Sign-In
• Interaction
• Request
• Action
• Meaning
• Access Data
• Output
TTS
NLU
Voice
prints
Verifi-
cation
© 2017 Versay Solutions
Speech Technology
Today
© 2017 Versay Solutions
Speech Agents, Apps, and APIs
Speech Agents:
• Amazon Alexa
• Echo, Dot, Echo Show
• Google Assistant
• Pixel, Android, Google Home, iPhone app
• Apple’s Siri
• iPhone, iPad, MacOS (Sierra), AppleTV
• Microsoft’s Cortana
• Windows 10, Windows Phone, Xbox, iPhone app
• Samsung’s Bixby
• Galaxy S8, Family Hub 2.0 Fridge
© 2017 Versay Solutions
Speech Agents, Apps, and APIs
Speech Agents can be extended with
“Voice Apps”
• Alexa Skills
• Google Actions
• SiriKit
• Cortana SDK
© 2017 Versay Solutions
Speech Agents, Apps, and APIs
Agent capabilities and apps are somewhat
determined by:
• Platform: Device
• Screen, keyboard, phone, mics, etc.
• Environment: Web site, apps that interact with
the agent
• Ecosystem: Underlying connections, technical
partnerships
© 2017 Versay Solutions
Platforms
© 2017 Versay Solutions
Environment
Google “Actions” or
“Apps”
• Curated
• Direct vs.
Conversational
Siri - Works via apps
Order Uber Order Lyft
© 2017 Versay Solutions New York Times
© 2017 Versay Solutions
Speech Agents, Apps, and APIs
APIs: Allow you access to the underlying
technology
• Amazon
• AVS (Alexa Voice Service) Create an “Alexa” on your
own device
• Amazon Lex, Amazon Polly
• Google
• Cloud Speech API
• API.ai
• Apple
• Apple Speech Framework
• Microsoft
• Bing Speech API
Ecobee Smart
Thermostat
© 2017 Versay Solutions
Use Cases
Use Case “Bakeoff” from Tech Insider
•Travel
•Email
•Messaging
•Sports
•Music
•Weather
•Calendar
•Social
• Translation
• Basic tasks
• General knowledge
• Personality
http://www.businessinsider.com/siri-vs-google-assistant-cortana-alexa-2016-11/
© 2017 Versay Solutions
Use Case “Bakeoff” from Tech Insider
• “wildly finicky when it comes to phrasing.”
• “Each assistant still feels like a fragile, thinly veiled
web of loosely connected services — because that's
what they are.”
• “incredibly uncomfortable to speak to an inanimate
thing in public.”
• “In Google Assistant's case, normalizing the need to
call on a brand ("OK Google") whenever you need a
hand is Orwellian.”
• “None of these things
are at a place I could
comfortably call
"good.””
© 2017 Versay Solutions
Personal Assistant vs. Home Assistant
The Google Pixel XL.
Hollis Johnson/Business Insider
Google.com
© 2017 Versay Solutions
Personal Assistant vs. Home Assistant
© 2017 Versay Solutions
Getting Specific With
Alexa
© 2017 Versay Solutions
“Layers” of Alexa
•Alexa Native Capabilities
•Alexa Skills
•Alexa Voice Services
© 2017 Versay Solutions
“Layers” of Alexa
• Alexa Native Capabilities
• Come out of the box
• Require Alexa wake word (can be changed)
• Alexa Skills
• Alexa’s “Extensions” or “Add-Ons”
• Designed for and deployed on Echo Device
• Skills must be downloaded to Echo
• Require Alexa wake word + Skill name
• Alexa Voice Services
• Add Alexa voice control to your own device
© 2017 Versay Solutions
Alexa “Native” Capabilities
Alexa, what’s 3 + 5?
Alexa, set an alarm
for 3 am.
Alexa, set a thirty
second timer.
Alexa, what’s the
weather?
Note:
Mix of TTS &
Pre-Recorded
Audio
Note:
“Hint”
© 2017 Versay Solutions
Design Considerations
•Proactive “Hints”
• Similar to “Hover Help” or “Tool Tip”
• But less avoidable!
• Pro: Can teach user about other capabilities
• Con: Can be annoying!
• Guideline: If used, be sparing
• Develop rules for when and how frequently to
offer
© 2017 Versay Solutions Amazon.com
Native & Skill
Skill
Skill
Skill
Native & Skill
Alexa Skills
© 2017 Versay Solutions Source: David Attwater, EIG Inc.
© 2017 Versay Solutions Amazon.com
Alexa Skills
Amazon.comAmazon.com
© 2017 Versay Solutions
Amazon.com
© 2017 Versay Solutions
Design Considerations
• Invoking Skills:
• Alexa, open Oprah Magazine
• Alexa, order a pizza from Domino’s
• Alexa, ask Cook Reference what’s the
safe temperature for chicken
• Syntax:
Open <skill>
Ask <skill> for (about, to, with, etc.)
<action>
Ask <skill> <question>
Also: Search, Tell, Talk to, Launch, Start,
Resume, Run, Load, Begin
Oprah Magazine
© 2017 Versay Solutions
Design Considerations
•Skills can be “installed” on the
fly
•If the user knows the name of
the skill
•Skills that require account
information will need extra
steps
Cook Reference
Domino’s
© 2017 Versay Solutions
Alexa App + Linking
© 2017 Versay Solutions
Design Considerations
•Managing access to skills may become
difficult or confusing.
© 2017 Versay Solutions
Design Considerations
•Attention (or lack of attention!) to
technical details can become “deal-
killing” part of overall experience
Domino’s
© 2017 Versay Solutions
Really?
Dominos.com
© 2017 Versay Solutions
No
Dominos.com
© 2017 Versay Solutions
Design Considerations
• Confirmation
• What’s the phone number?
• 214-555-1235
• You said 214-555-1235. Is that correct?
• Yes
• Note: System confirmed the phone number but
not the address
• Was the address really correct?
© 2017 Versay Solutions Dominos.com
© 2017 Versay Solutions
Design Considerations
• “Would you like to place your Easy
Order, reorder your most recent
order, or start a new order?”
• If I’m not logged into my account on
the Alexa app, options 1 and 2 don’t
make much sense.
• “Would you like” is ambiguous – could
be used for Yes / No questions or for
multi-item questions
• First part of the sentence runs into the
choices
• Reuse of the word “order” just seems
odd (but may be unavoidable).
• Could have used more pauses (SSML)
Domino’s
© 2017 Versay Solutions
Design Considerations: SSML
• Speech Synthesis Markup Language
• Can control the way your TTS playback sounds
• Very important if your output is mostly TTS
• Which is true of all most platforms
• Should be supported by all types of TTS
engine
• Amazon has platform specific options
• Plan on using it to fine tune your audio output
© 2017 Versay Solutions
New Prompts & SSML Examples
• Note: TTS Samples with SSML created
with Amazon Polly, not Alexa
• “You can: Place your easy order.
Reorder your most recent order. Or,
start a *new* order.”
• You can: <break time="500ms"/>Place your easy order,
<break time="500ms"/> Reorder your <emphasis
level="moderate">most recent</emphasis> order, <break
time="500ms"/> Or, start a <emphasis
level="strong">new</emphasis> order.
• Placing an order, great! Choose from:
My easy order. My most recent. Or,
start a *new* order.
• <speak>Placing an order. <prosody
pitch="high">Great!</prosody> Choose from: My easy
order. My most recent. Or, start a <emphasis> <prosody
pitch="high">new</prosody> </emphasis>
order.</speak>
Domino’s
© 2017 Versay Solutions
Still Trying To Order That Pizza
• Start of the interaction has changed!
• Probably due to login
• “Would you like to place an order, or
track an order?”
• What just happened!!!?
•System was expecting me to say
“Start a new order” and I only said
“New Order.”
Domino’s
© 2017 Versay Solutions
Design Considerations
• Make sure your input grammar covers all
possible logical utterances (what user can say)
• Don’t leave this stuff up to the programmers!
• Provide examples of coverage
• Coverage should match prompts
• Use some kind of markup to show coverage
• [] optional
• () grouping
• | or
• “Would you like to place your Easy Order,
reorder your most recent order, or start a new
order?”
• [place] [my | an] Easy Order
• [reorder] [my] most recent [order]
• [start a] new [order]
© 2017 Versay Solutions
Design Considerations
•Reprompts:
• What do you do when you didn’t
understand what the caller said?
• Probably don’t want to say “Sorry”
• This can be annoying
• But you CAN rephrase the prompt to make it
different
• Using the same prompt gives the user a sense
that something has gone wrong
© 2017 Versay Solutions
Pizza Pizza Pizza
• Hey you didn’t really need to
explain about the phone number
since I saved it but OK….
• Address has been saved to profile,
great!
• And then boom
Domino’s
© 2017 Versay Solutions
With Speech, you need to spend
a lot more time thinking about
what happens when things go
wrong.
© 2017 Versay Solutions
I Didn’t Really Want to Order Pizza
But By Now I Am Hungry
And So Is Somebody Else
• Note “Easy Order” and Credit Card
cannot be set up on the website
unless you’re actually placing an order.
• Give people enough time to talk!
• There’s that grammar coverage issue
again
• Bell pepper = Green pepper
• What synonyms is your user likely to say?
• At some point couldn’t you just give me a list?
• Notice how they screwed up the article +
the item “… adding a parmesan bread
twists”
Meow
Domino’s
© 2017 Versay Solutions
© 2017 Versay Solutions
© 2017 Versay Solutions
Design Considerations
• Confirm and correct
• “Do you want to add anything else?”
• “Yes, I want to add peppers.”
• Disambiguation
• “Olives”
• “Ok, we have two kinds of olives. Black olives, or
green olives.”
• A Voice User Interface design is a time-based
interface
• As a designer concerned with user experience
you’re going to be involved in things (such as
pauses) which may not occur to you
© 2017 Versay Solutions
How Did Google Home Do?
•“OK Google, Order Dominos”
• “There are stores at….”
• Had to go find the right “App Name” online
•“OK Google, Talk to Dominos”
• “You can link to your Domino’s account…”
• Had a terrible time finding the “Google
Apps.”
© 2017 Versay Solutions
How Did Google Home Do?
•Menu worked!
• System did not recognize “Ham” (Should
offer list of ingredients)
• System became very laggy
© 2017 Versay Solutions
How Did Google Home Do?
• Edited for time
• Original was 3:35
• This is 2:15
• Use of “Dom” persona and male voice
• “Hand off”
• Playback of address:
• Alexa: “Eighty seven twenty three”
• Google: “Eight thousand seven hundred twenty
three”
• Same issue with “twists”
• “Your day just got cheesier”
© 2017 Versay Solutions
Design Considerations
•Discoverability
• “OK Google, Order Dominos”
•Persona
• Google Home has more control over the
voice
• Branding considerations – “Dom” name and
male TTS
•Playback of Dynamic Data
• Attention to detail – don’t trust the platform
to do it the way you want it
© 2017 Versay Solutions
Design Considerations
Maintaining State:
•Between dialogs
• “Who is Seth McFarlane?”
• “Seth McFarlane is…”
• “When’s his birthday?”
• “I’m not sure what you’re talking about.”
•From session to session
Oprah Magazine
© 2017 Versay Solutions
Home Automation
•Onboarding issues are very similar to
“Skills,” but there is an additional layer of
complexity
• Companies are working to improve the
experience
• After setup, you get a lot of bang for the
buck
© 2017 Versay Solutions
“Computer,
turn on the
library lights”
© 2017 Versay Solutions
TP Link
© 2017 Versay Solutions Amazon
© 2017 Versay Solutions
Design Considerations: Summary
• Managing access to Skills
(App, Store)
• Managing the Onboarding
Experience
• Discoverability
• Invoking Skills
• Hints
• Confirmation
• Asking Yes/No Questions vs.
Multi-Item Questions
• SSML
• Silences
• Reprompting
• Coverage (prompt vs.
possible input)
• Managing technical
errors
• Timing and Timeouts
• Article matching the
noun
• Confirm and Correct
• Disambiguation
• Persona
• Playback of Dynamic
Data
• Maintaining State
© 2017 Versay Solutions
What Makes a Good VUI Designer?
•Concern with the overall experience
• All of the channels that go into making up
how something happens
•Attention to “small” technical details
• Pauses
• SSML
•Writing skills!
• Dialog, not tech doc
• English majors, screenwriters
© 2017 Versay Solutions
Session Description
• Amazon Skills for Alexa, Google Actions for Home
– Should your company build a conversational
voice interface for one of these systems, and if
so, how?
• What are the differences between a voice user
interface and other types of UIs? ✔
• What types of skills does a VUI designer need? ✔
• What are some best practices for these VUIs? ✔
• You’ll walk away with answers to the questions
“If, Why, and How” you might choose to explore
this interesting new area of design.
✔
© 2017 Versay Solutions
If, Why, How
•What are you trying to build?
•Existing guidelines / research
•User testing is key
• Especially if you’re trying to do something
complicated
© 2017 Versay Solutions
If, Why, How: Beyond Skills
Write an app (skill) for
an agent such as
Google Assistant /
Alexa
Use cloud APIs to add
ASR / NLU to your app /
device / page / gadget
Download software and
use full-featured
capabilities for more robust
recognition on a specific
device
Build your own
© 2017 Versay Solutions
If, Why, How: What’s the Use Case?
•Enabling application
• User can’t do it any other way
• New tasks
•Enhancing application
• User can do it now
• But speech makes it better
• Faster
• Safer
© 2017 Versay Solutions
API-Based
Device-
Based
Roll Your
Own /
Open-
Source
•Flexibility
•Power
•Customization
•Time
•Difficulty
© 2017 Versay Solutions
Existing Guidelines / Research
• Caveat: Best practices evolved in one
modality (e.g. voice-only) may not apply the
same way in another (e.g. combined voice +
touch)
• But they could be adapted
• Association for Voice Interaction Design
(AVIxD.org)
• Wiki
• Peer-Reviewed Journal
• Virtual “Brown Bags”
• Academic Sources, Books
© 2017 Versay Solutions
AVIxD.org
CUI Working Group is actively recruiting!
© 2017 Versay Solutions
@crispinTX
Crispin Reedy
Thank You!
1 of 76

Recommended

Easily emulating full systems on amazon fpg as by
Easily emulating full systems on amazon fpg asEasily emulating full systems on amazon fpg as
Easily emulating full systems on amazon fpg asRISC-V International
116 views45 slides
Android Crash analysis and The Dalvik Garbage collector – Tools and Tips by
Android Crash analysis and The Dalvik Garbage collector – Tools and TipsAndroid Crash analysis and The Dalvik Garbage collector – Tools and Tips
Android Crash analysis and The Dalvik Garbage collector – Tools and TipsDroidConTLV
13.7K views39 slides
Deep Dive on AWS Lambda by
Deep Dive on AWS LambdaDeep Dive on AWS Lambda
Deep Dive on AWS LambdaAmazon Web Services
6.3K views38 slides
Deep Dive on Amazon DynamoDB by
Deep Dive on Amazon DynamoDBDeep Dive on Amazon DynamoDB
Deep Dive on Amazon DynamoDBAmazon Web Services
763 views61 slides
AWS Kinesis Streams by
AWS Kinesis StreamsAWS Kinesis Streams
AWS Kinesis StreamsFernando Rodriguez
2.8K views21 slides
DevOps on AWS by
DevOps on AWSDevOps on AWS
DevOps on AWSAmazon Web Services
3.3K views77 slides

More Related Content

What's hot

Artificial Intelligence on AWS by
Artificial Intelligence on AWSArtificial Intelligence on AWS
Artificial Intelligence on AWSAmazon Web Services
1.6K views37 slides
DevOps on AWS by
DevOps on AWSDevOps on AWS
DevOps on AWSAmazon Web Services
4.8K views69 slides
Azure Monitoring Overview by
Azure Monitoring OverviewAzure Monitoring Overview
Azure Monitoring Overviewgjuljo
7K views37 slides
AWS Certified Cloud Practitioner Course S1-S6 by
AWS Certified Cloud Practitioner Course S1-S6AWS Certified Cloud Practitioner Course S1-S6
AWS Certified Cloud Practitioner Course S1-S6Neal Davis
10.7K views173 slides
Introduction to Amazon Web Services by
Introduction to Amazon Web ServicesIntroduction to Amazon Web Services
Introduction to Amazon Web ServicesAmazon Web Services
146.9K views46 slides
Amazon Kinesis by
Amazon KinesisAmazon Kinesis
Amazon KinesisAmazon Web Services
6.2K views49 slides

What's hot(20)

Azure Monitoring Overview by gjuljo
Azure Monitoring OverviewAzure Monitoring Overview
Azure Monitoring Overview
gjuljo7K views
AWS Certified Cloud Practitioner Course S1-S6 by Neal Davis
AWS Certified Cloud Practitioner Course S1-S6AWS Certified Cloud Practitioner Course S1-S6
AWS Certified Cloud Practitioner Course S1-S6
Neal Davis10.7K views
Introduction to Azure monitor by Praveen Nair
Introduction to Azure monitorIntroduction to Azure monitor
Introduction to Azure monitor
Praveen Nair962 views
Android Internals by Opersys inc.
Android InternalsAndroid Internals
Android Internals
Opersys inc.7.8K views
Policy as Code: IT Governance With HashiCorp Sentinel by Mitchell Pronschinske
Policy as Code: IT Governance With HashiCorp SentinelPolicy as Code: IT Governance With HashiCorp Sentinel
Policy as Code: IT Governance With HashiCorp Sentinel
Smart Home Skill API: Connect Any Device to Alexa & Control Any Feature (ALX3... by Amazon Web Services
Smart Home Skill API: Connect Any Device to Alexa & Control Any Feature (ALX3...Smart Home Skill API: Connect Any Device to Alexa & Control Any Feature (ALX3...
Smart Home Skill API: Connect Any Device to Alexa & Control Any Feature (ALX3...
Amazon Web Services1.4K views
Use Amazon Rekognition to Build a Facial Recognition System by Amazon Web Services
Use Amazon Rekognition to Build a Facial Recognition SystemUse Amazon Rekognition to Build a Facial Recognition System
Use Amazon Rekognition to Build a Facial Recognition System
Architecting for the Cloud using NetflixOSS - Codemash Workshop by Sudhir Tonse
Architecting for the Cloud using NetflixOSS - Codemash WorkshopArchitecting for the Cloud using NetflixOSS - Codemash Workshop
Architecting for the Cloud using NetflixOSS - Codemash Workshop
Sudhir Tonse39.6K views

Similar to Voice User Interface Design - Big Design 2017

Conversational User Interfaces, Past and Future by
Conversational User Interfaces, Past and FutureConversational User Interfaces, Past and Future
Conversational User Interfaces, Past and FutureCrispin Reedy
1.9K views58 slides
DEV206_Life of a Code Change to a Tier 1 Service by
DEV206_Life of a Code Change to a Tier 1 ServiceDEV206_Life of a Code Change to a Tier 1 Service
DEV206_Life of a Code Change to a Tier 1 ServiceAmazon Web Services
481 views37 slides
MCL301_Building a Voice-Enabled Customer Service Chatbot Using Amazon Lex and... by
MCL301_Building a Voice-Enabled Customer Service Chatbot Using Amazon Lex and...MCL301_Building a Voice-Enabled Customer Service Chatbot Using Amazon Lex and...
MCL301_Building a Voice-Enabled Customer Service Chatbot Using Amazon Lex and...Amazon Web Services
1.5K views45 slides
Alexa State of the Science - ALX321 - 2h amazonwebservices Deep Dive into Ama... by
Alexa State of the Science - ALX321 - 2h amazonwebservices Deep Dive into Ama...Alexa State of the Science - ALX321 - 2h amazonwebservices Deep Dive into Ama...
Alexa State of the Science - ALX321 - 2h amazonwebservices Deep Dive into Ama...Amazon Web Services
354 views15 slides
NEW LAUNCH! Deploy Alexa In Your Organization with Alexa for Business - BAP20... by
NEW LAUNCH! Deploy Alexa In Your Organization with Alexa for Business - BAP20...NEW LAUNCH! Deploy Alexa In Your Organization with Alexa for Business - BAP20...
NEW LAUNCH! Deploy Alexa In Your Organization with Alexa for Business - BAP20...Amazon Web Services
1K views32 slides
ARC214_Addressing Your Business Needs with AWS by
ARC214_Addressing Your Business Needs with AWSARC214_Addressing Your Business Needs with AWS
ARC214_Addressing Your Business Needs with AWSAmazon Web Services
732 views25 slides

Similar to Voice User Interface Design - Big Design 2017(20)

Conversational User Interfaces, Past and Future by Crispin Reedy
Conversational User Interfaces, Past and FutureConversational User Interfaces, Past and Future
Conversational User Interfaces, Past and Future
Crispin Reedy1.9K views
DEV206_Life of a Code Change to a Tier 1 Service by Amazon Web Services
DEV206_Life of a Code Change to a Tier 1 ServiceDEV206_Life of a Code Change to a Tier 1 Service
DEV206_Life of a Code Change to a Tier 1 Service
MCL301_Building a Voice-Enabled Customer Service Chatbot Using Amazon Lex and... by Amazon Web Services
MCL301_Building a Voice-Enabled Customer Service Chatbot Using Amazon Lex and...MCL301_Building a Voice-Enabled Customer Service Chatbot Using Amazon Lex and...
MCL301_Building a Voice-Enabled Customer Service Chatbot Using Amazon Lex and...
Amazon Web Services1.5K views
Alexa State of the Science - ALX321 - 2h amazonwebservices Deep Dive into Ama... by Amazon Web Services
Alexa State of the Science - ALX321 - 2h amazonwebservices Deep Dive into Ama...Alexa State of the Science - ALX321 - 2h amazonwebservices Deep Dive into Ama...
Alexa State of the Science - ALX321 - 2h amazonwebservices Deep Dive into Ama...
NEW LAUNCH! Deploy Alexa In Your Organization with Alexa for Business - BAP20... by Amazon Web Services
NEW LAUNCH! Deploy Alexa In Your Organization with Alexa for Business - BAP20...NEW LAUNCH! Deploy Alexa In Your Organization with Alexa for Business - BAP20...
NEW LAUNCH! Deploy Alexa In Your Organization with Alexa for Business - BAP20...
NEW LAUNCH! Bring Alexa to Work! Voice-enable Your Organization with Alexa fo... by Amazon Web Services
NEW LAUNCH! Bring Alexa to Work! Voice-enable Your Organization with Alexa fo...NEW LAUNCH! Bring Alexa to Work! Voice-enable Your Organization with Alexa fo...
NEW LAUNCH! Bring Alexa to Work! Voice-enable Your Organization with Alexa fo...
729 Solutions Helps Connect The Dots - Our Services At A Glance by Iron Mountain
729 Solutions Helps Connect The Dots - Our Services At A Glance729 Solutions Helps Connect The Dots - Our Services At A Glance
729 Solutions Helps Connect The Dots - Our Services At A Glance
Iron Mountain 76 views
Tools of a Successful Force.com Developer by Joshua Hoskins
Tools of a Successful Force.com DeveloperTools of a Successful Force.com Developer
Tools of a Successful Force.com Developer
Joshua Hoskins1.2K views
IOT328_Building an AWS IoT-Enabled Drink Dispenser by Amazon Web Services
IOT328_Building an AWS IoT-Enabled Drink DispenserIOT328_Building an AWS IoT-Enabled Drink Dispenser
IOT328_Building an AWS IoT-Enabled Drink Dispenser
IOT328_Building an AWS IoT-Enabled Drink Dispenser by Amazon Web Services
IOT328_Building an AWS IoT-Enabled Drink DispenserIOT328_Building an AWS IoT-Enabled Drink Dispenser
IOT328_Building an AWS IoT-Enabled Drink Dispenser
Building an AWS IoT-Enabled Drink Dispenser - IOT328 - re:Invent 2017 by Amazon Web Services
Building an AWS IoT-Enabled Drink Dispenser - IOT328 - re:Invent 2017Building an AWS IoT-Enabled Drink Dispenser - IOT328 - re:Invent 2017
Building an AWS IoT-Enabled Drink Dispenser - IOT328 - re:Invent 2017
Amazon Web Services2.3K views
MCL331_Building a Virtual Assistant with Amazon Polly and Amazon Lex Pollexy by Amazon Web Services
MCL331_Building a Virtual Assistant with Amazon Polly and Amazon Lex PollexyMCL331_Building a Virtual Assistant with Amazon Polly and Amazon Lex Pollexy
MCL331_Building a Virtual Assistant with Amazon Polly and Amazon Lex Pollexy
How to Implement Domain Driven Design in Real Life SDLC by Abdul Karim
How to Implement Domain Driven Design  in Real Life SDLCHow to Implement Domain Driven Design  in Real Life SDLC
How to Implement Domain Driven Design in Real Life SDLC
Abdul Karim3.3K views
Increasing Website Sales & Conversions with Brad Hauck by Brad Hauck
Increasing Website Sales & Conversions with Brad HauckIncreasing Website Sales & Conversions with Brad Hauck
Increasing Website Sales & Conversions with Brad Hauck
Brad Hauck212 views
Touch Screens | The very expensive mistake by Casey Dubbs
Touch Screens | The very expensive mistakeTouch Screens | The very expensive mistake
Touch Screens | The very expensive mistake
Casey Dubbs352 views

More from Crispin Reedy

Association for Voice Interaction Design - Annual Meeting 2018 by
Association for Voice Interaction Design - Annual Meeting 2018Association for Voice Interaction Design - Annual Meeting 2018
Association for Voice Interaction Design - Annual Meeting 2018Crispin Reedy
389 views15 slides
Assertive Niceness by
Assertive NicenessAssertive Niceness
Assertive NicenessCrispin Reedy
600 views48 slides
Adding Visuals to Voice Panel - SpeechTEK 2017 by
Adding Visuals to Voice Panel - SpeechTEK 2017Adding Visuals to Voice Panel - SpeechTEK 2017
Adding Visuals to Voice Panel - SpeechTEK 2017Crispin Reedy
622 views29 slides
Chatbots vs. Voicebots Sunrise Session SpeechTEK 2017-final by
Chatbots vs. Voicebots Sunrise Session SpeechTEK 2017-finalChatbots vs. Voicebots Sunrise Session SpeechTEK 2017-final
Chatbots vs. Voicebots Sunrise Session SpeechTEK 2017-finalCrispin Reedy
720 views27 slides
Association for Voice Interaction Design Annual Meeting 2017 by
Association for Voice Interaction Design Annual Meeting 2017Association for Voice Interaction Design Annual Meeting 2017
Association for Voice Interaction Design Annual Meeting 2017Crispin Reedy
486 views15 slides
Where's Jarvis? The Future of Voice Recognition and Natural Language User In... by
Where's Jarvis?  The Future of Voice Recognition and Natural Language User In...Where's Jarvis?  The Future of Voice Recognition and Natural Language User In...
Where's Jarvis? The Future of Voice Recognition and Natural Language User In...Crispin Reedy
974 views77 slides

More from Crispin Reedy(15)

Association for Voice Interaction Design - Annual Meeting 2018 by Crispin Reedy
Association for Voice Interaction Design - Annual Meeting 2018Association for Voice Interaction Design - Annual Meeting 2018
Association for Voice Interaction Design - Annual Meeting 2018
Crispin Reedy389 views
Adding Visuals to Voice Panel - SpeechTEK 2017 by Crispin Reedy
Adding Visuals to Voice Panel - SpeechTEK 2017Adding Visuals to Voice Panel - SpeechTEK 2017
Adding Visuals to Voice Panel - SpeechTEK 2017
Crispin Reedy622 views
Chatbots vs. Voicebots Sunrise Session SpeechTEK 2017-final by Crispin Reedy
Chatbots vs. Voicebots Sunrise Session SpeechTEK 2017-finalChatbots vs. Voicebots Sunrise Session SpeechTEK 2017-final
Chatbots vs. Voicebots Sunrise Session SpeechTEK 2017-final
Crispin Reedy720 views
Association for Voice Interaction Design Annual Meeting 2017 by Crispin Reedy
Association for Voice Interaction Design Annual Meeting 2017Association for Voice Interaction Design Annual Meeting 2017
Association for Voice Interaction Design Annual Meeting 2017
Crispin Reedy486 views
Where's Jarvis? The Future of Voice Recognition and Natural Language User In... by Crispin Reedy
Where's Jarvis?  The Future of Voice Recognition and Natural Language User In...Where's Jarvis?  The Future of Voice Recognition and Natural Language User In...
Where's Jarvis? The Future of Voice Recognition and Natural Language User In...
Crispin Reedy974 views
Voice Recognition and Natural Language - Dallas TechFest 2016 by Crispin Reedy
Voice Recognition and Natural Language - Dallas TechFest 2016Voice Recognition and Natural Language - Dallas TechFest 2016
Voice Recognition and Natural Language - Dallas TechFest 2016
Crispin Reedy1K views
Top 10 Tips for Making Complicated Things Simple by Crispin Reedy
Top 10 Tips for Making Complicated Things SimpleTop 10 Tips for Making Complicated Things Simple
Top 10 Tips for Making Complicated Things Simple
Crispin Reedy7.8K views
Association for Voice Interaction Design Annual Meeting 2016 by Crispin Reedy
Association for Voice Interaction Design Annual Meeting 2016Association for Voice Interaction Design Annual Meeting 2016
Association for Voice Interaction Design Annual Meeting 2016
Crispin Reedy431 views
Going Solo: Design and Productivity Techniques for the Team of One by Crispin Reedy
Going Solo: Design and Productivity Techniques for the Team of OneGoing Solo: Design and Productivity Techniques for the Team of One
Going Solo: Design and Productivity Techniques for the Team of One
Crispin Reedy645 views
Service Design and the Omnichannel Experience - SpeechTEK 2015 by Crispin Reedy
Service Design and the Omnichannel Experience - SpeechTEK 2015Service Design and the Omnichannel Experience - SpeechTEK 2015
Service Design and the Omnichannel Experience - SpeechTEK 2015
Crispin Reedy4.9K views
Association for Voice Interaction Design Annual Meeting 2015 by Crispin Reedy
Association for Voice Interaction Design Annual Meeting 2015Association for Voice Interaction Design Annual Meeting 2015
Association for Voice Interaction Design Annual Meeting 2015
Crispin Reedy816 views
SpeechTEK University Outtakes 2014: Zero Out Strategies by Crispin Reedy
SpeechTEK University Outtakes 2014: Zero Out StrategiesSpeechTEK University Outtakes 2014: Zero Out Strategies
SpeechTEK University Outtakes 2014: Zero Out Strategies
Crispin Reedy698 views
2013 Speech TEK - Alphanumeric Recognition Discussion by Crispin Reedy
2013 Speech TEK - Alphanumeric Recognition Discussion2013 Speech TEK - Alphanumeric Recognition Discussion
2013 Speech TEK - Alphanumeric Recognition Discussion
Crispin Reedy894 views
Design Thinking Action Lab Exercise 1 by Crispin Reedy
Design Thinking Action Lab Exercise 1Design Thinking Action Lab Exercise 1
Design Thinking Action Lab Exercise 1
Crispin Reedy327 views

Recently uploaded

KVM Security Groups Under the Hood - Wido den Hollander - Your.Online by
KVM Security Groups Under the Hood - Wido den Hollander - Your.OnlineKVM Security Groups Under the Hood - Wido den Hollander - Your.Online
KVM Security Groups Under the Hood - Wido den Hollander - Your.OnlineShapeBlue
75 views19 slides
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive by
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveAutomating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveNetwork Automation Forum
43 views35 slides
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue by
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlueShapeBlue
31 views23 slides
Mitigating Common CloudStack Instance Deployment Failures - Jithin Raju - Sha... by
Mitigating Common CloudStack Instance Deployment Failures - Jithin Raju - Sha...Mitigating Common CloudStack Instance Deployment Failures - Jithin Raju - Sha...
Mitigating Common CloudStack Instance Deployment Failures - Jithin Raju - Sha...ShapeBlue
54 views18 slides
How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ... by
How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ...How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ...
How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ...ShapeBlue
46 views28 slides
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ... by
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...Jasper Oosterveld
27 views49 slides

Recently uploaded(20)

KVM Security Groups Under the Hood - Wido den Hollander - Your.Online by ShapeBlue
KVM Security Groups Under the Hood - Wido den Hollander - Your.OnlineKVM Security Groups Under the Hood - Wido den Hollander - Your.Online
KVM Security Groups Under the Hood - Wido den Hollander - Your.Online
ShapeBlue75 views
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive by Network Automation Forum
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLiveAutomating a World-Class Technology Conference; Behind the Scenes of CiscoLive
Automating a World-Class Technology Conference; Behind the Scenes of CiscoLive
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue by ShapeBlue
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue
2FA and OAuth2 in CloudStack - Andrija Panić - ShapeBlue
ShapeBlue31 views
Mitigating Common CloudStack Instance Deployment Failures - Jithin Raju - Sha... by ShapeBlue
Mitigating Common CloudStack Instance Deployment Failures - Jithin Raju - Sha...Mitigating Common CloudStack Instance Deployment Failures - Jithin Raju - Sha...
Mitigating Common CloudStack Instance Deployment Failures - Jithin Raju - Sha...
ShapeBlue54 views
How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ... by ShapeBlue
How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ...How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ...
How to Re-use Old Hardware with CloudStack. Saving Money and the Environment ...
ShapeBlue46 views
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ... by Jasper Oosterveld
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
ESPC 2023 - Protect and Govern your Sensitive Data with Microsoft Purview in ...
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti... by ShapeBlue
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...
DRaaS using Snapshot copy and destination selection (DRaaS) - Alexandre Matti...
ShapeBlue26 views
Developments to CloudStack’s SDN ecosystem: Integration with VMWare NSX 4 - P... by ShapeBlue
Developments to CloudStack’s SDN ecosystem: Integration with VMWare NSX 4 - P...Developments to CloudStack’s SDN ecosystem: Integration with VMWare NSX 4 - P...
Developments to CloudStack’s SDN ecosystem: Integration with VMWare NSX 4 - P...
ShapeBlue60 views
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T by ShapeBlue
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&TCloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
CloudStack and GitOps at Enterprise Scale - Alex Dometrius, Rene Glover - AT&T
ShapeBlue38 views
Five Things You SHOULD Know About Postman by Postman
Five Things You SHOULD Know About PostmanFive Things You SHOULD Know About Postman
Five Things You SHOULD Know About Postman
Postman38 views
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda... by ShapeBlue
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...
Hypervisor Agnostic DRS in CloudStack - Brief overview & demo - Vishesh Jinda...
ShapeBlue44 views
Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ... by ShapeBlue
Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...
Backup and Disaster Recovery with CloudStack and StorPool - Workshop - Venko ...
ShapeBlue55 views
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit... by ShapeBlue
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...
Transitioning from VMware vCloud to Apache CloudStack: A Path to Profitabilit...
ShapeBlue40 views
PharoJS - Zürich Smalltalk Group Meetup November 2023 by Noury Bouraqadi
PharoJS - Zürich Smalltalk Group Meetup November 2023PharoJS - Zürich Smalltalk Group Meetup November 2023
PharoJS - Zürich Smalltalk Group Meetup November 2023
Noury Bouraqadi139 views
Business Analyst Series 2023 - Week 4 Session 7 by DianaGray10
Business Analyst Series 2023 -  Week 4 Session 7Business Analyst Series 2023 -  Week 4 Session 7
Business Analyst Series 2023 - Week 4 Session 7
DianaGray1042 views
Why and How CloudStack at weSystems - Stephan Bienek - weSystems by ShapeBlue
Why and How CloudStack at weSystems - Stephan Bienek - weSystemsWhy and How CloudStack at weSystems - Stephan Bienek - weSystems
Why and How CloudStack at weSystems - Stephan Bienek - weSystems
ShapeBlue81 views
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R... by ShapeBlue
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...
Setting Up Your First CloudStack Environment with Beginners Challenges - MD R...
ShapeBlue37 views

Voice User Interface Design - Big Design 2017

  • 1. © 2017 Versay Solutions Voice User Interface Design: Skills, Actions, And The Future Crispin Reedy, Versay Solutions @crispinTX crispinreedy.com #BigD17
  • 2. © 2017 Versay Solutions Voice User Interface Design: Skills, Actions, And The Future Disclaimer: This session was NOT sponsored by Dominos
  • 3. © 2017 Versay Solutions • Voice User Interface Designer • 15+ years in the field • Former coder; got interested in UX • President of the Association for Voice Interaction Design • Consultant for Versay Solutions @crispinTX crispinreedy.com
  • 4. © 2017 Versay Solutions Session Description • Amazon Skills for Alexa, Google Actions for Home – Should your company build a conversational voice interface for one of these systems, and if so, how? • What are the differences between a voice user interface and other types of UIs? • What types of skills does a VUI designer need? • What are some best practices for these VUIs? • You’ll walk away with answers to the questions “If, Why, and How” you might choose to explore this interesting new area of design.
  • 5. © 2017 Versay Solutions Easy Answer To #1 • If your company is involved in home automation: • Mostly likely Yes, and Yesterday • Although how you do it will depend on your platform • More on that later! • Everyone else • Let’s keep talking!
  • 6. © 2017 Versay Solutions Basic Terms
  • 7. © 2017 Versay Solutions Terms & Technologies •Speech Recognition •Natural Language Understanding •Voice Verification (Biometrics) •Text to Speech
  • 8. © 2017 Versay Solutions Speech Recognition “ASR” “See the cat.”
  • 9. © 2017 Versay Solutions Natural Language Understanding •Extracting meaning from natural text “Hello, yes, I’d like to pay my water bill. Can you help me with that? Intent = BillPay Entity (Bill Type) = Water
  • 10. © 2017 Versay Solutions Voice Verification “My voice is my password.” “Authenticated. Welcome, Mr. Smith.” ✓
  • 12. © 2017 Versay Solutions Speech Recognition • Hands-free command / control • Dictation • Input text • Small form factor device, etc. Text To Speech • Output text dynamically • Respond to input • Useful when no display is available Natural Language Understanding • Necessary for all language-based input • Extract meaning • Parse large volumes of text Voice Verification • Security
  • 13. ASR Application Data • Sign-In • Interaction • Request • Action • Meaning • Access Data • Output TTS NLU Voice prints Verifi- cation
  • 14. © 2017 Versay Solutions Speech Technology Today
  • 15. © 2017 Versay Solutions Speech Agents, Apps, and APIs Speech Agents: • Amazon Alexa • Echo, Dot, Echo Show • Google Assistant • Pixel, Android, Google Home, iPhone app • Apple’s Siri • iPhone, iPad, MacOS (Sierra), AppleTV • Microsoft’s Cortana • Windows 10, Windows Phone, Xbox, iPhone app • Samsung’s Bixby • Galaxy S8, Family Hub 2.0 Fridge
  • 16. © 2017 Versay Solutions Speech Agents, Apps, and APIs Speech Agents can be extended with “Voice Apps” • Alexa Skills • Google Actions • SiriKit • Cortana SDK
  • 17. © 2017 Versay Solutions Speech Agents, Apps, and APIs Agent capabilities and apps are somewhat determined by: • Platform: Device • Screen, keyboard, phone, mics, etc. • Environment: Web site, apps that interact with the agent • Ecosystem: Underlying connections, technical partnerships
  • 18. © 2017 Versay Solutions Platforms
  • 19. © 2017 Versay Solutions Environment Google “Actions” or “Apps” • Curated • Direct vs. Conversational Siri - Works via apps Order Uber Order Lyft
  • 20. © 2017 Versay Solutions New York Times
  • 21. © 2017 Versay Solutions Speech Agents, Apps, and APIs APIs: Allow you access to the underlying technology • Amazon • AVS (Alexa Voice Service) Create an “Alexa” on your own device • Amazon Lex, Amazon Polly • Google • Cloud Speech API • API.ai • Apple • Apple Speech Framework • Microsoft • Bing Speech API Ecobee Smart Thermostat
  • 22. © 2017 Versay Solutions Use Cases
  • 23. Use Case “Bakeoff” from Tech Insider •Travel •Email •Messaging •Sports •Music •Weather •Calendar •Social • Translation • Basic tasks • General knowledge • Personality http://www.businessinsider.com/siri-vs-google-assistant-cortana-alexa-2016-11/
  • 24. © 2017 Versay Solutions Use Case “Bakeoff” from Tech Insider • “wildly finicky when it comes to phrasing.” • “Each assistant still feels like a fragile, thinly veiled web of loosely connected services — because that's what they are.” • “incredibly uncomfortable to speak to an inanimate thing in public.” • “In Google Assistant's case, normalizing the need to call on a brand ("OK Google") whenever you need a hand is Orwellian.” • “None of these things are at a place I could comfortably call "good.””
  • 25. © 2017 Versay Solutions Personal Assistant vs. Home Assistant The Google Pixel XL. Hollis Johnson/Business Insider Google.com
  • 26. © 2017 Versay Solutions Personal Assistant vs. Home Assistant
  • 27. © 2017 Versay Solutions Getting Specific With Alexa
  • 28. © 2017 Versay Solutions “Layers” of Alexa •Alexa Native Capabilities •Alexa Skills •Alexa Voice Services
  • 29. © 2017 Versay Solutions “Layers” of Alexa • Alexa Native Capabilities • Come out of the box • Require Alexa wake word (can be changed) • Alexa Skills • Alexa’s “Extensions” or “Add-Ons” • Designed for and deployed on Echo Device • Skills must be downloaded to Echo • Require Alexa wake word + Skill name • Alexa Voice Services • Add Alexa voice control to your own device
  • 30. © 2017 Versay Solutions Alexa “Native” Capabilities Alexa, what’s 3 + 5? Alexa, set an alarm for 3 am. Alexa, set a thirty second timer. Alexa, what’s the weather? Note: Mix of TTS & Pre-Recorded Audio Note: “Hint”
  • 31. © 2017 Versay Solutions Design Considerations •Proactive “Hints” • Similar to “Hover Help” or “Tool Tip” • But less avoidable! • Pro: Can teach user about other capabilities • Con: Can be annoying! • Guideline: If used, be sparing • Develop rules for when and how frequently to offer
  • 32. © 2017 Versay Solutions Amazon.com Native & Skill Skill Skill Skill Native & Skill Alexa Skills
  • 33. © 2017 Versay Solutions Source: David Attwater, EIG Inc.
  • 34. © 2017 Versay Solutions Amazon.com
  • 36. © 2017 Versay Solutions Amazon.com
  • 37. © 2017 Versay Solutions Design Considerations • Invoking Skills: • Alexa, open Oprah Magazine • Alexa, order a pizza from Domino’s • Alexa, ask Cook Reference what’s the safe temperature for chicken • Syntax: Open <skill> Ask <skill> for (about, to, with, etc.) <action> Ask <skill> <question> Also: Search, Tell, Talk to, Launch, Start, Resume, Run, Load, Begin Oprah Magazine
  • 38. © 2017 Versay Solutions Design Considerations •Skills can be “installed” on the fly •If the user knows the name of the skill •Skills that require account information will need extra steps Cook Reference Domino’s
  • 39. © 2017 Versay Solutions Alexa App + Linking
  • 40. © 2017 Versay Solutions Design Considerations •Managing access to skills may become difficult or confusing.
  • 41. © 2017 Versay Solutions Design Considerations •Attention (or lack of attention!) to technical details can become “deal- killing” part of overall experience Domino’s
  • 42. © 2017 Versay Solutions Really? Dominos.com
  • 43. © 2017 Versay Solutions No Dominos.com
  • 44. © 2017 Versay Solutions Design Considerations • Confirmation • What’s the phone number? • 214-555-1235 • You said 214-555-1235. Is that correct? • Yes • Note: System confirmed the phone number but not the address • Was the address really correct?
  • 45. © 2017 Versay Solutions Dominos.com
  • 46. © 2017 Versay Solutions Design Considerations • “Would you like to place your Easy Order, reorder your most recent order, or start a new order?” • If I’m not logged into my account on the Alexa app, options 1 and 2 don’t make much sense. • “Would you like” is ambiguous – could be used for Yes / No questions or for multi-item questions • First part of the sentence runs into the choices • Reuse of the word “order” just seems odd (but may be unavoidable). • Could have used more pauses (SSML) Domino’s
  • 47. © 2017 Versay Solutions Design Considerations: SSML • Speech Synthesis Markup Language • Can control the way your TTS playback sounds • Very important if your output is mostly TTS • Which is true of all most platforms • Should be supported by all types of TTS engine • Amazon has platform specific options • Plan on using it to fine tune your audio output
  • 48. © 2017 Versay Solutions New Prompts & SSML Examples • Note: TTS Samples with SSML created with Amazon Polly, not Alexa • “You can: Place your easy order. Reorder your most recent order. Or, start a *new* order.” • You can: <break time="500ms"/>Place your easy order, <break time="500ms"/> Reorder your <emphasis level="moderate">most recent</emphasis> order, <break time="500ms"/> Or, start a <emphasis level="strong">new</emphasis> order. • Placing an order, great! Choose from: My easy order. My most recent. Or, start a *new* order. • <speak>Placing an order. <prosody pitch="high">Great!</prosody> Choose from: My easy order. My most recent. Or, start a <emphasis> <prosody pitch="high">new</prosody> </emphasis> order.</speak> Domino’s
  • 49. © 2017 Versay Solutions Still Trying To Order That Pizza • Start of the interaction has changed! • Probably due to login • “Would you like to place an order, or track an order?” • What just happened!!!? •System was expecting me to say “Start a new order” and I only said “New Order.” Domino’s
  • 50. © 2017 Versay Solutions Design Considerations • Make sure your input grammar covers all possible logical utterances (what user can say) • Don’t leave this stuff up to the programmers! • Provide examples of coverage • Coverage should match prompts • Use some kind of markup to show coverage • [] optional • () grouping • | or • “Would you like to place your Easy Order, reorder your most recent order, or start a new order?” • [place] [my | an] Easy Order • [reorder] [my] most recent [order] • [start a] new [order]
  • 51. © 2017 Versay Solutions Design Considerations •Reprompts: • What do you do when you didn’t understand what the caller said? • Probably don’t want to say “Sorry” • This can be annoying • But you CAN rephrase the prompt to make it different • Using the same prompt gives the user a sense that something has gone wrong
  • 52. © 2017 Versay Solutions Pizza Pizza Pizza • Hey you didn’t really need to explain about the phone number since I saved it but OK…. • Address has been saved to profile, great! • And then boom Domino’s
  • 53. © 2017 Versay Solutions With Speech, you need to spend a lot more time thinking about what happens when things go wrong.
  • 54. © 2017 Versay Solutions I Didn’t Really Want to Order Pizza But By Now I Am Hungry And So Is Somebody Else • Note “Easy Order” and Credit Card cannot be set up on the website unless you’re actually placing an order. • Give people enough time to talk! • There’s that grammar coverage issue again • Bell pepper = Green pepper • What synonyms is your user likely to say? • At some point couldn’t you just give me a list? • Notice how they screwed up the article + the item “… adding a parmesan bread twists” Meow Domino’s
  • 55. © 2017 Versay Solutions
  • 56. © 2017 Versay Solutions
  • 57. © 2017 Versay Solutions Design Considerations • Confirm and correct • “Do you want to add anything else?” • “Yes, I want to add peppers.” • Disambiguation • “Olives” • “Ok, we have two kinds of olives. Black olives, or green olives.” • A Voice User Interface design is a time-based interface • As a designer concerned with user experience you’re going to be involved in things (such as pauses) which may not occur to you
  • 58. © 2017 Versay Solutions How Did Google Home Do? •“OK Google, Order Dominos” • “There are stores at….” • Had to go find the right “App Name” online •“OK Google, Talk to Dominos” • “You can link to your Domino’s account…” • Had a terrible time finding the “Google Apps.”
  • 59. © 2017 Versay Solutions How Did Google Home Do? •Menu worked! • System did not recognize “Ham” (Should offer list of ingredients) • System became very laggy
  • 60. © 2017 Versay Solutions How Did Google Home Do? • Edited for time • Original was 3:35 • This is 2:15 • Use of “Dom” persona and male voice • “Hand off” • Playback of address: • Alexa: “Eighty seven twenty three” • Google: “Eight thousand seven hundred twenty three” • Same issue with “twists” • “Your day just got cheesier”
  • 61. © 2017 Versay Solutions Design Considerations •Discoverability • “OK Google, Order Dominos” •Persona • Google Home has more control over the voice • Branding considerations – “Dom” name and male TTS •Playback of Dynamic Data • Attention to detail – don’t trust the platform to do it the way you want it
  • 62. © 2017 Versay Solutions Design Considerations Maintaining State: •Between dialogs • “Who is Seth McFarlane?” • “Seth McFarlane is…” • “When’s his birthday?” • “I’m not sure what you’re talking about.” •From session to session Oprah Magazine
  • 63. © 2017 Versay Solutions Home Automation •Onboarding issues are very similar to “Skills,” but there is an additional layer of complexity • Companies are working to improve the experience • After setup, you get a lot of bang for the buck
  • 64. © 2017 Versay Solutions “Computer, turn on the library lights”
  • 65. © 2017 Versay Solutions TP Link
  • 66. © 2017 Versay Solutions Amazon
  • 67. © 2017 Versay Solutions Design Considerations: Summary • Managing access to Skills (App, Store) • Managing the Onboarding Experience • Discoverability • Invoking Skills • Hints • Confirmation • Asking Yes/No Questions vs. Multi-Item Questions • SSML • Silences • Reprompting • Coverage (prompt vs. possible input) • Managing technical errors • Timing and Timeouts • Article matching the noun • Confirm and Correct • Disambiguation • Persona • Playback of Dynamic Data • Maintaining State
  • 68. © 2017 Versay Solutions What Makes a Good VUI Designer? •Concern with the overall experience • All of the channels that go into making up how something happens •Attention to “small” technical details • Pauses • SSML •Writing skills! • Dialog, not tech doc • English majors, screenwriters
  • 69. © 2017 Versay Solutions Session Description • Amazon Skills for Alexa, Google Actions for Home – Should your company build a conversational voice interface for one of these systems, and if so, how? • What are the differences between a voice user interface and other types of UIs? ✔ • What types of skills does a VUI designer need? ✔ • What are some best practices for these VUIs? ✔ • You’ll walk away with answers to the questions “If, Why, and How” you might choose to explore this interesting new area of design. ✔
  • 70. © 2017 Versay Solutions If, Why, How •What are you trying to build? •Existing guidelines / research •User testing is key • Especially if you’re trying to do something complicated
  • 71. © 2017 Versay Solutions If, Why, How: Beyond Skills Write an app (skill) for an agent such as Google Assistant / Alexa Use cloud APIs to add ASR / NLU to your app / device / page / gadget Download software and use full-featured capabilities for more robust recognition on a specific device Build your own
  • 72. © 2017 Versay Solutions If, Why, How: What’s the Use Case? •Enabling application • User can’t do it any other way • New tasks •Enhancing application • User can do it now • But speech makes it better • Faster • Safer
  • 73. © 2017 Versay Solutions API-Based Device- Based Roll Your Own / Open- Source •Flexibility •Power •Customization •Time •Difficulty
  • 74. © 2017 Versay Solutions Existing Guidelines / Research • Caveat: Best practices evolved in one modality (e.g. voice-only) may not apply the same way in another (e.g. combined voice + touch) • But they could be adapted • Association for Voice Interaction Design (AVIxD.org) • Wiki • Peer-Reviewed Journal • Virtual “Brown Bags” • Academic Sources, Books
  • 75. © 2017 Versay Solutions AVIxD.org CUI Working Group is actively recruiting!
  • 76. © 2017 Versay Solutions @crispinTX Crispin Reedy Thank You!

Editor's Notes

  1. DO NOT FORGET TO BRING THE MINI-SPEAKERS!!!
  2. “Speech to Text” ? Spoken Language – Machine readable format
  3. Not necessarily tied to speech recognition
  4. Also called voiceprints, biometrics, voice authentication, etc. Not going to discuss this one in a lot of detail today but it’s important that you understand the difference between these technologies. Recognizes a person, not necessarily what they are saying. You can have ASR without Voice Verification And vice versa
  5. Human voice talent Hundreds of hours of recording Digitized Phonemes: Concatenated speech synthesis
  6. Alexa, Ask Capitol One What’s my current credit card balance?
  7. What do you need it for? What kind of device will you be running it on? Connectivity? Can you use cloud based ASR? How much control do you need over the application / user interface?
  8. DO NOT FORGET TO BRING THE MINI-SPEAKERS!!!