Look Ma! No Screens! Conducting UX research on non-visual user experiences.

#UXPA2017
www.uxpa2017.org
Download our mobile app UXPA.org/2017app to fill out our conference survey!
Look Ma! No Screens!
Conducting UX Research on Non-Visual User Experiences
Presented by:
James Thomas & Rachana Ranade

#UXPA2017
www.uxpa2017.org
Overview
5 Minutes Introductions
10 Minutes Voice-Based Interface Design
30 minutes Testing IVR & Voice Biometrics User Experiences
• Sample studies from two new services we tested
• Our methods and a sample of results
• Lessons learned for next time
5 minutes Summary
10 minutes Q&A

#UXPA2017
www.uxpa2017.org
About Us

#UXPA2017
www.uxpa2017.org
Who we are: Fidelity Investments
• Fidelity Investments is a 71 years old multinational financial services
corporation headquartered in Boston, Massachusetts.
• Fidelity currently manages 25 M individual investors accounts total
customer assets of $6.0 trillion (As of March 31, 2017)
• USA’s no. 1 record-keeper of 401(k) retirement savings plans and a
leading provider of 403(b) retirement plans.
• Fidelity serves its customers through 10 regional offices and more
than 190 Investor Centers in the United States.
• Approximately 40,000-plus associates spanning across eight other
countries across North America, Europe, Asia, and Australia
https://www.fidelity.com/about-fidelity/who-we-serve/overview

#UXPA2017
www.uxpa2017.org
Who we are: User Experience Design group
• About 300 employees work in the User Experience Design group which
encompasses of interaction and visual design, front-end development and
user research.
• Approximately 50 user researchers work across Boston, Jersey City and
North Carolina across different portfolios.
• In 2016, the UXD research team conducted about 300+ studies about 60%
qualitative and 40% quantitative.
• Last year we collected feedback from nearly 50,000 participants which
includes about 60% customers, 25% prospects and 15% Fidelity employees.
https://design.fidelity.com/

#UXPA2017
www.uxpa2017.org
Boston World
Trade Center Lab
Control
Room
Participant
Room

#UXPA2017
www.uxpa2017.org
Web UI vs. Voice UI Design & Research
Web UI
• Visual affordances
• Cues to visual attention
(e.g. cursor movements)
• Persistent in time – e.g reading a
list on a page
• Less bias towards
anthropomorphizing
Voice UI
• No visual affordances (mostly)
• No cues to what listeners are
attending to
• Transient in time – e.g. listening
to a list over a speaker
• Stronger anthropomorphism
increases users’ expectations
https://www.oreilly.com/ideas/basic-principles-for-designing-voice-user-interfaces
https://www.interaction-design.org/literature/article/how-to-design-voice-user-interfaces
http://www.uxmatters.com/mt/archives/2008/09/emotion-and-voice-user-interfaces.php

#UXPA2017
www.uxpa2017.org
Adoption is Increasing, Technology Improving

#UXPA2017
www.uxpa2017.org
Case 1, Cosmo
Yesterday
• Antiquated telephony system
• Phone tree prompts
• > 2 min to get through
prompts
• Minimal info known about the
customer
• Minimal routing options
• High transfer rates
Today and Beyond
• Flexible, scalable & intelligent
systems
• Tell me why you’re calling
• < 1 minute to get through new
experience
• More info known about the
customer & associate
• More accurate routing

#UXPA2017
www.uxpa2017.org
Case 1, Cosmo

#UXPA2017
www.uxpa2017.org
Using your touch-tone keypad, please enter
the first three letters of the movie title, now.
You've selected ... Agent Zero? If that's correct,
press one.
Ah, you've selected ... Brown-Eyed Girl? If this
is correct, press one.
Why don't you just tell me the name of the
movie you've selected?

#UXPA2017
www.uxpa2017.org
Problem Statement
Cosmo will provide a better phone experience
by understanding the caller’s stated need / intent
and routing the call to the correct associate.

#UXPA2017
www.uxpa2017.org
Case 1, Cosmo-User Research Methods
What methods did you use?
• The good news is that the fundamentals of user research methods remain
true, but we need to and think out of the box and be creative.
What method to choose and when?
• Depending on the research goals we decided the method. To test the full
flow experience and interactive elements we picked the traditional lab
and remote moderated method. Quantitative method such as surveys
were chosen when we had to test different variations, desirability and
get feedback on tone and language .
When did you choose remote vs. in-person session?
• When we testing Wizard of Oz type flow we went with the in-person for
other scenarios we went with remote. Whenever possible we tried
remote sessions as it replicated the real situation of the customer
interacting with the automated system as well as the associates.

#UXPA2017
www.uxpa2017.org
Case 1, Cosmo –Recruiting & Test Plan
• Recruiting the right audience for usability sessions is one of the most
important factor for one-on-one usability sessions.
• Defining the target audience based on their demographics and key
behaviors such as impatient-patient, direct-ramblers, accents.
One a scale of 1-7, where 1-not at all frustrating to 7- Very frustrating, how frustrating is it:
Scale: 1-2 (Patient), 3-4-5 (Intermediate), 6-7 (Impatient)
• To wait for restaurant services
• To wait on a telephone line
• To wait for on the call when you are getting transferred to a different agent
• To wait in front of the counters for your turn
Personality Question: Scale: 1-2 (Patient), 3-4-5 (Intermediate), 6-7 (Impatient)
• On a scale of 1-7, where is 1-Competitve and 7 is Relaxed, how would you rate yourself?
• On a scale of 1-7, where is 1-Aggressive and 7 is Accommodating, how would you rate yourself?
• On a scale of 1-7, where is 1-Impatient and 7 is Patient, how would you rate yourself?
• Last but not least, we made sure that the most common and real scenarios
are tested so that we receive most effective feedback

#UXPA2017
www.uxpa2017.org
Case 1, Cosmo- Web Content
FIDELITY.COM TAX FORM AVAILABILITY
“We believe it is important to post tax forms after we have received all
anticipated information. Sometimes issuers (e.g., the company or municipality
offering securities for sale to investors) are unable to send the final
information in time for us to meet IRS deadlines. In such cases, we request a
30-day extension to mail tax information for assets that do not provide final
information until after the February 16 initial deadline. Waiting for final
information allows us to send you one correct form, instead of multiple
versions, which could result in you having to amend your return.”
“For 1099 forms, Fidelity follows the IRS rule de minimis (i.e., not enough to be
considered). If the income on the tax form is less than $10.00, we don't mail a
tax form or report the income to the IRS. However, we post the form online
and notify you by email. When you view these forms online, you'll see a
watermark and a label on the top right of the form saying ‘de minimis’.”
https://www.fidelity.com/taxes/statement-mailing-dates

#UXPA2017
www.uxpa2017.org
Case 1, Cosmo- Improvised Content
COSMO TAX FORM AVAILABILITY
Alright. The 2015 tax forms will be available at fidelity.com/tax by the following
dates. If you usually receive your forms by mail, look for them 7-10 days after
these dates.
For retirement accounts, most tax forms, including the 1099-R and 5498, are
available online now. If you made a prior year IRA contribution, an amended
5498 will be available May 10th.
For non-retirement accounts, the consolidated 1099, which may include sections
–DIV, -INT, –B, -OID, and –MISC, will be available online on or before February
13th. Information about non-Fidelity holdings can take longer to process, which
may delay your tax forms.
For accounts with less than $10 of taxable activity, Fidelity will not mail you a tax
form.

#UXPA2017
www.uxpa2017.org
Case 1, Cosmo-Content
• Make the language and tone friendly, warm and direct; instill comfort
and confidence in user. Keep it more open and transparent, less
challenging and exclusive.
“Thanks, got it.” – Simple, “human” language,
confirms we understand
• Use the language as if you are speaking to an equal partner more like a
guide and less like an instructor
• Suggests to user why it’s important to let us know reason for calling and
encourage user to continue vs. pressing “0” or saying “Agent!”
“We heard that you asked to speak with a
representative. To get you to the right person, please
tell us why…”

#UXPA2017
www.uxpa2017.org
Case 1, Cosmo-Humanizing Voice
• Consistent feedback we received during the studies to make the tone
sound like a person and not robotic.
“Make the sound more human-like, more natural
sounding”
• We had one scenario reporting death, it was interesting to get the
reaction that want the system to respond in a humanized way.
“Sorry to hear about your loss….it should recognize
the sadness in the situation”
“Someone is calling on their worst day and trying to
handle difficult business. So anything sympathetic
would be nice there."

#UXPA2017
www.uxpa2017.org
Case 1, Cosmo-Learnings
• Based on the research goals choosing the right approach is very
important.
• Make sure you put efforts in recruiting to get the right audience and
test plan to replicate real scenarios.
• Humans process auditory very differently than visual medium. Hence,
do spend time on testing and refining the content.
• Convey the necessary information in the shortest amount of time and
keep it concise.
• In spite of being aware that they are speaking to automated system
customers do expect a natural human-like tone and reaction.

#UXPA2017
www.uxpa2017.org
Case 1, Cosmo - Outcome

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice
• 49% of users say phone channel authentication is
time-consuming and frustrating
• Companies who implemented voice biometrics
netted an average savings of 15M over a 3-year
period due to reduced handle time and customer
satisfaction
• Lets do this

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice
• Partnered with 3rd party vendor to implement free-
speech biometrics for the phone channel
• Stores a users’ digital “voiceprint”
• No passphrase necessary
• Customers are authenticated over the course of
natural conversation
• “Hi, I’m calling today because I’d like to get a few quotes
and maybe place a few trades.”

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice
PROBLEM STATEMENT
MyVoice will provide a better phone experience by
recognizing the customer by their voice alone, removing
the friction points in the current experience.

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice
• Kicked off research with an online study
• Research goals:
• Test alternative onboarding (A/B) and
authentication scripts (A/B)
• Understand drivers and barriers to enrollment

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice
“Lorem ipsum dolor sit amet, consectetur
adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore magna aliqua.
Ut enim ad minim veniam, quis nostrud
exercitation ullamco laboris nisi ut aliquip ex
ea cupidatat non proident, sunt in culpa qui
officia deserunt mollit anim id est laborum.”

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice
• Lessons learned
“Process is rather cold, lacks opportunity for exchanges with your
representatives… Futher dehumanizes relationship between me and your
company.”
“Sometimes just talking to robots is very frustrating. I can make transactions
by clicks after signing in. Why would I want to listen to a robot? Secure
transactions and accounts are paramount!”
“I could not hear well enough to understand.”
“There was nothing marked “play” to press,
therefore I didn’t hear about the new service.”

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice
• Lessons learned for next time
• Word instructions MORE carefully
• Add a “test clip” – and a means to opt out if
participants can’t hear it
• Do a trial run before launch (e.g. small N remote
unmoderated online study)

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice
• Followed up with a lab study with customers
• Remote participants
• Researcher role-play as phone representatives (fun!)
• Allowed us to address customers questions and concerns
just like a real rep would (…almost)
• A bit more colorful than your average lab study
• “Cheat sheets” helped us follow rep protocols
• FAQs from the 3rd party vendor
• Participants got really into it!

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice
“No, no thank you. The reason is, I’ve been with
voice recognition with other companies, and I
personally don’t think the technology is there yet...
It just didn’t recognize the alphanumeric terms that
I was saying over the bluetooth in my car and
sometimes I’d have to go back and repeat it
slower and clearer, which takes alot of time.”

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice - Outcomes
• Training materials for reps
• Comprehensive landing page with FAQs
• Email marketing campaign
• Happy customers
• “Your new voice recognition system is fabulous. I have a
complicated online password which is difficult to enter on
the phone keypad. The new system solved that. THANK
YOU.”

#UXPA2017
www.uxpa2017.org
Case 2, MyVoice - Outcomes
• 91% customer
adoption rate
during trial
rollout
• 98% verification
success for
enrolled callers

#UXPA2017
www.uxpa2017.org
In Recap
•Voice/speech user experiences are fast
proliferating, presenting exciting opportunities for
designers & researchers
•Learning we shared are applicable to other Voice
UIs like Amazon Echo, Google, Siri, etc.
•You CAN test voice/speech user experiences in the
lab and online
•Every study is a learning opportunity 

#UXPA2017
www.uxpa2017.org
Q&A
•Rachana Ranade | rachana.ranade@fmr.com
•James Thomas | james.thomas@fmr.com

Look Ma! No Screens! Conducting UX research on non-visual user experiences.

Recommended

Recommended

More Related Content

Similar to Look Ma! No Screens! Conducting UX research on non-visual user experiences.

Similar to Look Ma! No Screens! Conducting UX research on non-visual user experiences. (20)

More from UXPA International

More from UXPA International (20)

Recently uploaded

Recently uploaded (20)

Look Ma! No Screens! Conducting UX research on non-visual user experiences.

Editor's Notes