Miriam and Holly conducted research into voice technology by interviewing both current and non-users. Their usual research techniques did not work well with a voice assistant listening, so they had to adapt methods like using retrospective feedback and code names to refer to the device. Key findings from users included a lack of trust in the technology's accuracy and privacy concerns, while non-users saw voice assistants as lazy or a gimmick not fulfilling important use cases. Design challenges included vague error messages, unrealistic expectations of the technology's capabilities, and ensuring accessibility.

1. Researching
Voice Technology
Approach, challenges and themes
Dr Miriam Boyles & Holly Allison
@orangebus

2. Hello!
● Miriam - PhD in anthropology
and worked in health policy
before UX
● Holly - Psychology degree and
background in client services
● Working with large clients in the
public and private sectors
#Hol r O o
Holly Allison
UX Designer
Miriam BoylesUX Researcher

3. ● How is it perceived?
● How is it currently used?
● How easy is it to use?
● What are the opportunities for
development / new use cases?
Discovering VUI technology

4. ● 3 day diary study
○ Common uses
○ Context
○ Areas of frustration
● Contextual inquiry
○ Motivations
○ Perception
○ Use cases
○ Learnability and error
recovery
Approach: current users

5. Approach: non-users
● Interview
○ Reasons for
non-adoption
○ Perception
○ Understanding
● Usability test at OB
○ Ease of use
○ Learnability
○ Error recovery
○ Impact on perception

6. Some of our usual research
techniques didn’t quite work
with a third listener in the room…
… so we had to adapt.
Research
Challenges

7. Problem:
Talking during an exchange with the device will disrupt
the task and trigger an error.
Solution:
Gather retrospective feedback immediately after a task.
Consider playing back a video recording of the interaction
for the participant to respond to.
Losing the ‘think aloud’ method

8. Problem:
Referring to the device during interview makes it pipe up
- distracting!
Solution:
Agree on a code name with the participant that you can
use to refer to ‘it’ - we used ‘the device’ and ‘A’.
Don’t say the ‘A’ word!

9. Problem:
Easy for the researcher to lead the participant to personify
the device - calling it ‘he’ or ‘she’ rather than staying
neutral.
Solution:
Refer to the device by the code name or ‘the device’ at
first and then adopt the user’s language.
Non-leading pronouns

10. Problem:
You could give the participant clues to the correct
command phrase with the wording of your task
instructions.
Solution:
Avoid using command words or phrases in your task,
keep it neutral and non-leading and add more context.
Avoiding clues in the
task wording

11. Problem:
VUI is designed to fit naturally into users’ daily lives, but
demos and usability tests aren’t exactly natural scenarios.
Solution: (in progress!)
Use extended and more detailed diary studies to explore
context and usage. Tailor observational methods to
gather more realistic insights e.g. autoethnography,
family and friends research.
Observing natural interactions

12. Key Findings

13. ● Checking success
● Checking accuracy
● Checking visually
● Privacy concerns
Lack of trust

14. Laziness / Gimmick
● Technology perceived to be lazy by
users and non-users
● Not seen to fulfil important use cases
● “It's being promoted as making our
lives easier but I feel like it's making
us super lazy.”

15. Unhelpful errors
● Can’t do or won’t do?
● Vague and non-specific
● Self blame

16. Capability confusion
● Unrealistic expectations
● Encyclopedia / search engine
● “The advertising probably
overplays what it does”

17. Personification
● ‘She’ vs ‘It’
● Human characteristics
● Unmet expectations
● Feelings of rejection

18. Accessibility
● Hearing impaired user
● Missed words
● Insecurity and
embarrassment

19. Moments of delight
and frustration

20. Design challenge
● 8 opportunities for applying VUI
technology - 1 per group
● 5 mins to brainstorm ideas for your
challenge - talk to each other!
● Write each idea on a post-it and stick
them on your table
● 15 mins for each group to report back

21. Questions?

22. Thank you!
Dr Miriam Boyles & Holly Allison
@orangebus
www.orangebus.co.uk