Download as PDF, PPTX
Uploaded byAbi Reynolds
Voice usability testing with WOZ methodology - UX SCOT 2019
The document discusses the design and testing of voice applications, focusing on user-centered conversation design and the Wizard of Oz (WOZ) testing method. It outlines the importance of understanding user intents, creating dialogue flows, and iterating based on user feedback for effective implementation. Additionally, it emphasizes practical activities for creating conversational scripts and improving user experience in voice interactions.
More Related Content
Similar to Voice usability testing with WOZ methodology - UX SCOT 2019
![[BROCHURE] Italy Tour Project | @SlideON](https://cdn.slidesharecdn.com/ss_thumbnails/brochure8-251215152319-2805af68-thumbnail.jpg?width=640&height=640&fit=bounds)
![Chapter4_Initiation_of_Sediment_Motion_v2[1].pptx](https://cdn.slidesharecdn.com/ss_thumbnails/chapter4initiationofsedimentmotionv21-251208223747-f94ef163-thumbnail.jpg?width=640&height=640&fit=bounds)
Voice usability testing with WOZ methodology - UX SCOT 2019
- 1. Voice recognition inScotland? 1Source :https://giphy.com
- 2. Designing Voice applicationand testing with Wizard of Oz Abi Reynolds Steven Fullerton 2
- 3. 3 Abi Reynolds Principal UXconsultant @AbiJReynolds Steven Fullerton UX consultant @StevenFullerton Who are we?
- 4. Why are youinterested in Voice? 4
- 5. Our Role inConversation Design today. 5Image source: unsplash photo @bradneathery
- 6. Good and badimplementation 6
- 7. Good and badimplementation 7
- 8.
- 9.
- 10. What do wemean by voice? Google Home Amazon Echo 10Image sources: unsplash photos @andresurena and @@benceboros
- 11. Designing conversations 11 Keeping theuser at the centre What could a user centred design process for Voice look like?
- 12.
- 13. Intents = Purpose 13Source:https://giphy.com
- 14. What does asmart speaker hear? 14 Alexa, Ask Tile to find my Keys
- 15. What does asmart speaker hear? 15 Alexa, Ask Tile to find my Keys Utterance
- 16. What does asmart speaker hear? 16 Alexa, Ask Tile to find my Keys Wake Word Utterance
- 17. What does asmart speaker hear? 17 Alexa, Ask Tile to find my Keys Wake Word Invocation Utterance
- 18. What does asmart speaker hear? 18 Alexa, Ask Tile to find my Keys Wake Word Invocation Intent Slot Utterance
- 19. 19 “Design for howpeople talk, rather than how you wish them to talk.” - Cathy Pearl
- 20. Amy the HotelConcierge ▪ Our business objective: To cut down on calls to reception (save costs). ▪ Our customer experience objective: We think Amy will provide an innovative and useful service that will add to the overall the guest experience. A voice app should be easy to use hands free within hotel room whilst guests are doing other things. 20
- 21. Activity – 10Minutes Start the conversation! Write down: • A two way conversation between a customer who wants to know when checkout is and the smart assistant concierge. • The concierge can upsell late checkout at a cost of £10, IF the customer wants it. Create a short script based on the scenario above and work in pairs or small groups… 21 Activity – 10 Minutes Start the conversation! Write down: • A two way conversation between a customer who wants to know when checkout is and the smart assistant concierge. • The concierge can upsell late checkout at a cost of £10, IF the customer wants it. Create a short script based on the scenario above and work in pairs or small groups… 22
- 22. Feedback How did itgo? How many different utterances did we get? Lets act out your script How could the flow be improved or made to feel more natural? 22
- 23.
- 24. 24Source NNG Wireflowhttps://www.nngroup.com/articles/wireflows/
- 25. Dialogue flows givesyou an overview of all directions the conversations can go (based on intents). It plots how you plan to respond to questions and guide the user towards their goal. 25
- 26.
- 27. Activity – 10Minutes Complete the extended check-out flow The hotel offers extended check out at 3pm for an extra 10 pounds, which can be added to the room bill Work in groups and using the sheets provided 1. Complete the steps in the path until the user reaches their goal 27
- 28.
- 29. Remember 1. Core conversation- Y/N proceed. Until end of task 2. Prompt to go forward or finish task 3. Think how this could go wrong and how Amy could recover the conversation 29
- 30. Feedback How did itgo? How many steps did you come up with? What error messages did you come up with… Here is one we did earlier… 30
- 31. Think about… (whenstarting to write conversations..) ▪ Ask clarifying questions at start ▪ Make assumptions and build for there, make clarifications throughout ▪ Think about how to present choices (and cater for human’s limited working memory) ▪ Avoid overloading the skill with instructions ▪ Think personality and tone and vary responses ▪ Keep the conversation moving forward and bring user closer to their goal ▪ Consider how to repair flow if user goes off happy path – how to get user back on track or fail gracefully. 31
- 32.
- 33. WOZ Testing Testing voiceapp using WOZ Testing 33
- 34. Wizard of Oz(WOz) Approach In the WOz test, participants interact with a VUI such as Alexa, whom they believe to be ‘live’, but which is actually being operated by unseen researcher in an another room.
- 35. what the usersees the wizard Wizard of Oz in Voice usability testing
- 36. WOZ Testing 1. Understandmain use cases/tasks - user intents 2. Construct sample dialogue flows and responses for each task 3. Create a moderator script including scenarios and metrics for gathering feedback 4. Create audio files for each flow include messages for error recovery etc. (can use Amazon’s Polly or prototype tool such as Sayspring) 5. Connect audio files soundboard (can use soundboard lite) 6. Set up WOZ test 7. Record session (can use audacity, Evernote etc.) 36
- 37. Running a Voiceusability session 37 Research brief Test stimulus Test plan/moderator script Set up and record the session
- 38.
- 39.
- 40. 40 Welcome to UXScotland Image source: Unsplash photo @ugsdreamer Polly system Speech into text from Amazon Web Services
- 41. Moderator script 41 Very difficult Difficult Slightly difficult Neutral Slightlyeasy Easy Very easy 1 2 3 4 5 6 7
- 42. Run and recordsession 42 Amy How can I help?
- 43. What did youthink? Tasks/Flows? Language used? Amy’s Personality? Anything else? 43
- 44.
- 45. Finally… Some tipswhen testing Voice apps ▪ Test to see if your conversational flows match user intents ▪ It’s important not to mention command words or ask leading questions in TEST as we want to find our more what language and synonyms participants use in testing ▪ Check how your app handles feedback, contextual help and errors ▪ Collect feedback on Persona of app ▪ Use metrics such as ease of use and satisfaction to measure user experience ▪ Agile methodology and quick to iterate. *Remember it may be agile and quick to do but its important to follow a robust research process and have clear research aims and objectives 45
- 46.