Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Ux scot voice usability testing with woz - ar and sf - june 2019

79 views

Published on

Usability testing for Voice applications
Abi Reynolds
Steven Fullerton

Published in: Design
  • DOWNLOAD FULL BOOKS, INTO AVAILABLE FORMAT ......................................................................................................................... ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. PDF EBOOK here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. EPUB Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... 1.DOWNLOAD FULL. doc Ebook here { https://tinyurl.com/yxufevpm } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Ux scot voice usability testing with woz - ar and sf - june 2019

  1. 1. Voice recognition in Scotland? 1Source :https://giphy.com
  2. 2. Designing Voice application and testing with Wizard of Oz Abi Reynolds Steven Fullerton June 2019 2
  3. 3. 3 Abi Reynolds Principal UX consultant @AbiJReynolds Steven Fullerton UX consultant @StevenFullerton Who are we?
  4. 4. Why are you interested in Voice? 4
  5. 5. Our Role in Conversation Design today. 5Image source: unsplash photo @bradneathery
  6. 6. Good and bad implementation 6
  7. 7. Good and bad implementation 7
  8. 8. Why Voice? 8
  9. 9. Why Voice? 9
  10. 10. What do we mean by voice? Google Home Amazon Echo 10Image sources: unsplash photos @andresurena and @@benceboros
  11. 11. Designing conversations 11 Keeping the user at the centre What could a user centred design process for Voice look like?
  12. 12. Discovery Requirements= Intents Script= sample dialogues Prototype = dialogue flows Test =WOZ test/iterate Launch 12 Research DesignTest
  13. 13. Intents = Purpose 13Source :https://giphy.com
  14. 14. What does a smart speaker hear? 14 Alexa, Ask Tile to find my Keys
  15. 15. What does a smart speaker hear? 15 Alexa, Ask Tile to find my Keys Utterance
  16. 16. What does a smart speaker hear? 16 Alexa, Ask Tile to find my Keys Wake Word Utterance
  17. 17. What does a smart speaker hear? 17 Alexa, Ask Tile to find my Keys Wake Word Invocation Utterance
  18. 18. What does a smart speaker hear? 18 Alexa, Ask Tile to find my Keys Wake Word Invocation Intent Slot Utterance
  19. 19. 19 “Design for how people talk, rather than how you wish them to talk.” - Cathy Pearl
  20. 20. Amy the Hotel Concierge ▪ Our business objective: To cut down on calls to reception (save costs). ▪ Our customer experience objective: We think Amy will provide an innovative and useful service that will add to the overall the guest experience. A voice app should be easy to use hands free within hotel room whilst guests are doing other things. 20
  21. 21. Activity – 10 Minutes Start the conversation! Write down: • A two way conversation between a customer who wants to know when checkout is and the smart assistant concierge. • The concierge can upsell late checkout at a cost of £10, IF the customer wants it. Create a short script based on the scenario above and work in pairs or small groups… 21 Activity – 10 Minutes Start the conversation! Write down: • A two way conversation between a customer who wants to know when checkout is and the smart assistant concierge. • The concierge can upsell late checkout at a cost of £10, IF the customer wants it. Create a short script based on the scenario above and work in pairs or small groups… 22
  22. 22. Feedback How did it go? How many different utterances did we get? Lets act out your script How could the flow be improved or made to feel more natural? 22
  23. 23. Discovery Requirements= Intents Script= sample dialogues Prototype = dialogue flows Test =WOZ test/iterate Launch 23
  24. 24. 24Source NNG Wireflow https://www.nngroup.com/articles/wireflows/
  25. 25. Dialogue flows gives you an overview of all directions the conversations can go (based on intents). It plots how you plan to respond to questions and guide the user towards their goal. 25
  26. 26. 26 https://dialogflow.com
  27. 27. Activity – 10 Minutes Complete the extended check-out flow The hotel offers extended check out at 3pm for an extra 10 pounds, which can be added to the room bill Work in groups and using the sheets provided 1. Complete the steps in the path until the user reaches their goal 27
  28. 28. 28
  29. 29. Remember 1. Core conversation - Y/N proceed. Until end of task 2. Prompt to go forward or finish task 3. Think how this could go wrong and how Amy could recover the conversation 29
  30. 30. Feedback How did it go? How many steps did you come up with? What error messages did you come up with… Here is one we did earlier… 30
  31. 31. Think about… (when starting to write conversations..) ▪ Ask clarifying questions at start ▪ Make assumptions and build for there, make clarifications throughout ▪ Think about how to present choices (and cater for human’s limited working memory) ▪ Avoid overloading the skill with instructions ▪ Think personality and tone and vary responses ▪ Keep the conversation moving forward and bring user closer to their goal ▪ Consider how to repair flow if user goes off happy path – how to get user back on track or fail gracefully. 31
  32. 32. 32Source Google https://developers.google.com/actions/downloads/conversational-repair.pdf
  33. 33. WOZ Testing Testing voice app using WOZ Testing 33
  34. 34. Wizard of Oz (WOz) Approach In the WOz test, participants interact with a VUI such as Alexa, whom they believe to be ‘live’, but which is actually being operated by unseen researcher in an another room.
  35. 35. what the user sees the wizard Wizard of Oz in Voice usability testing
  36. 36. WOZ Testing 1. Understand main use cases/tasks - user intents 2. Construct sample dialogue flows and responses for each task 3. Create a moderator script including scenarios and metrics for gathering feedback 4. Create audio files for each flow include messages for error recovery etc. (can use Amazon’s Polly or prototype tool such as Sayspring) 5. Connect audio files soundboard (can use soundboard lite) 6. Set up WOZ test 7. Record session (can use audacity, Evernote etc.) 36
  37. 37. Running a Voice usability session 37 Research brief Test stimulus Test plan/moderator script Set up and record the session
  38. 38. Research brief 38
  39. 39. Test Stimulus 39 1 2 3
  40. 40. 40 Welcome to UX Scotland Image source: Unsplash photo @ugsdreamer Polly system Speech into text from Amazon Web Services
  41. 41. Moderator script 41 Very difficult Difficult Slightly difficult Neutral Slightly easy Easy Very easy 1 2 3 4 5 6 7
  42. 42. Run and record session 42 Amy How can I help?
  43. 43. What did you think? Tasks/Flows? Language used? Amy’s Personality? Anything else? 43
  44. 44. 44 https//:tortu.io Prototyping tools https://dialogflow.com https://www. sayspring.com https://botmock.com
  45. 45. Finally… Some tips when testing Voice apps ▪ Test to see if your conversational flows match user intents ▪ It’s important not to mention command words or ask leading questions in TEST as we want to find our more what language and synonyms participants use in testing ▪ Check how your app handles feedback, contextual help and errors ▪ Collect feedback on Persona of app ▪ Use metrics such as ease of use and satisfaction to measure user experience ▪ Agile methodology and quick to iterate. *Remember it may be agile and quick to do but its important to follow a robust research process and have clear research aims and objectives 45
  46. 46. @UserVision www.uservision.co.uk Abi Reynolds @Abijreynolds abi@uservision.co.uk hello@uservision.co.uk Steven Fullerton @StevenFullerton Steven.fullerton@uservision.co.uk

×