Why you should always test with users...
•Download as PPTX, PDF•
5 likes•15,225 views
...and why ‘guerrilla’ is better than nothing
Recommended
Recommended
More Related Content
Similar to Why you should always test with users...
Similar to Why you should always test with users... (20)
Recently uploaded
Recently uploaded (20)
Why you should always test with users...
- 1. #UXDclass Why you should always test with users @quiffboy @amazeltd #UXDclass Tuesday 10th Feb 2015 ‘guerrilla’ is and why gorillas are better than nothing
- 2. #UXDclass Leisa is Head of User Research @ GDS, GOV.UK
- 3. #UXDclass “…testing with customers…”Me Customers have a monetary value
- 4. #UXDclass
- 6. #UXDclass
- 7. #UXDclass
- 8. #UXDclass
- 9. #UXDclass
- 11. #UXDclass Common types of usability testing • Moderated lab testing • Remote testing • Eye tracking • Guerrilla testing
- 12. #UXDclass Moderated lab testing “Observe people using the product to discover…how well test subjects respond in four areas: efficiency, accuracy, recall, and emotional response.” Wikipedia cabinetoffice.gov.uk
- 13. #UXDclass
- 15. #UXDclass Eye tracking “Eye tracking is the process of measuring either the point of gaze (where one is looking) or the motion of an eye relative to the head.” Wikipedia grokdotcom.com
- 16. #UXDclass Eye tracking: complex & intrusive hardware, right? fffff.at acuity-ets.com Not so scary anymore
- 17. #UXDclass Eye tracking: findings are still scary though • Generates lots of complex graphs and diagrams • Needs experts to interpret results • Time consuming & expensive Heatmap from kunocreative.com Gaze plot from uxmatters.com
- 18. #UXDclass But it’s a no brainer, right? You should test your stuff with real people
- 19. #UXDclass • http://econsultancy.com/uk/reports/us er-experience-survey-report • http://nuxuk.org/2013/09/05/ux- challenge-choose-right-approach- guest-blog-ian-franklin-freelance-ux- consultant/ User Experience Survey 2013
- 22. #UXDclass So why don’t we always test?
- 23. #UXDclass So why don’t we always test? Too expensive Time or resource consuming Not a primary concern
- 24. #UXDclass £15 for 15 minutes • Based on moderated lab testing • Uses think-aloud protocols • Test in guerrilla environments • Recruit “in the wild” • Prototype “good enough” • Focus on key tasks • Report on problem areas • Client expectations are focussed
- 27. #UXDclassCase study: ASICS America
- 28. #UXDclass
- 29. #UXDclass + + Axure RP Pro Silverback (optional)
- 30. #UXDclass
- 31. #UXDclass What we did Prep • Recruit from same office building – posters in lobby • Identified 2-3 key user journeys & features • Prototype new features for journeys – medium fidelity Test • Test in spare meeting room in Amaze office • Slots 12-2pm and 4-6pm Report • Brief, focussed report - bullet point list • Export & upload videos & prototype • Talk client through findings
- 32. #UXDclass What we did Prep • 3 days Test • 1 day Report • 1 day
- 33. #UXDclass What we did Prep • 3 days Test • 1 day Report • 1 day 1 week start to finish Easily digestible report Actionable findings
- 34. #UXDclass • Make your report a one-pager • Bullet-point list not reams of copy • Make findings actionable • Highlight when a test passes • Focus on the things you can fix The report
- 35. #UXDclass Success: All participants were able to create a new to-do list item. Observation: A number of participants commented that they expected the newly created to-do list item to automatically appear on all their devices. Issue: 3 participants did not understand the flag icon ⚐ was for setting a reminder for their to-do list item. Consider adding a label or using a different icon.
- 36. #UXDclass One day to write your test report
- 38. #UXDclass 1) Avoid recruiting experts
- 39. #UXDclass 2) Test small, test early, test often
- 40. #UXDclass 3) If you can, test in real environments
- 41. #UXDclass 4) Focus on what needs fixing (or refining)
- 42. #UXDclass 5) Give tasks, not instructions
- 43. #UXDclass 6) Rotate order of tasks
- 44. #UXDclass 7) Your prototype’s fidelity matters, but it’s not critical
- 45. #UXDclass 8) Five people’s opinions are not conclusive proof
- 46. #UXDclass 9) People say the darndest things
- 47. #UXDclass 10) “Testing with 1 person is 100% better than testing with none” Steve Krug “Rocket Surgery Made Easy”
- 48. #UXDclass Useful links • Why You Only Need to Test with 5 Users http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ • The Art of Guerrilla Usability Testing http://www.uxbooth.com/articles/the-art-of-guerilla-usability-testing/ • Remote Usability and UX Research Tools http://remoteresear.ch/tools/ • New to Usability Testing? You’re not alone! http://userexperiencerocks.wordpress.com/2013/07/16/getting-started-with-usability- testing/ • User Experience Survey 2013 http://econsultancy.com/uk/reports/user-experience-survey-report http://nuxuk.org/2013/09/05/ux-challenge-choose-right-approach-guest-blog-ian- franklin-freelance-ux-consultant/ • Steve Krug: “Don’t make me think” & “Rocket Surgery Made Easy” and more… http://www.sensible.com
- 49. #UXDclass Thank you t: @quiffboy t: @amazeltd w: www.amaze.com
Editor's Notes
- Pet hate - “User testing” GDS = Government Digital Service in the Cabinet Office
- I sometimes like to reframe it as Testing with Customers (not “users”) Makes it commercial Customers = revenue
- 3 of my favourite things….
- Senior UX Consultant at amaze Used to be a front end dev Drifted into UX in 2005 Been doing this for 10 years Joined amaze in April 2012 Amaze founded in 1995 Lots and lots of people (about 220!) Offices in Manchester, Liverpool & London This is the Manchester office opposite the old Hacienda Full service consultancy for B2C and B2B: Strategy & creative, Design & build, Digital commerce and technology, Integrated comms, Global implementation support We’re quite good…
- One of the NUX organisers Free UX meet-ups and big all-day conferences in Manchester, Leeds and Newcastle
- Most common form of usability testing 1 hour sessions Lots of findings Lots of dedicated consultancies for this Also many digital agencies do it (we do it) Number of one hour session, with full written reports Audio and video capture, edited highlights etc etc Often clients watch from behind one way mirror Worked with BBC for Cbeebies William Hill anecdote of screaming at screen – needs to be soundproofed
- This sort of thing Pros: Users are often in a more informal environment (comfortable and relaxed) Great for bringing in teams together to observe users (and discuss) Help and support when defining tasks and scenarios Less likely to introduce moderator bias Detailed external provider analysis and reports More likely to use advanced data collection (Eye tracking) Cons: Can be costly Less likely to want to invest in early stage testing (Prototypes and early beta code) Less likely to perform iterative tests
- Browser toolbar & plugin gives users tasks and captures screen & video/audio Consultancies have a pool of people they use Good for quantitative testing – large generate enormous amount of data and feedback from participants Answering specific questions about how people use a user interface for relatively simple and straightforward tasks
- “sciency” Clients love it
- Perception vs reality
- Despite the complicated stuff, we know it should be done But how can we make it manageable?
- What Users Do & Econsultancy Report into attitudes and approaches to UX
- One area was why don’t you test? More reasons: Findings not actionable Will create more work “It can’t be that bad” “You should already know”
- One area was why don’t you test? More reasons: Findings not actionable Will create more work “It can’t be that bad” “You should already know”
- Our answer to this is 15 15 Environments – has ecological validity, behaviours that occur in natural settings, not lab environment We managed to achieve some of these, but not all
- Sweetspot 5 users = 85% of issues / Multiple iterations Saturation at about 10 users Exceptions: Quantitative studies (aiming at statistics, not insights): Test at least 20 users to get statistically significant numbers; tight confidence intervals require even more users Card sorting: Test at least 15 users Eyetracking: Test 39 users if you want stable heatmaps Any users are better than none
- The solution
- Take ASICS from non-transactional to transactional Requires whole sales, warehouse and distribution network, and customer services establishing America - live Japan - live Europe - soon Australia Then on to China, South America and rest of world
- Put these up in the lifts and the lobby
- Use these Only need Vimeo if you’re sharing the videos Only need the Pro version of Vimeo if you want to password-protect them
- We did some stuff – it looked like this
- Identified 2-3 key user journeys & features Prototype new features for journeys – medium fidelity Recruit from same office building – posters in lobby Test in spare meeting room in Amaze office Slots 12 - 2pm & 4 - 6pm Brief, focussed report - bullet point list will suffice Export & upload videos & prototype
- Identified 2-3 key user journeys & features Prototype new features for journeys – medium fidelity Recruit from same office building – posters in lobby Test in spare meeting room in Amaze office Slots 12 - 2pm & 4 - 6pm Brief, focussed report - bullet point list will suffice Export & upload videos & prototype
- Identified 2-3 key user journeys & features Prototype new features for journeys – medium fidelity Recruit from same office building – posters in lobby Test in spare meeting room in Amaze office Slots 12 - 2pm & 4 - 6pm Brief, focussed report - bullet point list will suffice Export & upload videos & prototype
- Eg: IT and design bods Unless they really are your audience
- Eg: IT and design bods Unless they really are your audience
- 15 minutes isn’t long - 2 mins will be intro - People like to talk Focus on 2 or 3 key areas for iteration You will have more iterations to test other stuff Small guerrilla throughout the project to keep it on track, but then a big lab test at the end to rubber stamp it for key stakeholders
- 15 minutes isn’t long - 2 mins will be intro - People like to talk Focus on 2 or 3 key areas for iteration You will have more iterations to test other stuff Small guerrilla throughout the project to keep it on track, but then a big lab test at the end to rubber stamp it for key stakeholders
- Basic principle of testing “You need a new pair of running shoes, what would you do?” not “Click on ‘running’ and find a pair of green size 8s suitable for road or trail”
- ABC BAC CAB Etc Eg: find returns info – could have seen during “buy a shoe” task Order effects
- Important thing is prototype should look realistic enough to be usable Don’t test wireframes Having said that, you can “explain away” inconsistencies or quirks during the session
- You’re looking for something you can make a decision based on Not statistically valid rigorous data Consider remote testing if you need to thrown numbers at something
- Never cease to be amazed by what people say Have open mind Be flexible Its fun, amusing, and exhausting