Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Usability Lab within Agile (by Ian Franklin at NUX Leeds January 2018)


Published on

Ian Franklin from IdeaSmiths discussing fitting Usability Labs into Agile sprints.

Traditionally, usability labs took a long time to organise; often just a usability bug hunt and resulted in a lengthy report of recommendations that no one read and took weeks to produce.

This talk covers how to adapt the usability lab to include discovery and co-creation, yet still record results rigorously while completing analysis and reporting within a couple of days.

It also covers how to counter the common objections to user feedback (“its only 5 users”, “it’s just anecdotes”) and how to use the lab to get stakeholders on side.

  • Ryan Shed Plans 12,000 Shed Plans and Designs For Easy Shed Building! ▶▶▶
    Are you sure you want to  Yes  No
    Your message goes here

Usability Lab within Agile (by Ian Franklin at NUX Leeds January 2018)

  1. 1. Usability Lab within Agile Ian Franklin
  2. 2. What We Will Do •Labs within Agile •Why we do a usability lab •Issues of Usability Testing •Plan a session •Do a session •Analyse and report •Questions
  3. 3. Labs within Agile
  4. 4. User Centred Design and Agile Process Usability Labs Fit in these phases
  5. 5. Why do a lab?
  6. 6. 6 Real User Needs
  7. 7. User centred design NOT Client centred design
  8. 8. We need answers •Can the user use it? •From Discovery are the user needs valid? •In Lean UX – discovery in alpha •Is the MVP right? •Is the new stuff (from backlog) working? •Did we break it? •Is the site successful?
  9. 9. Create ownership Agile team: •Designers – design iterations •Product Owner – improving decisions •Delivery Manager – prioritisation •Techies – why the solution is the solution
  10. 10. Create ownership Stakeholders •Ammunition for political battles •See real users
  11. 11. Other benefits •Labs look good – Face Validity •very ‘sciency’ and ‘shiny’ •Fun for the team •Known technique – 50/60 years old •Rigorous methodology •Good Politics •Can fit into two week sprint
  12. 12. The Bigger Toolbox Usability Lab Usability Bug Hunt
  13. 13. The Bigger Toolbox
  14. 14. The Bigger Toolbox •Depth interviews •Naturalistic Ethnography/Contextual Inquiry •Co-creation workshops •Portable usability labs •Remote moderated & unmoderated •Pop-up •Guerrilla (&^%$£) – IF YOU REALLY MUST
  15. 15. Issues
  16. 16. Its only 5 Users Its just what someone said, its anecdotal Its not a scientific statistically representative sample Its all subjective, we need more hard evidence
  17. 17. Everyone knows your job better than you… …Because every other *&%^£)(*&^ person in your organisation is a Scientific Einstein… …Except YOU
  18. 18. “Real Science”
  19. 19. “Social Science”
  20. 20. The Geiger counter of user behaviour
  21. 21. Sample Size You only bring me 5 $%£^&ing Users? You useless F*&kWit!!!! Feel my Wrath and DIE in AGONY.
  22. 22. F**kWit Head Of
  23. 23. Sampling 101
  24. 24. Sampling 101
  25. 25. Sampling 101
  26. 26. Sampling 101
  27. 27. Sampling 101
  28. 28. Sampling 101 IF we are dealing with a … UNITARY concept… WITHOUT weighting
  29. 29. Not a scientific statistically representative sample BUT At prototype we only need good user representations
  30. 30. Big expensive prototype Only 3 Users - Test Pilots …sHH professional testers
  31. 31. Only 2 Users – Test Pilots
  32. 32. Just 5 users so… … involve 5 users every Agile sprint It all adds up and patterns emerge
  33. 33. Just qualitative so… … add some Quantitative - scales, SUS, counts of user behaviour
  34. 34. “Its subjective” NO! We are collecting OBJECTIVE observations of the users’ behaviour (EVENTS) and… … could even count it
  35. 35. Usability Lab: What can we do?
  36. 36. Plan a Session
  37. 37. Two week sprint •Week 1 •Get users (week before) •Plan and prepare •Get materials together •Try the session plan
  38. 38. Two week sprint •Week 2 •Do the day •Analysis •Report & Presentation
  39. 39. What do we need •Getting users •Profile and requirements •Market research recruiter •Location •Hire a lab/Hire a room •Go to users •Users come to you •Props and fake data
  40. 40. What can we do? •Usability bug hunt – usability •Test the Concept – wireframe paper and screenshots •Card sort/Tree tests •Co-creation - “show me an example” •Discovery interviews/Contextual inquiry •Supporting materials and processes
  41. 41. Session Plan •Scenario versus Tasks •How many tasks? • 2 to 3 – must do • 1 to 2 – nice if we can •Don’t forget breaks
  42. 42. A simple session plan •Background and informed consent •About the user and device/internet usage •Build rapport •The tasksscenario •Final questions •Best thing today •Worst thing today •One thing we need to change
  43. 43. Typical Hour •10 to 15 minutes background •20 minutes: discovery and context •20 minutes: user does tasks, uses the tech •10 minutes debrief and final questions
  44. 44. Who owns the plan •The Team! •Develop with UX and content designers •Agree with Product Owner and Delivery Manager •Clear link to the sprint and backlog •Clear hypotheses and aim
  45. 45. Kit •Devices – take your own •Screen wipes •Yellow stickies – shed loads •Sharpies - black •Props and fake data •Session plan •Roll of brown paper •Glue Prit sticks – spot weld stickies •Blu Tack •Scissors
  46. 46. Tools
  47. 47. Doing a Lab
  48. 48. Roles •Facilitators – work with the user •Observers – take notes •Designers •The team •Visitors – take notes (e.g. stakeholders) •Sticky collector(s) •Transcribers???????
  49. 49. Observation Room – ground rules •Snacks and refreshments •No side meetings •Devices off •No phone calls •No design decision conversations •Split observations and quotes
  50. 50. Everyone does work
  51. 51. Brown Paper Method •A long roll for each user •Put stickies on in timeline •Which follows the session plan •Roll up at end of each user •New blank roll on wall
  52. 52. Anatomy of a Sticky: Quote “ Yeah, no problem really [as adult] … but my kids, no, not sharing that” U 01 User number. Squarebrackets to give context. Ellipsis… for missing words, pauses One idea. Cansplit over two. Stickies must stand on their own If a quote use quote marks. NOT insights, not yet
  53. 53. Anatomy of a Sticky: Observation User had problems entering one time password. Error at input U 01 If observationno quotes, start with User NOT insights, not yet User number. One idea. Cansplit over two. Stickies must stand on their own
  54. 54. If nothing else •Put the participant code somewhere on the sticky. •Put quotes in “...” •Start observations with User •Write observations clearly •Tear stickies from the side •Ban biros
  55. 55. Analyse and Report: As a Team!
  56. 56. De-duplicationof same quotes and observations. Use best written and most readable to photo Collate all users onto one sheet
  57. 57. Affinity mapping •Put stickies on wall or brown paper •Read and have a quick think •Group into themes (emergent!) •Big Groups OK – so are Small •Read, think and group again •Use Orange stickies to label groups •Use Pink stickies for super-groups •Green for User Stories
  58. 58. Results Capture •Take photos of the analysis •it’s the evidence •Store in a project accessible way: •Sharepoint, Gdrive, Confluence •Videos – again store.
  59. 59. Report Reporting
  60. 60. Report •Photo the brown paper •Do a one pager of findings •Illustrative video clips 25secs t0 2mins •Show and Tell - slide set •Store in a project accessible way •audit trail of design decisions
  61. 61. Show and Tell Structure •Background & Hypotheses •What you did in the session •With whom •Photo of the day •Photo of the results •Video clips with sound •What next – how has the project been informed
  62. 62. Next steps •Work with UX designers and product owner to •Propose design changes •Propose user needs work (discovery) •Write new hypotheses •Then test again
  63. 63. Questions?