Your SlideShare is downloading. ×
Why keep it to yourself? Teaching everyone on the team to do usability testing
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Why keep it to yourself? Teaching everyone on the team to do usability testing


Published on

Techniques for teaching people on your development team how to gather data from users.

Techniques for teaching people on your development team how to gather data from users.

Published in: Design

  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. 1 Why keep it to yourself? ? Getting everyone on the team to do usability testing Dana Chisnell IA Summit 2010
  • 2. 2 Influence = Data + Gut
  • 3. 3 Influence = Data < Gut
  • 4. 4 Influence = Data x Gut
  • 5. 5 Influence = Data^Gut
  • 6. 6 Throughput is a problem.
  • 7. 7 Influencing design decisions
  • 8. 8
  • 9. 9 ? Dev IA IX Dev D D U
  • 10. 10 ? U
  • 11. 11 Dev D IX D IA Dev D
  • 12. 12 Not either/or
  • 13. 13 Do it together
  • 14. 14 U D D E V D DEV U I A D U
  • 15. 15 Yay! DEV DEV IA D IX D U
  • 16. 16 Influencing design decisions Give up control (you don’t have it, anyway) Get the team to value data over gut Research by lots of amateurs produces better results Otherwise, you’ll burn out Coach and advise instead Work on more interesting, harder questions
  • 17. 17 Improving the Gut Yay! DEV DEV IA D IX D U
  • 18. 18 Tasks Moderating Analysis
  • 19. 19 Tasks: Seeing through the users’ eyes Help each team member think of a user scenario they care about Act out the scenario as the user
  • 20. 20 App: Church social network ✤ Roles: ✤ Senior Pastor ✤ Social Networking Pastor ✤ Volunteer ✤ Lay Leader ✤ Member ✤ First time visitor ✤ Administrative Assistant
  • 21. 21 ✤ App: Organizing students to help homebound people vote ✤ User roles: ✤ Student ✤ Voter ✤ Organizer ✤ Trainer ✤ Scheduler
  • 22. 22 What happened?
  • 23. 23 Task scenarios set the scene, give context You want to create realistic and relatable stories
  • 24. 24 Moderating: Active presence Role play practice (in dry runs) Video of the moderator
  • 25. 25 Moderating, Impartial, unbiased observing not training No teaching! Listen and watch Ask open-ended questions: Why? How? What? Correct and train at the end
  • 26. 26 Who should moderate? Quick learner Develops rapport, Excellent memory Good listener empathetic
  • 27. 27 Flight attendant Ensuring the safety and comfort of the participant Joan Dorsey, American Airlines: Sportscaster Echoing with play-by-play Maximizing flow of information to observers John Madden, thisSpeed=15000 Scientist Planning, managing data, producing reports Marie Curie, Smithsonian Institution
  • 28. 28 Moderators: Run the session Observers: Note behaviors and quotes Participant: Do what you normally would do, try to think aloud
  • 29. 29 What to look for Hesitating Comments Questions Body language Behaviors
  • 30. 30 Technique Think aloud - ask: Review at end - ask: “Tell me what you’re “Walk me through what you doing” did” “Tell me what you’re “How’d that go?” Use the thinking” ballot as a guide for the discussion “What was confusing or frustrating?”
  • 31. 31 Review ✤ Participant: What was it like? ✤ Observers: What did you see? ✤ Moderators: What questions do you have?
  • 32. 32 Mid-session reminders “You are not being Meanwhile, you track time and tested” tasks remaining in the session “That’s useful Don’t be afraid to move on if feedback, thank something is taking too long you” Decide ahead what is low priority “Please tell me what or optional you’re thinking”
  • 33. 33 Narration Don’t be afraid to Examples: interact “I see you just clicked on the Zit button.” Say what you are “Sorry, I didn’t catch that. Would you say observing - don’t that again?” interpret “That really, really stinks? Could you say what about it stinks?”
  • 34. 34 Maximizing data ✤ If the participant says “hmmm” or “oops” or “I wonder...” Say: “What questions do you have right now?” ✤ If the participant is silent for 10 - 20 seconds (count!) Say: “What are you thinking?” ✤ If the participant stops because she thinks she’s done or she’s stuck - Summarize what you saw her do - Ask what she will do next
  • 35. 35 Analysis: Structured discussion Tell stories KJ - to set priorities Guess the reason - to exercise inference-making Observation-inference-opinion-theory
  • 36. 36 KJ Analysis reach consensus from subjective data similar to affinity diagramming invented by Jiro Kawakita objective, quick 8 simple steps
  • 37. 37 1. Focus question What needs to be fixed in Product X to improve the user experience? (observations, data) What obstacles do teams face in implementing UE practices? (opinion)
  • 38. 38 2. Organize the group Call together everyone concerned For user research, only those who observed Typically takes an hour
  • 39. 39 3. Put opinions or data on notes For a usability test, ask for observations (not inferences, not opinion) No discussion
  • 40. 40 4. Put notes on a wall Random Read others’ Add items No discussion
  • 41. 41 5. Group similar items In another part of the room Start with 2 items that seem like they belong together Place them together, one below the other Move all stickies Review and move among groups Every item in a group No discussion
  • 42. 42 6. Name each group Use a different color Each person gives each group a name Names must be noun clusters Split groups Join groups Everyone must give every group a name No discussion
  • 43. 43 7. Vote for the most important groups Democratic sharing of opinions Independent of influence or coercion Each person writes their top 3 Rank the choices from most to least important Record votes on the group sticky No discussion
  • 44. 44 8. Rank the groups Pull notes with votes Order by the number of votes Read off the groups Discuss combining groups Agreement must be unanimous Add combined groups’ votes together Stop at 3-5 clear top priorities
  • 45. 45
  • 46. 46 Observations to direction Quantifying subjective data in 4 easy steps
  • 47. 47 Observations Sources: What you saw usability testing What you heard user research sales feedback support calls and emails training
  • 48. 48 Inferences Judgements Conclusions Guesses Intuition
  • 49. 49 Opinions Review the inferences What are the causes? How likely is this inference to be the cause? How often did the observation happen? Are there any patterns in what kinds of users had issues?
  • 50. 50 Direction What’s the evidence for a design change? What does the strength of the cause suggest about a solution? Test theories
  • 51. 51 Resistance from the team I don’t have time It’s not my job I don’t know how What if I screw up
  • 52. 52 Resistance is futile.
  • 53. 53 Resistance from the team Exchange time in meetings for time with users It will make you better at your job I’ll help you learn You won’t; it doesn’t matter
  • 54. 54 Stop pouting. Yep, you’re qualified and they aren’t. Go peddle your fish. Haven’t you wanted more visibility?
  • 55. 55 Improving the Gut A lot of sloppy data is better than a little excellent data Trust your team to be delegated to Shift your mindset from your butt to their Gut
  • 56. 56 -+ Desktop X? Users Data ? Y? U Recommendations D Direction This?
  • 57. 57 Yay! DEV DEV IA D IX D U Or this?
  • 58. 58 Why keep it to yourself? IX Dev IA D U Dev D
  • 59. 59 Where to learn more Dana’s blog: http:// Download templates, examples, and links to other resources from
  • 60. 60 Dana Chisnell 415.519.1148