1



Why keep it to
yourself?                      ?
Getting everyone on the team
to do usability testing




Dana Chisnel...
2




Influence = Data + Gut
3




Influence = Data < Gut
4




Influence = Data x Gut
5




Influence = Data^Gut
6




Throughput is a problem.
7




Influencing design decisions
8
9




    ?
         Dev
               IA
    IX
                    Dev
    D
                          D
U
10




?

U
11




         Dev

D


               IX
                D
    IA              Dev


         D
12




Not either/or
13




Do it together
14




                                  U


                          D   D
                              E
             ...
15




                              Yay!




                DEV
     DEV
           IA         D
IX
D
                  ...
16



Influencing design
decisions
Give up control (you don’t have it, anyway)

Get the team to value data over gut

Resear...
17




Improving the Gut
                                  Yay!




                    DEV
         DEV
               IA...
18




Tasks
Moderating
Analysis
19




Tasks: Seeing through the users’ eyes
 Help each team member think of a user scenario
 they care about
 Act out the...
20



App: Church social network
✤   Roles:
     ✤   Senior Pastor
     ✤   Social Networking Pastor
     ✤   Volunteer
  ...
21




✤ App:
  Organizing students to help homebound people
  vote
✤ User roles:

  ✤ Student

  ✤ Voter

  ✤ Organizer

...
22




What happened?
23




Task scenarios set the scene, give
context
You want to create realistic and relatable
stories
24




Moderating: Active presence
 Role play practice (in dry runs)
 Video of the moderator
25




Moderating,    Impartial, unbiased
               observing


not training   No teaching!

               Listen an...
26




 Who should moderate?




Quick learner   Develops rapport,   Excellent memory   Good listener
                empa...
27


     Flight attendant
     Ensuring the safety and comfort of the participant


Joan Dorsey, American Airlines: http:...
28




Moderators:
Run the session
Observers:
Note behaviors and quotes
Participant:
Do what you normally would do,
try to...
29




What to look for   Hesitating
                   Comments
                   Questions
                   Body lang...
30




Technique
Think aloud - ask:     Review at end - ask:
“Tell me what you’re   “Walk me through what you
doing”      ...
31




Review


✤ Participant: What was it like?
✤ Observers: What did you see?
✤ Moderators: What questions do you
  have?
32




Mid-session reminders
“You are not being     Meanwhile, you track time and
tested”                tasks remaining i...
33




Narration
Don’t be afraid to   Examples:
interact             “I see you just clicked on the Zit button.”
Say what ...
34




Maximizing data

✤   If the participant says “hmmm” or “oops” or “I wonder...”
    Say: “What questions do you have...
35




Analysis: Structured discussion
 Tell stories
 KJ - to set priorities
 Guess the reason - to exercise inference-mak...
36




KJ Analysis

 reach consensus from subjective data

 similar to affinity diagramming

 invented by Jiro Kawakita

 o...
37



1. Focus
question
What needs to be fixed in Product
X to improve the user experience?
(observations, data)

What obst...
38



2. Organize the
group
Call together everyone concerned

For user research, only those who
observed

Typically takes ...
39




3. Put opinions
or data on notes
For a usability test, ask for
observations

(not inferences, not opinion)



No di...
40



4. Put notes on
a wall
Random

Read others’

Add items



No discussion
41



5. Group
similar items
In another part of the room

Start with 2 items that seem like
they belong together

Place th...
42



6. Name each
group
Use a different color

Each person gives each group a
name

Names must be noun clusters

Split gr...
43




7. Vote for the most
important groups
Democratic sharing of opinions

Independent of influence or
coercion

Each per...
44



8. Rank the
groups
Pull notes with votes

Order by the number of votes

Read off the groups

Discuss combining group...
45
46



Observations
to direction
Quantifying subjective data in 4
easy steps
47




Observations
                   Sources:

   What you saw     usability testing

  What you heard    user research
...
48




Inferences
             Judgements

             Conclusions

             Guesses

             Intuition
49




Opinions
           Review the inferences

           What are the causes?

           How likely is this inference...
50




Direction
            What’s the evidence for a design change?

            What does the strength of the cause
   ...
51




Resistance from the team
  I don’t have time
  It’s not my job
  I don’t know how
  What if I screw up
52




Resistance is futile.
53




Resistance from the team
  Exchange time in meetings for time with users
  It will make you better at your job
  I’...
54




Stop pouting.
 Yep, you’re qualified and they aren’t.
 Go peddle your fish.
 Haven’t you wanted more visibility?
55




Improving the Gut
A lot of sloppy data is better than a little excellent
data

Trust your team to be delegated to

...
56




                                     -+   Desktop
                                                    X?
Users
    ...
57




                                Yay!




                  DEV
     DEV
             IA         D
IX
D
            ...
58




Why keep it to yourself?

                             IX
    Dev   IA
               D   U   Dev
                 ...
59




Where to learn more


    Dana’s blog: http://
    usabilitytestinghowto.blogspot.com/

    Download templates, exa...
60



Dana
Chisnell
dana@usabilityworks.net

www.usabilityworks.net
415.519.1148
Upcoming SlideShare
Loading in...5
×

Why keep it to yourself? Teaching everyone on the team to do usability testing

4,143

Published on

Techniques for teaching people on your development team how to gather data from users.

Published in: Design

Transcript of "Why keep it to yourself? Teaching everyone on the team to do usability testing"

  1. 1. 1 Why keep it to yourself? ? Getting everyone on the team to do usability testing Dana Chisnell IA Summit 2010
  2. 2. 2 Influence = Data + Gut
  3. 3. 3 Influence = Data < Gut
  4. 4. 4 Influence = Data x Gut
  5. 5. 5 Influence = Data^Gut
  6. 6. 6 Throughput is a problem.
  7. 7. 7 Influencing design decisions
  8. 8. 8
  9. 9. 9 ? Dev IA IX Dev D D U
  10. 10. 10 ? U
  11. 11. 11 Dev D IX D IA Dev D
  12. 12. 12 Not either/or
  13. 13. 13 Do it together
  14. 14. 14 U D D E V D DEV U I A D U
  15. 15. 15 Yay! DEV DEV IA D IX D U
  16. 16. 16 Influencing design decisions Give up control (you don’t have it, anyway) Get the team to value data over gut Research by lots of amateurs produces better results Otherwise, you’ll burn out Coach and advise instead Work on more interesting, harder questions
  17. 17. 17 Improving the Gut Yay! DEV DEV IA D IX D U
  18. 18. 18 Tasks Moderating Analysis
  19. 19. 19 Tasks: Seeing through the users’ eyes Help each team member think of a user scenario they care about Act out the scenario as the user
  20. 20. 20 App: Church social network ✤ Roles: ✤ Senior Pastor ✤ Social Networking Pastor ✤ Volunteer ✤ Lay Leader ✤ Member ✤ First time visitor ✤ Administrative Assistant
  21. 21. 21 ✤ App: Organizing students to help homebound people vote ✤ User roles: ✤ Student ✤ Voter ✤ Organizer ✤ Trainer ✤ Scheduler
  22. 22. 22 What happened?
  23. 23. 23 Task scenarios set the scene, give context You want to create realistic and relatable stories
  24. 24. 24 Moderating: Active presence Role play practice (in dry runs) Video of the moderator
  25. 25. 25 Moderating, Impartial, unbiased observing not training No teaching! Listen and watch Ask open-ended questions: Why? How? What? Correct and train at the end
  26. 26. 26 Who should moderate? Quick learner Develops rapport, Excellent memory Good listener empathetic
  27. 27. 27 Flight attendant Ensuring the safety and comfort of the participant Joan Dorsey, American Airlines: http://aviationblog.dallasnews.com/archives/aviation-historytrivia/ Sportscaster Echoing with play-by-play Maximizing flow of information to observers John Madden, http://www.forbes.com/2008/01/28/television-sports-madden-biz-sports-cx_lh_0128broadcasters_slide_3.html? thisSpeed=15000 Scientist Planning, managing data, producing reports Marie Curie, Smithsonian Institution http://photography.si.edu/SearchImage.aspx?t=5&id=3523&q=SIL14-C6-05
  28. 28. 28 Moderators: Run the session Observers: Note behaviors and quotes Participant: Do what you normally would do, try to think aloud
  29. 29. 29 What to look for Hesitating Comments Questions Body language Behaviors
  30. 30. 30 Technique Think aloud - ask: Review at end - ask: “Tell me what you’re “Walk me through what you doing” did” “Tell me what you’re “How’d that go?” Use the thinking” ballot as a guide for the discussion “What was confusing or frustrating?”
  31. 31. 31 Review ✤ Participant: What was it like? ✤ Observers: What did you see? ✤ Moderators: What questions do you have?
  32. 32. 32 Mid-session reminders “You are not being Meanwhile, you track time and tested” tasks remaining in the session “That’s useful Don’t be afraid to move on if feedback, thank something is taking too long you” Decide ahead what is low priority “Please tell me what or optional you’re thinking”
  33. 33. 33 Narration Don’t be afraid to Examples: interact “I see you just clicked on the Zit button.” Say what you are “Sorry, I didn’t catch that. Would you say observing - don’t that again?” interpret “That really, really stinks? Could you say what about it stinks?”
  34. 34. 34 Maximizing data ✤ If the participant says “hmmm” or “oops” or “I wonder...” Say: “What questions do you have right now?” ✤ If the participant is silent for 10 - 20 seconds (count!) Say: “What are you thinking?” ✤ If the participant stops because she thinks she’s done or she’s stuck - Summarize what you saw her do - Ask what she will do next
  35. 35. 35 Analysis: Structured discussion Tell stories KJ - to set priorities Guess the reason - to exercise inference-making Observation-inference-opinion-theory
  36. 36. 36 KJ Analysis reach consensus from subjective data similar to affinity diagramming invented by Jiro Kawakita objective, quick 8 simple steps
  37. 37. 37 1. Focus question What needs to be fixed in Product X to improve the user experience? (observations, data) What obstacles do teams face in implementing UE practices? (opinion)
  38. 38. 38 2. Organize the group Call together everyone concerned For user research, only those who observed Typically takes an hour
  39. 39. 39 3. Put opinions or data on notes For a usability test, ask for observations (not inferences, not opinion) No discussion
  40. 40. 40 4. Put notes on a wall Random Read others’ Add items No discussion
  41. 41. 41 5. Group similar items In another part of the room Start with 2 items that seem like they belong together Place them together, one below the other Move all stickies Review and move among groups Every item in a group No discussion
  42. 42. 42 6. Name each group Use a different color Each person gives each group a name Names must be noun clusters Split groups Join groups Everyone must give every group a name No discussion
  43. 43. 43 7. Vote for the most important groups Democratic sharing of opinions Independent of influence or coercion Each person writes their top 3 Rank the choices from most to least important Record votes on the group sticky No discussion
  44. 44. 44 8. Rank the groups Pull notes with votes Order by the number of votes Read off the groups Discuss combining groups Agreement must be unanimous Add combined groups’ votes together Stop at 3-5 clear top priorities
  45. 45. 45
  46. 46. 46 Observations to direction Quantifying subjective data in 4 easy steps
  47. 47. 47 Observations Sources: What you saw usability testing What you heard user research sales feedback support calls and emails training
  48. 48. 48 Inferences Judgements Conclusions Guesses Intuition
  49. 49. 49 Opinions Review the inferences What are the causes? How likely is this inference to be the cause? How often did the observation happen? Are there any patterns in what kinds of users had issues?
  50. 50. 50 Direction What’s the evidence for a design change? What does the strength of the cause suggest about a solution? Test theories
  51. 51. 51 Resistance from the team I don’t have time It’s not my job I don’t know how What if I screw up
  52. 52. 52 Resistance is futile.
  53. 53. 53 Resistance from the team Exchange time in meetings for time with users It will make you better at your job I’ll help you learn You won’t; it doesn’t matter
  54. 54. 54 Stop pouting. Yep, you’re qualified and they aren’t. Go peddle your fish. Haven’t you wanted more visibility?
  55. 55. 55 Improving the Gut A lot of sloppy data is better than a little excellent data Trust your team to be delegated to Shift your mindset from your butt to their Gut
  56. 56. 56 -+ Desktop X? Users Data ? Y? U Recommendations D Direction This?
  57. 57. 57 Yay! DEV DEV IA D IX D U Or this?
  58. 58. 58 Why keep it to yourself? IX Dev IA D U Dev D
  59. 59. 59 Where to learn more Dana’s blog: http:// usabilitytestinghowto.blogspot.com/ Download templates, examples, and links to other resources from www.wiley.com/go/usabilitytesting
  60. 60. 60 Dana Chisnell dana@usabilityworks.net www.usabilityworks.net 415.519.1148
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×