Why keep it to yourself? Teaching everyone on the team to do usability testing

1 Why keep it to yourself? ? Getting everyone on the team to do usability testing Dana Chisnell IA Summit 2010

2 Inﬂuence = Data + Gut

3 Inﬂuence = Data < Gut

4 Inﬂuence = Data x Gut

5 Inﬂuence = Data^Gut

6 Throughput is a problem.

7 Inﬂuencing design decisions

9 ? Dev IA IX Dev D D U

10 ? U

11 Dev D IX D IA Dev D

12 Not either/or

13 Do it together

14 U D D E V D DEV U I A D U

15 Yay! DEV DEV IA D IX D U

16 Inﬂuencing design decisions Give up control (you don’t have it, anyway) Get the team to value data over gut Research by lots of amateurs produces better results Otherwise, you’ll burn out Coach and advise instead Work on more interesting, harder questions

17 Improving the Gut Yay! DEV DEV IA D IX D U

18 Tasks Moderating Analysis

19 Tasks: Seeing through the users’ eyes Help each team member think of a user scenario they care about Act out the scenario as the user

20 App: Church social network ✤ Roles: ✤ Senior Pastor ✤ Social Networking Pastor ✤ Volunteer ✤ Lay Leader ✤ Member ✤ First time visitor ✤ Administrative Assistant

21 ✤ App: Organizing students to help homebound people vote ✤ User roles: ✤ Student ✤ Voter ✤ Organizer ✤ Trainer ✤ Scheduler

22 What happened?

23 Task scenarios set the scene, give context You want to create realistic and relatable stories

24 Moderating: Active presence Role play practice (in dry runs) Video of the moderator

25 Moderating, Impartial, unbiased observing not training No teaching! Listen and watch Ask open-ended questions: Why? How? What? Correct and train at the end

26 Who should moderate? Quick learner Develops rapport, Excellent memory Good listener empathetic

27 Flight attendant Ensuring the safety and comfort of the participant Joan Dorsey, American Airlines: http://aviationblog.dallasnews.com/archives/aviation-historytrivia/ Sportscaster Echoing with play-by-play Maximizing ﬂow of information to observers John Madden, http://www.forbes.com/2008/01/28/television-sports-madden-biz-sports-cx_lh_0128broadcasters_slide_3.html? thisSpeed=15000 Scientist Planning, managing data, producing reports Marie Curie, Smithsonian Institution http://photography.si.edu/SearchImage.aspx?t=5&id=3523&q=SIL14-C6-05

28 Moderators: Run the session Observers: Note behaviors and quotes Participant: Do what you normally would do, try to think aloud

29 What to look for Hesitating Comments Questions Body language Behaviors

30 Technique Think aloud - ask: Review at end - ask: “Tell me what you’re “Walk me through what you doing” did” “Tell me what you’re “How’d that go?” Use the thinking” ballot as a guide for the discussion “What was confusing or frustrating?”

31 Review ✤ Participant: What was it like? ✤ Observers: What did you see? ✤ Moderators: What questions do you have?

32 Mid-session reminders “You are not being Meanwhile, you track time and tested” tasks remaining in the session “That’s useful Don’t be afraid to move on if feedback, thank something is taking too long you” Decide ahead what is low priority “Please tell me what or optional you’re thinking”

33 Narration Don’t be afraid to Examples: interact “I see you just clicked on the Zit button.” Say what you are “Sorry, I didn’t catch that. Would you say observing - don’t that again?” interpret “That really, really stinks? Could you say what about it stinks?”

34 Maximizing data ✤ If the participant says “hmmm” or “oops” or “I wonder...” Say: “What questions do you have right now?” ✤ If the participant is silent for 10 - 20 seconds (count!) Say: “What are you thinking?” ✤ If the participant stops because she thinks she’s done or she’s stuck - Summarize what you saw her do - Ask what she will do next

35 Analysis: Structured discussion Tell stories KJ - to set priorities Guess the reason - to exercise inference-making Observation-inference-opinion-theory

36 KJ Analysis reach consensus from subjective data similar to afﬁnity diagramming invented by Jiro Kawakita objective, quick 8 simple steps

37 1. Focus question What needs to be ﬁxed in Product X to improve the user experience? (observations, data) What obstacles do teams face in implementing UE practices? (opinion)

38 2. Organize the group Call together everyone concerned For user research, only those who observed Typically takes an hour

39 3. Put opinions or data on notes For a usability test, ask for observations (not inferences, not opinion) No discussion

40 4. Put notes on a wall Random Read others’ Add items No discussion

41 5. Group similar items In another part of the room Start with 2 items that seem like they belong together Place them together, one below the other Move all stickies Review and move among groups Every item in a group No discussion

42 6. Name each group Use a different color Each person gives each group a name Names must be noun clusters Split groups Join groups Everyone must give every group a name No discussion

43 7. Vote for the most important groups Democratic sharing of opinions Independent of inﬂuence or coercion Each person writes their top 3 Rank the choices from most to least important Record votes on the group sticky No discussion

44 8. Rank the groups Pull notes with votes Order by the number of votes Read off the groups Discuss combining groups Agreement must be unanimous Add combined groups’ votes together Stop at 3-5 clear top priorities

46 Observations to direction Quantifying subjective data in 4 easy steps

47 Observations Sources: What you saw usability testing What you heard user research sales feedback support calls and emails training

48 Inferences Judgements Conclusions Guesses Intuition

49 Opinions Review the inferences What are the causes? How likely is this inference to be the cause? How often did the observation happen? Are there any patterns in what kinds of users had issues?

50 Direction What’s the evidence for a design change? What does the strength of the cause suggest about a solution? Test theories

51 Resistance from the team I don’t have time It’s not my job I don’t know how What if I screw up

52 Resistance is futile.

53 Resistance from the team Exchange time in meetings for time with users It will make you better at your job I’ll help you learn You won’t; it doesn’t matter

54 Stop pouting. Yep, you’re qualiﬁed and they aren’t. Go peddle your ﬁsh. Haven’t you wanted more visibility?

55 Improving the Gut A lot of sloppy data is better than a little excellent data Trust your team to be delegated to Shift your mindset from your butt to their Gut

56 -+ Desktop X? Users Data ? Y? U Recommendations D Direction This?

57 Yay! DEV DEV IA D IX D U Or this?

58 Why keep it to yourself? IX Dev IA D U Dev D

59 Where to learn more Dana’s blog: http:// usabilitytestinghowto.blogspot.com/ Download templates, examples, and links to other resources from www.wiley.com/go/usabilitytesting

60 Dana Chisnell dana@usabilityworks.net www.usabilityworks.net 415.519.1148

http://www.boxesandarrows.com	721
http://boxesandarrows.com	278
http://www.slideshare.net	21
http://xss.yandex.net	8
http://banda.publicsquarehq.cucinamedia.com	6
https://admin.instantpage.me	5
http://www.netvibes.com	3
http://uxlibrary.org	2
http://jeffhiles.com	2
http://meltwaternews.com	2
http://wiki.autodesk.com	1
http://dayangolden.com	1
http://www.newsblur.com	1
http://webcache.googleusercontent.com	1
http://boxesandarrows.mistermachinework.com	1

Why keep it to yourself? Teaching everyone on the team to do usability testing

by Dana Chisnell on Apr 15, 2010

Accessibility

Categories

Upload Details

Usage Rights

15 Embeds 1,053

Statistics

Why keep it to yourself? Teaching everyone on the team to do usability testing Presentation Transcript