Keeping Your Moderating Skills Sharp

2,023 views

Published on

Published in: Technology, Education
1 Comment
2 Likes
Statistics
Notes
No Downloads
Views
Total views
2,023
On SlideShare
0
From Embeds
0
Number of Embeds
95
Actions
Shares
0
Downloads
36
Comments
1
Likes
2
Embeds 0
No embeds

No notes for slide

Keeping Your Moderating Skills Sharp

  1. 1. Keeping Your Moderating Skills SharpSusan MercerSenior Experience Researchersmercer@madpow.com@susanamercer
  2. 2. The Importance of Neutrality•  Clients want objective answers•  Asking biased questions lead to biased answers•  Participants want to please us •  Providing types of feedback can subtly communicate our expectations •  They are more likely to answer future questions that way2
  3. 3. The Reality of Moderation•  Observed 17 practitioners•  Many usability professionals get sloppy: •  Ask leading questions •  Participant: “I would share this article with friends.” •  Moderator: “By printing it?” •  Ask closed-ended questions •  Moderator: “Would you print this?” •  Focus on known problems for “ammunition” •  Use biasing response phrases “Good job”, “Yes, that’s a problem.”3 Source: Norgaard & Hornbaek, 2006
  4. 4. What does this mean?•  Current practices are not consistent•  Moderating is taught through mentorship •  bad habits can easily be perpetuated•  We need to pay attention to our moderation and remain neutral to capture unbiased data. 4
  5. 5. 5 Rules of Great Moderating
(A Quick Review)
  6. 6. 5 Rules of Great Moderating1.  Know your goals2.  Stay neutral 3.  Be responsible4.  Be confident5.  Be genuine6
  7. 7. 1. Know Your Goals•  What are you trying to achieve? •  Formative •  Summative •  What topics are highest priority?These will influence •  What to focus on when time runs short •  How you fine-tune your moderating style7
  8. 8. 5 Rules of Great Moderating1.  Know your goals2.  Stay neutral 3.  Be responsible4.  Be confident5.  Be genuine8
  9. 9. 2. Stay Neutral•  Be Quiet!•  Don’t insinuate they gave a wrong answer ✘ “Why did you do that?” ✘ “What made you think to click that link?”•  Don’t put ideas into their heads ✘ “Does entering your SSN make you feel uncomfortable?”  “What do you think about entering your SSN?”•  Use their words •  If they call the dashboard a “chart area”, then call it a “chart area”•  Keep your tone naturally curious, not derisive •  “What did you expect here?”9
  10. 10. 2. Stay Neutral•  Don’t start with a verb ✘  id you see that? D ✘  as that difficult? W ✘ s this frustrating? I•  Start with What, When, Where, Why, How   hat would you change to make this easier? W   hen in this process would you be ready to buy? W   here did you expect to find that information? W   hy did you say this was easy? W   ow does that error message make you feel? H10
  11. 11. 2. Stay Neutral•  Use neutral, “continuing” response phrases•  Acknowledge that they gave feedback•  Don’t imply whether it is good or bad ✘ “Oh”, “Hmm”, “Interesting” - Assessing ✘ “OK”, “Yeah”, “That’s good” - Agreeing   Uh huh”, “Mhmm”, “Tell me more” – Continuing “•  Other tips to staying neutral?11 Source: Boren & Ramey, 2000
  12. 12. 5 Rules of Great Moderating1.  Know your goals2.  Stay neutral 3.  Be responsible4.  Be confident5.  Be genuine12
  13. 13. 3. Be ResponsibleYou have responsibilities to:•  Participant•  Company•  Future Users •  Study Integrity13
  14. 14. 3. Be ResponsibleSituations happen during testing.You may need to juggle your responsibilities.What are your priorities?•  Study Integrity•  Company•  Future Users•  Participant14
  15. 15. What to do?If…•  Participant overshares personal information•  The participant rants about the website and calls it “crap”15
  16. 16. 5 Rules of Great Moderating1.  Know your goals2.  Stay neutral 3.  Be responsible4.  Be confident5.  Be genuine16
  17. 17. 4. Be Confident•  Project confidence•  If you make a mistake, keep going•  Don’t over-apologize•  Pause if you need to collect your thoughts•  “Fake it until you make it”17
  18. 18. 5 Rules of Great Moderating1.  Know your goals2.  Stay neutral 3.  Be responsible4.  Be confident5.  Be genuine18
  19. 19. 5. Be Genuine•  Don’t play the notes on the page……play the music•  Don’t just read the questions from the page……have a conversation19
  20. 20. 5 Rules of Great Moderating1.  Know your goals2.  Stay neutral 3.  Be responsible4.  Be confident5.  Be genuine6. What else?20
  21. 21. Keeping sharp
  22. 22. Keep Improving1.  Identify your improvement areas •  Watch your videos •  Have colleagues give you feedback22
  23. 23. Keep Improving2.  Learn from others •  Watch others moderate •  Be a participant •  Listen to talk radio interviews23
  24. 24. Keep Improving3.  Practice skills in everyday life •  People watch (observe) in public •  Be quiet and listen in everyday conversations24
  25. 25. Your Thoughts?25
  26. 26. Thank You! (Full presentation is on Slideshare. Follow me on twitter for link) Susan Mercer Senior Experience Researcher smercer@madpow.com @susanamercer26
  27. 27. References / Photo CreditsBoren, T. and Ramey, J. (2000) Thinking aloud: reconciling theory and practice. IEEE Transactions on Professional Communication, 43 (3), 261-278.Dumas, J. and Loring, B. (2008) Moderating usability tests, Morgan Kaufman.Dumas, J. and Redish, J. (1999) A practical guide to usability testing, Intellect Ltd.Ericsson, K. and Simon, H. (1980) Verbal reports as data. Psychological review. 87 (3), 215-251.Krahmer, E. and Ummelen, N. (2004) Thinking about thinking aloud: A comparison of two verbal protocols for usability testing. IEEE Transactions on Professional Communication, 47 (2), 105-117.Norgaard, M. and Hornbaek, K. (2006) What do usability evaluators do in practice? An exploratory study of think-aloud testing. DIS 2006, 209-218.Slide 4: Samantha Louras Photography Slide 5: flickr: torgeauxSlide 14: flickr: mkorcusa, flickr: Calsidyrose, flickr: marioanima, flickr: breity, flickr: Qole PejorianSlides 27, 29 and 30: flickr: Raphael Quinet, flickr: acme, flickr: betsyweber, flickr: KirkOlsSlide 32: flickr: bberburbSlide 34: flickr: Tulane Public RelationsSlide 35: flickr: apdk, flickr: marktristanSlide 38: flickr: Raphael QuinetSlide 39: flickr: l-i-n-kSlide 40: flickr: loppear27

×