0
Boston	  CHI	  Monthly	  Mee>ng	  STRENGTHENING	  DESIGN	  RESEARCH	  	  MODERATION	  SKILLS	  	  	  APRIL	  9,	  2013	  	...
Strengthening Moderating and Interviewing Skills§  Once I learned the basics§  And ran a couple of studies,§  Now what?...
Story TimeDon’t play the notes on the page……play the musicThis applies to moderating andinterviewing too.Don’t just read t...
The Art of Moderating and Interviewing§  Not just reading questions§  It’s also…  §  Building rapport with the particip...
What Is Our Goal?§  To elicit honest thoughts from participants in an unbiased manner.§  “Would you like to try one of m...
The Rules of PolitenessLinguistic anthropologists have studied this phenomenonThe Rules of Politeness 1                   ...
How Do We Get Beyond Politeness?§  In other words, we’re often polite rather than honest to strangers.§  As human beings...
How Do We Get Beyond Politeness?Build their trust     Politeness                            Trust                       Fa...
Building Trust in an Interview1.  Be trustworthy upfront2.  Build rapport3.  Engage in comfortable conversation4.  Stay ne...
1. Be Trustworthy Upfront§          Confidentiality      §      How are you recording what is said?      §      Who is l...
2. Build Rapport                                                              A	  persons	  name	  is	  §          Greeti...
2. Build Rapport§          Be Empathetic      §      Apologize if they had trouble finding the office      §      Show yo...
3. Engage in Comfortable ConversationTwo schools of usability testing moderating techniques§  Moderator keeps silent and ...
3. Engage in Comfortable ConversationWhat is a Comfortable Conversation?§  Conversational cues and turn-taking are expect...
3. Engage in Comfortable Conversation§  Some may introduce bias  §  “Oh!”, “Interesting” – indicating unexpected answer ...
4. Stay Neutral and Accepting§          Ask Open-ended Questions      §      Start with Who, What, When, Where, Why, How...
4. Stay Neutral and Accepting§            Watch Your Reactions      §          Don’t show surprise            §     May...
4. Stay Neutral and Accepting§          Be yourself§          No one is perfectly neutral§          Recover gracefully ...
5. Encourage them to talk§  Be Quiet!  §  Most agreements happen immediately. Most people delay before      disagreeing....
Building Trust in an Interview1.  Be trustworthy upfront2.  Build rapport3.  Engage in comfortable conversation4.  Stay ne...
KEEPING	  YOUR	  SKILLS	  SHARP	                                           21
1. Identify your improvement areas§  Watch your videos§  Have colleagues give you feedback                              ...
2. Learn From Others§  Watch others moderate§  Be a participant§  Listen to talk radio interviews                      ...
3. Practice These Skills in Everyday Life§  Be quiet and listen in everyday conversations§  Ask open-ended questions mor...
The most important thing is…PRACTICE!                               25	  
ReferencesBoren, T. and Ramey, J. (2000) Thinking aloud: reconciling theory and practice. IEEE Transactions on Professiona...
Coming Soon!New book on Moderating•    Local Authors•    Coming this Fall•    Follow @ModSurvivalUX                       ...
Questions?             28	  
Thank You!	  	  	  	  Susan	  Mercer	  Senior	  Experience	  Researcher	  smercer@madpow.com	  @susanamercer	             ...
Upcoming SlideShare
Loading in...5
×

Strengthening Design Research Moderating Skills

556

Published on

Slides from a talk that I gave for the Boston CHI meeting on April 9, 2013.

Moderating usability studies and interviewing research participants is more than just reading questions off of a page - it's about having a conversation.

In this talk I discuss why it's important to have a conversation with participants and how to do so in a way that encourages them to talk honestly.

I also discuss how to maintain neutrality and minimize introducing bias into the conversation.

Published in: Design, Technology, Business
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
556
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
9
Comments
0
Likes
4
Embeds 0
No embeds

No notes for slide

Transcript of "Strengthening Design Research Moderating Skills"

  1. 1. Boston  CHI  Monthly  Mee>ng  STRENGTHENING  DESIGN  RESEARCH    MODERATION  SKILLS      APRIL  9,  2013        Susan  Mercer  Senior  Experience  Researcher  smercer@madpow.com  @susanamercer  
  2. 2. Strengthening Moderating and Interviewing Skills§  Once I learned the basics§  And ran a couple of studies,§  Now what?§  How do I strengthen my skills? 2  
  3. 3. Story TimeDon’t play the notes on the page……play the musicThis applies to moderating andinterviewing too.Don’t just read the questions from the page……have a conversation 3  
  4. 4. The Art of Moderating and Interviewing§  Not just reading questions§  It’s also… §  Building rapport with the participant §  Creating a smooth conversation §  Encouraging the participant to talk §  Remaining objective §  Listening effectively §  Being human 4  
  5. 5. What Is Our Goal?§  To elicit honest thoughts from participants in an unbiased manner.§  “Would you like to try one of my chocolate chip oyster and herring brownies? They’re really good!”§  Sometimes it’s hard to be fully honest 5  
  6. 6. The Rules of PolitenessLinguistic anthropologists have studied this phenomenonThe Rules of Politeness 1 “Human  beings  are   always  balancing  the  1.  Don’t Impose paradoxical  fact  that  2.  Give Options they  are  simultaneously  3.  Be Friendly individuals  and  social   creatures.”   -­‐  Deborah  Tannen   Sources: 1 Lakoff, 1976; 2 Tannen, 2005 6  
  7. 7. How Do We Get Beyond Politeness?§  In other words, we’re often polite rather than honest to strangers.§  As human beings we have a need to be socially accepted Social Acceptance > Honesty§  We guess what is expected and answer to avoid offending§  It’s second nature – we’re often not aware we’re doing it 7  
  8. 8. How Do We Get Beyond Politeness?Build their trust Politeness Trust Familiarity Strangers Good Friend 8  
  9. 9. Building Trust in an Interview1.  Be trustworthy upfront2.  Build rapport3.  Engage in comfortable conversation4.  Stay neutral and accepting 9  
  10. 10. 1. Be Trustworthy Upfront§  Confidentiality §  How are you recording what is said? §  Who is listening? §  What will you do with the information you collect?§  Neutrality §  You’re not emotionally involved in the design/project §  There are no right or wrong answers §  Your job is to get honest opinions§  Consent Form §  Put it all in writing – using their language
  11. 11. 2. Build Rapport A  persons  name  is  §  Greeting a Participant to  that  person  the   §  Smile sweetest  and  most   §  Use their name important  sound  in   any  language.   §  Handshake / Warm phone greeting -­‐  Dale  Carnegie   §  Make them feel important - sincerely§  Small Talk – Find Common Ground §  Safe topics: travel to office, traffic, weather §  Avoid asking direct questions §  Listen and look for shared experiences Source: Carnegie, 1936.
  12. 12. 2. Build Rapport§  Be Empathetic §  Apologize if they had trouble finding the office §  Show you understand their point of view §  “Oh, it’s raining there? It is here too. I hate rainy days.”§  Inject Some Humor §  Joke about yourself §  Joke about the situation §  Don’t joke about them§  Short version – Show them that you are human too
  13. 13. 3. Engage in Comfortable ConversationTwo schools of usability testing moderating techniques§  Moderator keeps silent and says “Keep talking” 1 §  Awkward feeling for participant §  Reminds them that they are in a “study”§  Moderator creates conversation where participant is primary speaker 2 §  Still accomplishes goals of gathering information §  Meets the participant’s expectations of a socially acceptable conversation §  Moderator can still be neutral and minimize bias Sources: 1 Ericsson and Simon, 1980, 2 Boren and Raney, 2000. 13  
  14. 14. 3. Engage in Comfortable ConversationWhat is a Comfortable Conversation?§  Conversational cues and turn-taking are expected§  Acknowledgement tokens – “Uh huh”, etc. §  Encourage the continuation of the other speaker’s talk §  Usually implies that the other speaker’s prior talk is incomplete§  Some indicate change of speakership §  “Yeah” – more than half the time indicates changing turns in speaking §  “Yeah. I heard that the other day and…” §  “Oh!” - may indicate noticing something, then transitioning to another topic §  “Oh! That reminded me…”
  15. 15. 3. Engage in Comfortable Conversation§  Some may introduce bias §  “Oh!”, “Interesting” – indicating unexpected answer §  “Yes”, “Perfect”, “Great” – indicating agreement §  “Hmmm.”, “Really?” – indicating disagreement §  Notice that tone is key§  Neutral is best §  “Mhmm”, “Uh huh”, “Continue”, “Tell me more”, “OK” §  “Mhmm” or “Uh huh” vs. silence à interviewees saying 31% more phrases. 1 Source: 1 Matarazzo et. al., 1964
  16. 16. 4. Stay Neutral and Accepting§  Ask Open-ended Questions §  Start with Who, What, When, Where, Why, How?§  Really Listen §  Pay attention – stay in the moment §  Look at the participant §  Take notes if you can §  Be quiet - give them time to say what they need to
  17. 17. 4. Stay Neutral and Accepting§  Watch Your Reactions §  Don’t show surprise §  May make them think that they are giving a wrong answer §  Don’t overly agree §  May make them think that they are giving the right answer §  Don’t be negative §  Watch your tone – stay neutral and accepting §  Try not to laugh
  18. 18. 4. Stay Neutral and Accepting§  Be yourself§  No one is perfectly neutral§  Recover gracefully and move on §  “Perfect” – “That’s the level of detailed feedback we’re looking for.” §  “Interesting!” – “I haven’t heard that perspective yet, tell me more.” §  (something surprising) – “I can understand that.”§  Interject some Rapport-building comments when needed §  Quiet or uncomfortable participants §  “I hate it when that happens.”, “I can imagine that was challenging”, etc. §  Again, showing that you are human like them
  19. 19. 5. Encourage them to talk§  Be Quiet! §  Most agreements happen immediately. Most people delay before disagreeing.1 §  If you don’t respond to their answer, it encourages them to talk more §  People often delay speaking before disagreeing – give them time §  Some people are uncomfortable with silence, so they will keep talking §  The best way to stay neutral J Source: 1 Goodwin and Heritage, 1990.
  20. 20. Building Trust in an Interview1.  Be trustworthy upfront2.  Build rapport3.  Engage in comfortable conversation4.  Stay neutral and accepting5.  Be Quiet 20  
  21. 21. KEEPING  YOUR  SKILLS  SHARP   21
  22. 22. 1. Identify your improvement areas§  Watch your videos§  Have colleagues give you feedback 22  
  23. 23. 2. Learn From Others§  Watch others moderate§  Be a participant§  Listen to talk radio interviews 23  
  24. 24. 3. Practice These Skills in Everyday Life§  Be quiet and listen in everyday conversations§  Ask open-ended questions more frequently 24  
  25. 25. The most important thing is…PRACTICE! 25  
  26. 26. ReferencesBoren, T. and Ramey, J. (2000) Thinking aloud: reconciling theory and practice. IEEE Transactions on Professional Communication, 43 (3), 261-278.Carnegie, D. (1936) How to win friends and influence people. Simon & Shuster.Dumas, J. and Loring, B. (2008) Moderating usability tests, Morgan Kaufman.Dumas, J. and Redish, J. (1999) A practical guide to usability testing, Intellect Ltd.Ericsson, K. and Simon, H. (1980) Verbal reports as data. Psychological review. 87 (3), 215-251.Goodwin, C., & Heritage, J. (1990). “Conversation analysis.” Annual review of anthropology 19 (1990): 283-307.Matarazzo, J.D.., Wiens, A. N., Saslow, G., Allen, B. V., & Weitman, M. (1064). Interviewer Mm-Hmm and interviewee speech durations. Psychotherapy: Theory, Research & Practice, 1(3), 109.Tannen, D. (2005). Conversational Style: Analyzing Talk among Friends. Oxford University Press, USA. 26  
  27. 27. Coming Soon!New book on Moderating•  Local Authors•  Coming this Fall•  Follow @ModSurvivalUX 27  
  28. 28. Questions? 28  
  29. 29. Thank You!        Susan  Mercer  Senior  Experience  Researcher  smercer@madpow.com  @susanamercer   29  
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×