Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Moderating to the Max: Refining Your Interviewing and Moderating Skills


Published on

Published in: Design, Technology, Education
  • Be the first to comment

Moderating to the Max: Refining Your Interviewing and Moderating Skills

  1. 1. Susan Mercer – Senior Experience Researcher@susanamercerMay 29, 2013UXPA Boston 2013 ConferenceModerating to the MaxREFINING YOUR MODERATING &INTERVIEWING SKILLS
  2. 2. Hello, I’m Susan Mercer BA and MSc in Geophysics 19 years in software and web UI and UX design Developer Designer Web Producer Product Manager Researcher MS Human Factors, Bentley University Twitter: @susanAmercer2
  3. 3. Interviewing and Moderating3Research & StrategyInteractionDesignVisual Design DevelopmentInterviewsFormativeUsabilitySummativeUsability
  4. 4. Moderating and Interviewing Skills On paper, they seem straightforward Write a screener Recruit Write a script Conduct the study Once I ran a couple easy ones, I thought I had it down.4InterviewsFormativeUsability
  5. 5. But Then… I got the participants who…And I learned an important lesson:Interviews and usability tests aremore than just a series of questions.5
  6. 6. My EpiphanyDon’t play the notes on the page……play the musicThis applies to moderating andinterviewing too.Don’t just read the questions from the page……have a conversation6
  7. 7. What Is Our Goal? To elicit honest thoughts from participants in an unbiased manner.Sometimes it’s hard to be fully honest.7
  8. 8. The Rules of PolitenessWe’re often polite rather than honest to strangersSocial Acceptance > Honesty8
  9. 9. How Do We Get Beyond Politeness?Build their trust9
  10. 10. 10LeadingQuestionsIntroduceBias
  11. 11. Participant ComfortTwo schools of usability testing moderating techniques Moderator keeps silent and says “Keep talking” 1 Awkward feeling for participant Reminds them that they are in a “study” Moderator creates conversation where participant is primaryspeaker 2 Still accomplishes goals of gathering information Meets the participant’s expectations of a socially acceptable conversation Moderator can still be neutral and minimize bias11Sources: 1 Ericsson and Simon, 1980, 2 Boren and Ramey, 2000.
  12. 12. Conversations• We carry on conversations all of the time• We convey a lot of subtle meaning in our conversations• We have rules for conversation• We know what is acceptable in which situations12
  13. 13. What Do We Do in Usability tests and interviews?We ask participants to: Come into a usability lab / meet us in a strange location Talk to a stranger about important topics Use a product or website and make mistakes While being observed Without knowing the right social norms on how to behave13
  14. 14. To Do List• Elicit honest feedback• In an unbiased manner• Instill trust in us• Make the participant comfortable• …and get them to answer all of your questions14
  15. 15. HOW TO DO IT – STEP BY STEP15
  16. 16. Structure of an Interview/Usability Session16TrustComfortInterviewBuild RapportNeutralObservationDefine SocialInteraction RulesBe TransparentComfortableConversationBe AcceptingThanksManage the FlowGreeting IntroductionInterview /EvaluationWrap-Up
  17. 17. 1. Greeting – Building Rapport Be a Friendly Person Smile Use their name Be a good listener Make the other person feel important – and do so sincerelyA persons name is to that person thesweetest and most important sound inany language.- Dale Carnegie
  18. 18. 1. Greeting – Building Rapport Small Talk – Find Common Ground Safe topics: travel to office, traffic, weather, local sports Avoid asking direct questions Listen and look for shared experiences
  19. 19. 1. Greeting – Building Rapport Be Empathetic Apologize if they had trouble finding the office “Oh, it’s raining there? It is here too. I hate rainy days.” Show you understand their point of view “I can understand that…” “I can see that…” “That does sound very frustrating…”
  20. 20. 1. Greeting – Building Rapport Inject Some Humor Joke about yourself Joke about the situation Don’t joke about them Rapport is about showing them that you are human too
  21. 21. 2. Introduction – Be Transparent Confidentiality How are you recording what is said? Who is listening? What will you do with the information you collect? Consent Form Put it all in writing – using their language
  22. 22. 2. Introduction – Define Social Interaction Rules Describe the Session List the activities Describe the roles Social Niceties Do Not Apply You’re not emotionally involved in the design/project There are no right or wrong answers Your job is to get honest opinions
  23. 23. 3. Interview/Evaluation – Be Accepting Watch Your Reactions Don’t show surprise May make them think that they are giving a wrong answer Don’t overly agree May make them think that they are giving the right answer Don’t be negative Watch your tone – stay neutral and accepting Try not to laugh
  24. 24. 3. Interview/Evaluation – Be Accepting Be yourself No one is perfectly neutral Recover gracefully and move on “Perfect” – “That’s the level of detailed feedback we’re looking for.” “Interesting!” – “I haven’t heard that perspective yet, tell me more.” (something surprising) – “I can understand that.” Interject some Rapport-building comments when needed Quiet or uncomfortable participants “I hate it when that happens.”, “I can imagine that was challenging”, etc. Again, showing that you are human like them
  25. 25. 3. Interview/Evaluation – Comfortable ConversationWhat is a Comfortable Conversation? Conversational cues and turn-taking are expected Acknowledgement tokens – “Uh huh”, etc. Encourage the continuation of the other speaker’s talk Usually implies that the other speaker’s prior talk is incompleteSource: 1 Drummond and Hopper, 1993.
  26. 26. 3. Interview/Evaluation – Comfortable Conversation Some acknowledgement tokens indicate other things “Yeah” – half the time indicates changing turns in speaking 1 “Yeah. I heard that the other day and…” “Oh!” - may indicate noticing something, then transitioning to anothertopic “Oh! That reminded me…” “Okay” – may indicate that the listener wants the speaker to stopspeaking1Source: 1 Drummond and Hopper, 1993.
  27. 27. 3. Interview/Evaluation – Neutral Observation Some may introduce bias “Oh!”, “Interesting” – indicating unexpected answer “Yes”, “Perfect”, “Great” – indicating agreement “Hmmm.”, “Really?” – indicating disagreement Notice that tone is key Neutral is best “Mhmm”, “Uh huh”, “Continue”, “Tell me more”, “OK” “Mhmm” or “Uh huh” vs. silence  interviewees saying 31% morephrases. 1• Body Language• Head nodding while participant is speaking  interviewees speak 50%longer. 2Source: [1] Matarazzo et. al., 1964, [2] Matarazzo et al., 1963
  28. 28. 3. Interview/Evaluation – Neutral Observation Ask Open-ended Questions Start with Who, What, When, Where, Why, How?
  29. 29. 3. Interview/Evaluation – Neutral Observation Really Listen Pay attention – stay in the moment Look at the participant Take notes if you can Be quiet - give them time to say what they need to
  30. 30. 3. Interview/Evaluation – Neutral Observation Be Quiet! Most agreements happen immediately. Most people delay beforedisagreeing.1 If you don’t respond to their answer, it encourages them to talk more People often delay speaking before disagreeing – give them time Some people are uncomfortable with silence, so they will keep talking “People speak in paragraphs.” (Steve Portugal) The best way to stay neutral Source: 1 Goodwin and Heritage, 1990.
  31. 31. 3. Interview/Evaluation – Manage the Flow• Wait for topics to come up organically• If the participant mentions something that you have later in your questionlist, ask it now.• Transitions• Don’t just switch topics – provide context.• Summarize last topic, then introduce new one.• “That’s great about XYZ, but now I’d like to talk about ABC”. Question.• “You mentioned XYZ, tell me a bit more about how you blah blah..”
  32. 32. Structure of an Interview/Usability Session32TrustComfortInterviewBuild RapportNeutralObservationDefine SocialInteraction RulesBe TransparentComfortableConversationBe AcceptingThanksManage the FlowGreeting IntroductionInterview /EvaluationWrap-Up
  33. 33. The most important thing is…33PRACTICE!
  34. 34. KEEPING SHARP34
  35. 35. Keep Improving1. Identify your improvement areas Watch your videos Have colleagues give you feedback35
  36. 36. Keep Improving2. Learn from others Watch others moderate Be a participant Listen to talk radio interviewsTerri Gross, Fresh Air, Ira Glass, This American Life36
  37. 37. Keep Improving3. Practice skills in everyday life People watch (observe) in public Be quiet and listen in everyday conversations37
  38. 38. MORE INFO38
  39. 39. Classic Books on Moderating39
  40. 40. Coming Soon!New book on Moderating• Donna Tedesco, Fiona Tranquada• Book coming this Fall• Follow @ModSurvivalUXSee them today:• “Expecting the Unexpected: Preparing for Successful User ResearchSessions”• 3:00 PM, Back Bay C40
  41. 41. Interviewing41
  42. 42. 42Questions?
  43. 43. ReferencesBoren, T. and Ramey, J. (2000) Thinking aloud: reconciling theory and practice. IEEE Transactions on ProfessionalCommunication, 43 (3), 261-278.Carnegie, D. (1936) How to win friends and influence people. Simon & Shuster.Dumas, J. and Loring, B. (2008) Moderating usability tests, Morgan Kaufman.Dumas, J. and Redish, J. (1999) A practical guide to usability testing, Intellect Ltd.Ericsson, K. and Simon, H. (1980) Verbal reports as data. Psychological review. 87 (3), 215-251.Goodwin, C., & Heritage, J. (1990). “Conversation analysis.” Annual review of anthropology 19 (1990): 283-307.Matarazzo, J.D.., Saslow, G., Wiens, A. N., Weitman, M. & Allen, B. V. (1963). Interviewer head nodding and intervieweespeech durations. Psychotherapy, 1, 54-63.Matarazzo, J.D.., Wiens, A. N., Saslow, G., Allen, B. V., & Weitman, M. (1964). Interviewer Mm-Hmm and intervieweespeech durations. Psychotherapy: Theory, Research & Practice, 1(3), 109.Tannen, D. (2005). Conversational Style: Analyzing Talk among Friends. Oxford University Press, USA.
  44. 44. 44Thank You!Susan MercerSenior Experience