Moderating to the Max: Refining Your Interviewing and Moderating Skills


Published on

Published in: Design, Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Here to talk about moderating and interviewingInterviews and Usability tests are often done in different parts of the project lifecycle, but they are quite similar in the skills requiredFormative done during the design process, to get qualitative feedback used to iterate on the designSummative done at the end of development. More structured, quantitative, to measure specific aspects of the softwareInterviews and Formative tests are similar in structure and skillsets needed to effectively moderate them. That’s what I’ll be focusing on today.
  • There are lots of soft skills involved.
  • “Would you like to try one of my chocolate chip oyster and herring brownies? They’re really good!”
  • Rules of Politeness govern social conversations – and are stronger with people we do not know wellDon’t Impose“Would you like something to drink?”“Yes, that would be nice.”VS“Yes, I’ll have an Orange Juice mixed with Raspberry Lime seltzer.”Give Options“Would you like something to drink?”“Water, tea, or whatever you have – don’t go to any trouble.”Be Friendly“What do you think of this website?”“Well, it’s not my favorite…”VS.“This is crap.”
  • A warm smile is the universal language of kindness – William Arthur Ward
  • Moderating to the Max: Refining Your Interviewing and Moderating Skills

    1. 1. Susan Mercer – Senior Experience Researcher@susanamercerMay 29, 2013UXPA Boston 2013 ConferenceModerating to the MaxREFINING YOUR MODERATING &INTERVIEWING SKILLS
    2. 2. Hello, I’m Susan Mercer BA and MSc in Geophysics 19 years in software and web UI and UX design Developer Designer Web Producer Product Manager Researcher MS Human Factors, Bentley University Twitter: @susanAmercer2
    3. 3. Interviewing and Moderating3Research & StrategyInteractionDesignVisual Design DevelopmentInterviewsFormativeUsabilitySummativeUsability
    4. 4. Moderating and Interviewing Skills On paper, they seem straightforward Write a screener Recruit Write a script Conduct the study Once I ran a couple easy ones, I thought I had it down.4InterviewsFormativeUsability
    5. 5. But Then… I got the participants who…And I learned an important lesson:Interviews and usability tests aremore than just a series of questions.5
    6. 6. My EpiphanyDon’t play the notes on the page……play the musicThis applies to moderating andinterviewing too.Don’t just read the questions from the page……have a conversation6
    7. 7. What Is Our Goal? To elicit honest thoughts from participants in an unbiased manner.Sometimes it’s hard to be fully honest.7
    8. 8. The Rules of PolitenessWe’re often polite rather than honest to strangersSocial Acceptance > Honesty8
    9. 9. How Do We Get Beyond Politeness?Build their trust9
    10. 10. 10LeadingQuestionsIntroduceBias
    11. 11. Participant ComfortTwo schools of usability testing moderating techniques Moderator keeps silent and says “Keep talking” 1 Awkward feeling for participant Reminds them that they are in a “study” Moderator creates conversation where participant is primaryspeaker 2 Still accomplishes goals of gathering information Meets the participant’s expectations of a socially acceptable conversation Moderator can still be neutral and minimize bias11Sources: 1 Ericsson and Simon, 1980, 2 Boren and Ramey, 2000.
    12. 12. Conversations• We carry on conversations all of the time• We convey a lot of subtle meaning in our conversations• We have rules for conversation• We know what is acceptable in which situations12
    13. 13. What Do We Do in Usability tests and interviews?We ask participants to: Come into a usability lab / meet us in a strange location Talk to a stranger about important topics Use a product or website and make mistakes While being observed Without knowing the right social norms on how to behave13
    14. 14. To Do List• Elicit honest feedback• In an unbiased manner• Instill trust in us• Make the participant comfortable• …and get them to answer all of your questions14
    15. 15. HOW TO DO IT – STEP BY STEP15
    16. 16. Structure of an Interview/Usability Session16TrustComfortInterviewBuild RapportNeutralObservationDefine SocialInteraction RulesBe TransparentComfortableConversationBe AcceptingThanksManage the FlowGreeting IntroductionInterview /EvaluationWrap-Up
    17. 17. 1. Greeting – Building Rapport Be a Friendly Person Smile Use their name Be a good listener Make the other person feel important – and do so sincerelyA persons name is to that person thesweetest and most important sound inany language.- Dale Carnegie
    18. 18. 1. Greeting – Building Rapport Small Talk – Find Common Ground Safe topics: travel to office, traffic, weather, local sports Avoid asking direct questions Listen and look for shared experiences
    19. 19. 1. Greeting – Building Rapport Be Empathetic Apologize if they had trouble finding the office “Oh, it’s raining there? It is here too. I hate rainy days.” Show you understand their point of view “I can understand that…” “I can see that…” “That does sound very frustrating…”
    20. 20. 1. Greeting – Building Rapport Inject Some Humor Joke about yourself Joke about the situation Don’t joke about them Rapport is about showing them that you are human too
    21. 21. 2. Introduction – Be Transparent Confidentiality How are you recording what is said? Who is listening? What will you do with the information you collect? Consent Form Put it all in writing – using their language
    22. 22. 2. Introduction – Define Social Interaction Rules Describe the Session List the activities Describe the roles Social Niceties Do Not Apply You’re not emotionally involved in the design/project There are no right or wrong answers Your job is to get honest opinions
    23. 23. 3. Interview/Evaluation – Be Accepting Watch Your Reactions Don’t show surprise May make them think that they are giving a wrong answer Don’t overly agree May make them think that they are giving the right answer Don’t be negative Watch your tone – stay neutral and accepting Try not to laugh
    24. 24. 3. Interview/Evaluation – Be Accepting Be yourself No one is perfectly neutral Recover gracefully and move on “Perfect” – “That’s the level of detailed feedback we’re looking for.” “Interesting!” – “I haven’t heard that perspective yet, tell me more.” (something surprising) – “I can understand that.” Interject some Rapport-building comments when needed Quiet or uncomfortable participants “I hate it when that happens.”, “I can imagine that was challenging”, etc. Again, showing that you are human like them
    25. 25. 3. Interview/Evaluation – Comfortable ConversationWhat is a Comfortable Conversation? Conversational cues and turn-taking are expected Acknowledgement tokens – “Uh huh”, etc. Encourage the continuation of the other speaker’s talk Usually implies that the other speaker’s prior talk is incompleteSource: 1 Drummond and Hopper, 1993.
    26. 26. 3. Interview/Evaluation – Comfortable Conversation Some acknowledgement tokens indicate other things “Yeah” – half the time indicates changing turns in speaking 1 “Yeah. I heard that the other day and…” “Oh!” - may indicate noticing something, then transitioning to anothertopic “Oh! That reminded me…” “Okay” – may indicate that the listener wants the speaker to stopspeaking1Source: 1 Drummond and Hopper, 1993.
    27. 27. 3. Interview/Evaluation – Neutral Observation Some may introduce bias “Oh!”, “Interesting” – indicating unexpected answer “Yes”, “Perfect”, “Great” – indicating agreement “Hmmm.”, “Really?” – indicating disagreement Notice that tone is key Neutral is best “Mhmm”, “Uh huh”, “Continue”, “Tell me more”, “OK” “Mhmm” or “Uh huh” vs. silence  interviewees saying 31% morephrases. 1• Body Language• Head nodding while participant is speaking  interviewees speak 50%longer. 2Source: [1] Matarazzo et. al., 1964, [2] Matarazzo et al., 1963
    28. 28. 3. Interview/Evaluation – Neutral Observation Ask Open-ended Questions Start with Who, What, When, Where, Why, How?
    29. 29. 3. Interview/Evaluation – Neutral Observation Really Listen Pay attention – stay in the moment Look at the participant Take notes if you can Be quiet - give them time to say what they need to
    30. 30. 3. Interview/Evaluation – Neutral Observation Be Quiet! Most agreements happen immediately. Most people delay beforedisagreeing.1 If you don’t respond to their answer, it encourages them to talk more People often delay speaking before disagreeing – give them time Some people are uncomfortable with silence, so they will keep talking “People speak in paragraphs.” (Steve Portugal) The best way to stay neutral Source: 1 Goodwin and Heritage, 1990.
    31. 31. 3. Interview/Evaluation – Manage the Flow• Wait for topics to come up organically• If the participant mentions something that you have later in your questionlist, ask it now.• Transitions• Don’t just switch topics – provide context.• Summarize last topic, then introduce new one.• “That’s great about XYZ, but now I’d like to talk about ABC”. Question.• “You mentioned XYZ, tell me a bit more about how you blah blah..”
    32. 32. Structure of an Interview/Usability Session32TrustComfortInterviewBuild RapportNeutralObservationDefine SocialInteraction RulesBe TransparentComfortableConversationBe AcceptingThanksManage the FlowGreeting IntroductionInterview /EvaluationWrap-Up
    33. 33. The most important thing is…33PRACTICE!
    34. 34. KEEPING SHARP34
    35. 35. Keep Improving1. Identify your improvement areas Watch your videos Have colleagues give you feedback35
    36. 36. Keep Improving2. Learn from others Watch others moderate Be a participant Listen to talk radio interviewsTerri Gross, Fresh Air, Ira Glass, This American Life36
    37. 37. Keep Improving3. Practice skills in everyday life People watch (observe) in public Be quiet and listen in everyday conversations37
    38. 38. MORE INFO38
    39. 39. Classic Books on Moderating39
    40. 40. Coming Soon!New book on Moderating• Donna Tedesco, Fiona Tranquada• Book coming this Fall• Follow @ModSurvivalUXSee them today:• “Expecting the Unexpected: Preparing for Successful User ResearchSessions”• 3:00 PM, Back Bay C40
    41. 41. Interviewing41
    42. 42. 42Questions?
    43. 43. ReferencesBoren, T. and Ramey, J. (2000) Thinking aloud: reconciling theory and practice. IEEE Transactions on ProfessionalCommunication, 43 (3), 261-278.Carnegie, D. (1936) How to win friends and influence people. Simon & Shuster.Dumas, J. and Loring, B. (2008) Moderating usability tests, Morgan Kaufman.Dumas, J. and Redish, J. (1999) A practical guide to usability testing, Intellect Ltd.Ericsson, K. and Simon, H. (1980) Verbal reports as data. Psychological review. 87 (3), 215-251.Goodwin, C., & Heritage, J. (1990). “Conversation analysis.” Annual review of anthropology 19 (1990): 283-307.Matarazzo, J.D.., Saslow, G., Wiens, A. N., Weitman, M. & Allen, B. V. (1963). Interviewer head nodding and intervieweespeech durations. Psychotherapy, 1, 54-63.Matarazzo, J.D.., Wiens, A. N., Saslow, G., Allen, B. V., & Weitman, M. (1964). Interviewer Mm-Hmm and intervieweespeech durations. Psychotherapy: Theory, Research & Practice, 1(3), 109.Tannen, D. (2005). Conversational Style: Analyzing Talk among Friends. Oxford University Press, USA.
    44. 44. 44Thank You!Susan MercerSenior Experience