How to get more than opinionsInterview techniques and adviceJohanna Kollmann- @johannakollLean UX Machine Tel Aviv, 4 August 2011Photo by NASA Marshall Space Flight Center http://www.flickr.com/photos/28634332@N05/5038531149/
UX helps you to get out of the buildingPhoto by Bottleleaf  http://www.flickr.com/photos/bottleleaf/2258627441/
(Some) research methods (yeah we have a lot)QuantitativeQualitativeContextual inquiryMental modelsInterviewsDiary studiesSurveysInterviewsGenerativeUsability testingModerated card sortWizard of OzAutomated card sortSurveysAutomated studiesAnalyticsA/B TestingMulti-variant testingEvaluativeAdapted from figures by Janice Fraser, Nate Bolt, Christian Rohrer
Non-leading interviews are a conversationPhoto byDave Gilbert http://www.flickr.com/photos/eye2eye/50892860/
Non-leading interviews Are generative
 Focus on people’s behaviours and goals
 Capture their way of thinking and vocabulary
Are about listening to stories
 Can be conducted in-person or remotely
 Should be done with people who haveimmediate experienceMental Models byIndi Young
Planning the interviewPhoto byangelamaphonehttp://www.flickr.com/photos/angelamaphone/2663422833//
Define your goals Who do you need to talk to?
 What do you need to observe?
 What do you hope to find out?
 What’s  your hypothesis?Also, make sure you identify your own bias and beliefs!
What topics shall the interview cover?DietingBuying foodExercisePreparing foodEating outBusy lifestyleStruggles
Prompts rather than set questionsDay-in-a-life (today, yesterday)Decide what to eatLast time on a dietHow active (want vs. do)Preparing food for oneselfPreparing food for family/friends
Have a ‘softball question’ readyPlease tell me a little bit about your cooking this week.Could you tell me about the last dish you prepared yourself?
During the interviewPhoto by Anders Zakrisson http://www.flickr.com/photos/anders-zakrisson/4982281184/
Photo byJitter Buffer http://www.flickr.com/photos/ph0t0s/5984587230/ We are so used to being interrupted that we have developed highly effective interruption defense mechanisms. 
Photo byEd Yourdon http://www.flickr.com/photos/yourdon/2574628438/Really listening lets you understand someone, or a situation, on several different levels.
Active listeningReceiveAppreciateSummariseAskFrom Julian Treasure’s TED talk ‘5 ways to listen better’
Ask open questions – don’t leadYAY Who
 What
 When
 Where
 Why
 HowNAY Did
 Have

How to get more than opinions: Interview techniques and advice

  • 1.
    How to getmore than opinionsInterview techniques and adviceJohanna Kollmann- @johannakollLean UX Machine Tel Aviv, 4 August 2011Photo by NASA Marshall Space Flight Center http://www.flickr.com/photos/28634332@N05/5038531149/
  • 2.
    UX helps youto get out of the buildingPhoto by Bottleleaf http://www.flickr.com/photos/bottleleaf/2258627441/
  • 3.
    (Some) research methods(yeah we have a lot)QuantitativeQualitativeContextual inquiryMental modelsInterviewsDiary studiesSurveysInterviewsGenerativeUsability testingModerated card sortWizard of OzAutomated card sortSurveysAutomated studiesAnalyticsA/B TestingMulti-variant testingEvaluativeAdapted from figures by Janice Fraser, Nate Bolt, Christian Rohrer
  • 4.
    Non-leading interviews area conversationPhoto byDave Gilbert http://www.flickr.com/photos/eye2eye/50892860/
  • 5.
  • 6.
    Focus onpeople’s behaviours and goals
  • 7.
    Capture theirway of thinking and vocabulary
  • 8.
  • 9.
    Can beconducted in-person or remotely
  • 10.
    Should bedone with people who haveimmediate experienceMental Models byIndi Young
  • 11.
    Planning the interviewPhotobyangelamaphonehttp://www.flickr.com/photos/angelamaphone/2663422833//
  • 12.
    Define your goalsWho do you need to talk to?
  • 13.
    What doyou need to observe?
  • 14.
    What doyou hope to find out?
  • 15.
    What’s your hypothesis?Also, make sure you identify your own bias and beliefs!
  • 16.
    What topics shallthe interview cover?DietingBuying foodExercisePreparing foodEating outBusy lifestyleStruggles
  • 17.
    Prompts rather thanset questionsDay-in-a-life (today, yesterday)Decide what to eatLast time on a dietHow active (want vs. do)Preparing food for oneselfPreparing food for family/friends
  • 18.
    Have a ‘softballquestion’ readyPlease tell me a little bit about your cooking this week.Could you tell me about the last dish you prepared yourself?
  • 19.
    During the interviewPhotoby Anders Zakrisson http://www.flickr.com/photos/anders-zakrisson/4982281184/
  • 20.
    Photo byJitter Bufferhttp://www.flickr.com/photos/ph0t0s/5984587230/ We are so used to being interrupted that we have developed highly effective interruption defense mechanisms. 
  • 21.
    Photo byEd Yourdonhttp://www.flickr.com/photos/yourdon/2574628438/Really listening lets you understand someone, or a situation, on several different levels.
  • 22.
    Active listeningReceiveAppreciateSummariseAskFrom JulianTreasure’s TED talk ‘5 ways to listen better’
  • 23.
    Ask open questions– don’t leadYAY Who
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
  • 30.
  • 31.
  • 32.
    WillWere youtrying to do A or B?What were you trying to do?
  • 33.
    How to keeppeople talkingCan you tell me the story about that?What else can you tell me about…Help me understand betterWhat do you mean by…Tell me more…
  • 34.
    Echoing and rephrasingThisis confusing...Confusing...Yes, confusing. I wasn't sure whether...Example from ‘Storytelling for User Experience’ by Whitney Quesenbery & Kevin Brooks
  • 35.
    Echoing and rephrasing…andso I decided to click on that link to go to the next page.Okay, so let me get this straight: first, you saw the link, and...what, again?I saw the link, and I thought to myself....(paraphrases self)Example from ‘Remote Research’ by Nate Bolt & Tony Tulathimutte
  • 36.
    Conversational disequilibriumI wantedto download that application, but the instructions were so confusing… (trails off and stops talking)The instructions were confusing?And you expected…Confusing?...Because….So then you…Mmmm hmmm.Example from ‘Storytelling for User Experience’ by Whitney Quesenbery & Kevin Brooks
  • 37.
    How to dealwith difficult peoplePhoto by David Anderson http://www.flickr.com/photos/venndiagram/4667350842/
  • 38.
    The quiet oneSo, tell me what you're trying to do here
  • 39.
    What areyou trying to get done right now?
  • 40.
    How doesthis (part/page) compare with what you were expecting?
  • 41.
    If theuser falls quiet repeatedly: And by the way, if you could just let me know what's going through your head as you’re doing this...Example from ‘Remote Research’ by Nate Bolt & Tony Tulathimutte
  • 42.
    The bored oneI noticed that you just hesitated a bit before clicking on that button. Can you tell me why?
  • 43.
    Why don'twe back up a bit? I was curious about what drew your attention to the tab you just clicked on?
  • 44.
    Before wemove on from here, I wanted to ask you about this part a bit more. What do you think about the range of choices they give you here? Is anything missing?Example from ‘Remote Research’ by Nate Bolt & Tony Tulathimutte
  • 45.
    The chatty oneThat's really interesting, thanks for telling me about that. To come back to....
  • 46.
    Can Iinterrupt you? Sorry, I was actually curious if you could...Example from ‘Remote Research’ by Nate Bolt & Tony Tulathimutte
  • 47.
    Do’s and don’tsPhotoby Hilde Skjølberg http://www.flickr.com/photos/hebe/3004800079/
  • 48.
    DoBe the learner,not the expertAsk naïve questionsAsk for specific storiesAllow people time to thinkListen!Take notes or recordTake photos or collect artefactsPhoto by Tomas Hellberg http://www.flickr.com/photos/tomhe/35312882/
  • 49.
    Don’tBe an interrogatorAskquestions that sound like blame, or argumentativeAsk for solutionsTry to solve problems during the interviewAsk what features people wantAsk people to imagine theoretical situationsPhoto by G Meyer http://www.flickr.com/photos/kainet/144703613/
  • 50.
    Have fun!Photo byEd Stevenson http://www.flickr.com/photos/estevenson/2641282945/
  • 51.
    ResourcesMental Models byIndi YoungStorytelling for User Experience by Whitney Quesenbery & Kevin BrooksRemote Research by Nate Bolt & Tony TulathimutteUndercover User Experience by Cennydd BowlesDesigning for the Digital Age by Kim GoodwinLUXrresources and materials by Janice Fraser (http://www.slideshare.net/clevergirl/) and Lane Halley (http://www.slideshare.net/LaneHalley/)User Interview Techniques - Guidelines for Obtaining Better Results by Michael Hawley in User Experience, Volume 8, Issue 3, 3rd Quarter 2009How to ask ‘why’ without asking ‘why by Karl Sabino (http://www.thinkflowinteractive.com/2009/09/01/how-to-ask-why/)Articles on User Interface Engineering (http://www.uie.com/browse/usability_testing/)
  • 52.
    Thank youJanice Fraser,Lane Halley & Josh Seiden from LUXr for sharing their materials under a CC licenseFlow Interactive alumni friends for sharing their adviceGraham Uff for feedbackAnd big thanks to everybody who made Lean UX Machine Tel Aviv happen!

Editor's Notes

  • #3 Getting out of the building  UX!UX research is useful for Customer discoveryCustomer developmentValidating your hypothesisMaking something that solves a problem or addresses a needMaking something usable and delightfulIt’s “generative” and “evaluative” It’s all about people skills!
  • #4 Explain quickly that UX offers tons of methods, techniques and materials. You will have to get out of the building as part of this event, so we’ll focus on these (click).
  • #5 Not an interrogation!
  • #7 !
  • #9 Non-leading interviews allow you to capture what a person is thinking in their terms, with their structure and vocabulary intact. Indi deliberately writes prompts rather than interview questions. Also easier to parse quickly. if you go for a non-directed interview using prompts, make sure everybody in your team has a shared understanding of the intent behind each topic. Janice calls this topic map.
  • #10 Non-leading interviews allow you to capture what a person is thinking in their terms, with their structure and vocabulary intact. Indi deliberately writes prompts rather than interview questions. Also easier to parse quickly. if you go for a non-directed interview using prompts, make sure everybody in your team has a shared understanding of the intent behind each topic. Janice calls this topic map.
  • #11 begin interviews with a 'softball' question - a question that is simple to answer and puts the participant at ease. 
  • #12 Make people feel comfortable and they will tell you all kind of thingsPeople generally like to talk about themselves, and being listened to!
  • #13 !
  • #14 !
  • #15 RASA (sanskrit word for juice, essence)Receive - pay attentionAppreciate - making little noisesSummarise - 'so’Ask - questions afterwardsAnother thing that makes it challenging: people want to keep the conversation balanced – you have to break that a bit
  • #16 Be careful with WHY. ‘How did you know that X?’ ‘What were you thinking at the moment when X?’ This does not interrupt the recounting process. So ‘tell me how it was that you came to be looking for this site that day’ does the work of ‘why were you looking... ?If you’ve made people comfortable, Why should be ok.
  • #19 on remote/using the phone. won't be able to rely on body language, which makes the moderator's responsibilities more complex. Active listening - practice of regularly nodding and saying mm-hmm to demonstrate you get what they're saying - may encourage people to keep going in person, but over the phone, moderator should cut back active listening because it encourages people to wrap up what they're saying. Don't be afraid to sit back and listen!Reflecting = paraphrasing or repeating things the user has just said. This can be risky, as 1) it almost always has same suppressive effect as active listening, and 2) if your paraphrase is inaccurate, it can lead users to agreeing to propositions or coming up with ideas that they may not have otherwise. Better alternative: begin as if you're going to paraphrase them but then have them do the bulk of the work by trailing off and letting them fill in their own thoughts, for (example on this slide)Achieves the same as reflecting, with less moderator bias.Over the phone, all you are is your voice. 
  • #20 let your statements trail off and end in an upward pitch, as if you were asking a question. the other person will naturally complete your statement. this is another way to 'take your turn' in the conversation and toss it right back at them.
  • #22 n quiet participants, who probably forget to think aloud as they are engaged in a task: You want to strike a balance between engagement and talking so that users are speaking undeliberatively about what they're feeling and doing, and what problems they're facing in the moment, rather than their opinions about the interface. Encourage participant to speak up.
  • #23 Come up with questions that are both specific and hard to give short responses to. Ask about specific on-screen behaviours.
  • #26 Manage expectations