It’s hard to understand people
How to get and make sense of qualitative insights


Johanna Kollmann - @johannakoll
Imperial College London, 20 November 2012




                                     Photo by NASA JSC Photo: http://www.flickr.com/photos/nasa_jsc_photo/7257865176
(Some) research methods (yeah we have a lot)
               Quantitative          Qualitative



             Surveys              Contextual inquiry
Generative   Interviews           Mental models
                                  Interviews
                                  Diary studies


             Automated card sort Usability testing
             Surveys               Moderated card sort
Evaluative   Automated studies Wizard of Oz
             Analytics
             A/B Testing
             Multi-variant testing


                                          Adapted from figures by Janice Fraser, Nate Bolt, Christian Rohrer
Before you leave the building
Plan who to talk to where about what and why


                     Photo by angelamaphone http://www.flickr.com/photos/angelamaphone/2663422833//
What topics shall the interview cover?

                                 Dieting
  Buying food

                                                      Exercise

                Preparing food



   Eating out
                                     Busy lifestyle


                    Struggles
Prompts rather than set questions

 Day-in-a-life (today, yesterday)
 Decide what to eat
 Last time on a diet
 How active (want vs. do)
 Preparing food for oneself
 Preparing food for family/friends
Have a ‘softball question’ ready


 Please tell me a little bit about your
         cooking this week.




                                          Could you tell me about the last
                                            dish you prepared yourself?
Photo by TheeErin: http://www.flickr.com/photos/theeerin/4729019845/
Out in the wild
Ask open questions – don’t lead

                YAY           NAY
                • Who         • Did
                • What        • Have
                • When        • Are
                • Where       • Were
                • Why         • Will
                • How

                                       Were you trying to do A or B?

What were you trying to do?
Some great all-purpose questions

• Has there ever been a time when you had x experience?
• Could you tell me about that?
• What was great about that?
• What was awful about that?
• Why did you do that?
• And then, what happened?
• If you had a magic wand, what would you make the situation be like?




                                                                        By Janice Fraser
Do’s and don’ts


                  Photo by Hilde Skjølberg http://www.flickr.com/photos/hebe/3004800079/
Do
Be the learner, not the expert
Ask naïve questions
Ask for specific stories
Allow people time to think
Listen!
Take notes or record
Take photos or collect artefacts




                                   Photo by Tomas Hellberg http://www.flickr.com/photos/tomhe/35312882/
Don’t
Be an interrogator
Ask questions that sound like blame, or argumentative
Ask for solutions
Try to solve problems during the interview
Ask what features people want
Ask people to imagine theoretical situations




                                               Photo by G Meyer http://www.flickr.com/photos/kainet/144703613/
Making sense of what you saw and heard
Photos taken at DesignJam London events by Rachel Winch and falkowata
Source: http://www.uie.com/articles/research_to_personas/
Business Model Canvas
Human activity systems
Soft Systems Methodology



                           Peter Checkland
Leverage points…

…places within a complex system where a small shift in
one thing can produce big changes in everything.
…are often counterintuitive.
Rich Picture




1. Construction of the Humber Bridge (adapted from Stewart and
Fortune, 1994) © The Open University




                                                                 2. Distance Learning Situation © Wood-Harper et al, Information
                                                                 Systems Definition: The Multiview Approach, Blackwell Scientific
                                                                 Publications 1985
Rich Picture elements


Stakeholders


Worldview


Connections


Conflicts




                        2. Distance Learning Situation © Wood-Harper et al, Information
                        Systems Definition: The Multiview Approach, Blackwell Scientific
                        Publications 1985
Resources
Notes from my Leancamp session on this topic http://johannakoll.posterous.com/ux-research-tips-
for-customer-development-not
Mental Models by Indi Young
Storytelling for User Experience by Whitney Quesenbery & Kevin Brooks
Remote Research by Nate Bolt & Tony Tulathimutte
Undercover User Experience by Cennydd Bowles
Designing for the Digital Age by Kim Goodwin
LUXr resources and materials by Janice Fraser (http://www.slideshare.net/clevergirl/) and Lane Halley
(http://www.slideshare.net/LaneHalley/)
Articles on User Interface Engineering (http://www.uie.com/browse/usability_testing/)
The intuitive mind is a sacred gift and the rational mind is a
faithful servant. We have created a society that honors the
servant and has forgotten the gift.

We will not solve the problems of the world from the same
level of thinking we were at when we created them. More
than anything else, this new century demands new thinking:

We must change our materially based analyses of the world
around us to include broader, more multidimensional
perspectives.

~Albert Einstein

ICStartup 3: customer development

  • 1.
    It’s hard tounderstand people How to get and make sense of qualitative insights Johanna Kollmann - @johannakoll Imperial College London, 20 November 2012 Photo by NASA JSC Photo: http://www.flickr.com/photos/nasa_jsc_photo/7257865176
  • 2.
    (Some) research methods(yeah we have a lot) Quantitative Qualitative Surveys Contextual inquiry Generative Interviews Mental models Interviews Diary studies Automated card sort Usability testing Surveys Moderated card sort Evaluative Automated studies Wizard of Oz Analytics A/B Testing Multi-variant testing Adapted from figures by Janice Fraser, Nate Bolt, Christian Rohrer
  • 3.
    Before you leavethe building
  • 4.
    Plan who totalk to where about what and why Photo by angelamaphone http://www.flickr.com/photos/angelamaphone/2663422833//
  • 7.
    What topics shallthe interview cover? Dieting Buying food Exercise Preparing food Eating out Busy lifestyle Struggles
  • 8.
    Prompts rather thanset questions Day-in-a-life (today, yesterday) Decide what to eat Last time on a diet How active (want vs. do) Preparing food for oneself Preparing food for family/friends
  • 9.
    Have a ‘softballquestion’ ready Please tell me a little bit about your cooking this week. Could you tell me about the last dish you prepared yourself?
  • 10.
    Photo by TheeErin:http://www.flickr.com/photos/theeerin/4729019845/
  • 11.
  • 12.
    Ask open questions– don’t lead YAY NAY • Who • Did • What • Have • When • Are • Where • Were • Why • Will • How Were you trying to do A or B? What were you trying to do?
  • 13.
    Some great all-purposequestions • Has there ever been a time when you had x experience? • Could you tell me about that? • What was great about that? • What was awful about that? • Why did you do that? • And then, what happened? • If you had a magic wand, what would you make the situation be like? By Janice Fraser
  • 14.
    Do’s and don’ts Photo by Hilde Skjølberg http://www.flickr.com/photos/hebe/3004800079/
  • 15.
    Do Be the learner,not the expert Ask naïve questions Ask for specific stories Allow people time to think Listen! Take notes or record Take photos or collect artefacts Photo by Tomas Hellberg http://www.flickr.com/photos/tomhe/35312882/
  • 16.
    Don’t Be an interrogator Askquestions that sound like blame, or argumentative Ask for solutions Try to solve problems during the interview Ask what features people want Ask people to imagine theoretical situations Photo by G Meyer http://www.flickr.com/photos/kainet/144703613/
  • 17.
    Making sense ofwhat you saw and heard
  • 19.
    Photos taken atDesignJam London events by Rachel Winch and falkowata
  • 20.
  • 21.
  • 23.
    Human activity systems SoftSystems Methodology Peter Checkland
  • 24.
    Leverage points… …places withina complex system where a small shift in one thing can produce big changes in everything. …are often counterintuitive.
  • 25.
    Rich Picture 1. Constructionof the Humber Bridge (adapted from Stewart and Fortune, 1994) © The Open University 2. Distance Learning Situation © Wood-Harper et al, Information Systems Definition: The Multiview Approach, Blackwell Scientific Publications 1985
  • 26.
    Rich Picture elements Stakeholders Worldview Connections Conflicts 2. Distance Learning Situation © Wood-Harper et al, Information Systems Definition: The Multiview Approach, Blackwell Scientific Publications 1985
  • 29.
    Resources Notes from myLeancamp session on this topic http://johannakoll.posterous.com/ux-research-tips- for-customer-development-not Mental Models by Indi Young Storytelling for User Experience by Whitney Quesenbery & Kevin Brooks Remote Research by Nate Bolt & Tony Tulathimutte Undercover User Experience by Cennydd Bowles Designing for the Digital Age by Kim Goodwin LUXr resources and materials by Janice Fraser (http://www.slideshare.net/clevergirl/) and Lane Halley (http://www.slideshare.net/LaneHalley/) Articles on User Interface Engineering (http://www.uie.com/browse/usability_testing/)
  • 30.
    The intuitive mindis a sacred gift and the rational mind is a faithful servant. We have created a society that honors the servant and has forgotten the gift. We will not solve the problems of the world from the same level of thinking we were at when we created them. More than anything else, this new century demands new thinking: We must change our materially based analyses of the world around us to include broader, more multidimensional perspectives. ~Albert Einstein

Editor's Notes

  • #8 Non-leading interviews allow you to capture what a person is thinking in their terms, with their structure and vocabulary intact. Indi deliberately writes prompts rather than interview questions. Also easier to parse quickly. if you go for a non-directed interview using prompts, make sure everybody in your team has a shared understanding of the intent behind each topic. Janice calls this topic map.
  • #9 Non-leading interviews allow you to capture what a person is thinking in their terms, with their structure and vocabulary intact. Indi deliberately writes prompts rather than interview questions. Also easier to parse quickly. if you go for a non-directed interview using prompts, make sure everybody in your team has a shared understanding of the intent behind each topic. Janice calls this topic map.
  • #10 begin interviews with a 'softball' question - a question that is simple to answer and puts the participant at ease. 
  • #13 Be careful with WHY. ‘How did you know that X?’ ‘What were you thinking at the moment when X?’ This does not interrupt the recounting process. So ‘tell me how it was that you came to be looking for this site that day’ does the work of ‘why were you looking... ?If you’ve made people comfortable, Why should be ok.
  • #16 Manage expectations
  • #19 what we saw/heard – what it means – why it matters
  • #22 BM channels = connections
  • #23 Map out connections – Rich Picture
  • #24 examples: eg hard system = thermostat, motherboard. soft system = game of poker, soccer game, meeting, healthcare.
  • #27 Worldview is a concept for empathyConsider:- roles that people adopt in the situation (which may be formally recognised or quite informal); the norms which govern people’s behaviour; and the values they espouse.- political aspects of the situation, in other words recognition of the different interests that are represented and how these different interests are accommodated.