SlideShare a Scribd company logo
1 of 29
Will Evans

Yana Kuchirko
"They've done studies, you
           know. 60% of the time, it
           works every time. ”
                            - Brian Fantana




06/16/12                              2
Surveys Aren’t Objective




They are created by people who are biased and analyze their
       results based on their own (mis)perceptions.

06/16/12                                                  3
Epistemological Dead End?




              No. Awareness that the researcher plays an integral
              role in the process of measuring any given
              phenomena by deciding how to measure is key.

              There are ways to minimize researcher bias by
              creating better questions.
06/16/12                                                      4
Methods are not “just methods”




How you measure what you are studying shapes what you find.

 06/16/12                                                5
Let’s say you are studying…




              How often teenagers use your website

06/16/12                                             6
You might ask teenagers…

“How often do you my website?”

c.Very rarely
d.Rarely
e.Occasionally
f.Frequently
g.Very frequently

 Response options source: http://www.dataguru.org
    06/16/12                                        7
You might ask teenagers…

“How often do you my website?”

c.Very rarely
d.Rarely                                            And what’s wrong
e.Occasionally                                      with these????
f.Frequently
g.Very frequently

 Response options source: http://www.dataguru.org
    06/16/12                                                           8
You might ask teenagers…

“How often do you my website?” !  !!
                           VI TY
                     EC TI
c.Very rarely
                  BJ
d.Rarely       SU           And what’s wrong
e.Occasionally                                      with these????
f.Frequently
g.Very frequently

 Response options source: http://www.dataguru.org
    06/16/12                                                         9
“Hmmm, for me “frequently”
                really means….”




     A few times a week      A few times a day

So what are we really measuring? NOISE.
 06/16/12                                        10
What is “noise”?




What we observe…

 06/16/12                      11
What is “noise”?

     We WANT this!




 We DON’T WANT this!


06/16/12                                12
An imaginary scenario
Let’s really stretch our thinking a bit here to
provide a more concrete understanding of
“noise” in your data.

Imagine you have AT&T phone service and
you’re trying to make a call. The signal isn’t
clearly going through and you end up hearing
everything but the other person’s voice.
Frustrating, right? Thank God this is only a
pretend scenario.
06/16/12                                          13
How is AT&T related to “Noise”?




When you develop bad questions, you don’t
“hear” the message of your data clearly.
  06/16/12                                  14
So…why is “noise” bad???
• Badly structured and poorly worded
  questions that obfuscate meaning for
  participants provide bad data.

• Bad data
      – Doesn’t answer your research question
      – Makes it difficult to interpret results
      – Is pretty much useless.

06/16/12                                          15
More bad examples
of frequently used response options
  a.   Completely satisfied       a.   Totally like
  b.   Very satisfied             b.   Very much like
  c.   Fairly well satisfied      c.   Moderately like
  d.   Somewhat dissatisfied      d.   Somewhat like
  e.   Very dissatisfied          e.   Not like




  What’s the difference between       What’s the difference between
  “fairly well” and “somewhat”?   “moderately like” and “somewhat like”?




06/16/12                                                            16
More bad examples
of frequently used response options
  a.   Completely satisfied            a.   Totally like
  b.   Very satisfied                  b.   Very much like
  c.   Fairly well satisfied           c.   Moderately like
  d.   Somewhat dissatisfied           d.   Somewhat like
  e.   Very dissatisfied               e.   Not like




  What’s the difference between            What’s the difference between
  “fairly well” and “somewhat”?        “moderately like” and “somewhat like”?




                                  Semantics
06/16/12                                                                 17
Words like….
 •   Sometimes
 •   Often
 •   Moderately
 •   Very
 •   Not very much


                                          Are subjective
            = they mean different things for different people

 06/16/12                                                 18
A better alternative?




Actually asking people how often they do certain
                   activities.
 06/16/12                                    19
In order to gather objective data…
…your questions must mean the same thing
for everyone.

How often do you use my website?
d.Never
e.A few times a year
f.Once a month
g.2-3 times a month
h.Once a week
i.A few times a week
j.Every day
                       “Never” means never for everyone!
06/16/12                                                   20
Importance of “Anchors”




Responses options serve as “anchors” for each question,
determining the “location” of each responses as qualitatively
distinct from the other.

   06/16/12                                                 21
Importance of “Anchors”
Responses options serve as “anchors” for each
question, determining the “location” of each
responses as qualitatively distinct from the other.




  Strongly                                        Strongly
  disagree    Disagree    Neutral      Agree       Agree




06/16/12                                                     22
(Ideal) Response Symmetry
• Good questions aim to have symmetrical
  quantitative/qualitative distance between
  anchors
    a.     Never
    b.     A few times a year
                                The distance between “never” and
    c.     Once a month
                                “a few times a year” is proportionate to
    d.     2-3 times a month    “every day” and “a few times a week”.
    e.     Once a week
    f.     A few times a week
    g.     Every day


06/16/12                                                                   23
Adherence to Logic & Linearity




People are accustomed to paradigms that are
        intuitive, and often “linear”.

           Anchors should not be the exception.
06/16/12                                          24
Intuitive Anchor Directions

 Never                             Always

Disagree                           Agree


Very poor                          Very good

Not very                          Very
important                         Important
 06/16/12                                   25
Value of Clarity




If questions are clear and concise, participants would
spend less time analyzing the questions themselves and
more time on answering them.                    Mystic Arts, LLC



   06/16/12                                                 26
So do good questions guarantee valid
             results?
• No. But good questions offer more
  assurance that you are listening to signal
  and not the noise.

• But answering your research question can be
  done in other ways
           • Behavioral Observations
           • Open ended qualitative questions
           • Many more…

06/16/12                                        27
The End.

  So was this presentation simply awesome?




                       a. Hell yeah!
                       b. Definitely!
                       c. Totally rocked!

06/16/12                                     28
Thanks!

More Related Content

What's hot

Just Enough Research
Just Enough ResearchJust Enough Research
Just Enough ResearchErika Hall
 
User Experience Audit by Gridle
User Experience Audit by GridleUser Experience Audit by Gridle
User Experience Audit by GridleClientjoy.io
 
An Introduction to Usability Testing
An Introduction to Usability TestingAn Introduction to Usability Testing
An Introduction to Usability TestingLennart Overkamp
 
Teresa Torres - An introduction to modern product discovery - Productized16
Teresa Torres - An introduction to modern product discovery - Productized16Teresa Torres - An introduction to modern product discovery - Productized16
Teresa Torres - An introduction to modern product discovery - Productized16Productized
 
Design Thinking 101 - An Introduction to Design Thinking for Developers
Design Thinking 101 - An Introduction to Design Thinking for DevelopersDesign Thinking 101 - An Introduction to Design Thinking for Developers
Design Thinking 101 - An Introduction to Design Thinking for DevelopersBill Bulman
 
Persona & User Journey in Practice
Persona & User Journey in PracticePersona & User Journey in Practice
Persona & User Journey in PracticeAfif Bimantara
 
How to Break the Requirements into User Stories
How to Break the Requirements into User StoriesHow to Break the Requirements into User Stories
How to Break the Requirements into User StoriesShriKant Vashishtha
 
UX Experience Design: Processes and Strategy
UX Experience Design: Processes and StrategyUX Experience Design: Processes and Strategy
UX Experience Design: Processes and StrategyCHI UX Indonesia
 
Ui ux designing principles
Ui ux designing principlesUi ux designing principles
Ui ux designing principlesDzung Nguyen
 
Intro to UX: Enterprise UX
Intro to UX: Enterprise UXIntro to UX: Enterprise UX
Intro to UX: Enterprise UXBart Van Hecke
 
Microcopy & UX writing for product people
Microcopy & UX writing for product peopleMicrocopy & UX writing for product people
Microcopy & UX writing for product peopleYuval Keshtcher ✍
 
Integrating User Centered Design with Agile Development
Integrating User Centered Design with Agile DevelopmentIntegrating User Centered Design with Agile Development
Integrating User Centered Design with Agile DevelopmentJulia Borkenhagen
 
Usability and User Experience
Usability and User ExperienceUsability and User Experience
Usability and User ExperienceShawn Calvert
 
Prioritizing Your Product Backlog
Prioritizing Your Product BacklogPrioritizing Your Product Backlog
Prioritizing Your Product BacklogMike Cohn
 

What's hot (20)

Just Enough Research
Just Enough ResearchJust Enough Research
Just Enough Research
 
Basics of UX Research
Basics of UX ResearchBasics of UX Research
Basics of UX Research
 
User Experience Audit by Gridle
User Experience Audit by GridleUser Experience Audit by Gridle
User Experience Audit by Gridle
 
An Introduction to Usability Testing
An Introduction to Usability TestingAn Introduction to Usability Testing
An Introduction to Usability Testing
 
UX Fundamentals for Beginners
UX Fundamentals for BeginnersUX Fundamentals for Beginners
UX Fundamentals for Beginners
 
Teresa Torres - An introduction to modern product discovery - Productized16
Teresa Torres - An introduction to modern product discovery - Productized16Teresa Torres - An introduction to modern product discovery - Productized16
Teresa Torres - An introduction to modern product discovery - Productized16
 
Design Thinking 101 - An Introduction to Design Thinking for Developers
Design Thinking 101 - An Introduction to Design Thinking for DevelopersDesign Thinking 101 - An Introduction to Design Thinking for Developers
Design Thinking 101 - An Introduction to Design Thinking for Developers
 
Persona & User Journey in Practice
Persona & User Journey in PracticePersona & User Journey in Practice
Persona & User Journey in Practice
 
How to Break the Requirements into User Stories
How to Break the Requirements into User StoriesHow to Break the Requirements into User Stories
How to Break the Requirements into User Stories
 
UX Experience Design: Processes and Strategy
UX Experience Design: Processes and StrategyUX Experience Design: Processes and Strategy
UX Experience Design: Processes and Strategy
 
Ui ux designing principles
Ui ux designing principlesUi ux designing principles
Ui ux designing principles
 
Intro to UX: Enterprise UX
Intro to UX: Enterprise UXIntro to UX: Enterprise UX
Intro to UX: Enterprise UX
 
Ux design process
Ux design processUx design process
Ux design process
 
Interaction design
Interaction designInteraction design
Interaction design
 
Microcopy & UX writing for product people
Microcopy & UX writing for product peopleMicrocopy & UX writing for product people
Microcopy & UX writing for product people
 
UX Usability Heuristics
UX Usability HeuristicsUX Usability Heuristics
UX Usability Heuristics
 
Integrating User Centered Design with Agile Development
Integrating User Centered Design with Agile DevelopmentIntegrating User Centered Design with Agile Development
Integrating User Centered Design with Agile Development
 
What is UX?
What is UX?What is UX?
What is UX?
 
Usability and User Experience
Usability and User ExperienceUsability and User Experience
Usability and User Experience
 
Prioritizing Your Product Backlog
Prioritizing Your Product BacklogPrioritizing Your Product Backlog
Prioritizing Your Product Backlog
 

Viewers also liked

Design Synthesis
Design SynthesisDesign Synthesis
Design Synthesisfrog
 
Writing surveysthatwork
Writing surveysthatworkWriting surveysthatwork
Writing surveysthatworkrebeccaweiss
 
Design Strategy: Aligning Business Goals and User Needs
Design Strategy: Aligning Business Goals and User NeedsDesign Strategy: Aligning Business Goals and User Needs
Design Strategy: Aligning Business Goals and User NeedsChris Avore
 
UX Flows - Skillshare
UX Flows - SkillshareUX Flows - Skillshare
UX Flows - SkillshareCarl Nelson
 
Information Architecture: Making Information Accessible and Useful
Information Architecture: Making Information Accessible and UsefulInformation Architecture: Making Information Accessible and Useful
Information Architecture: Making Information Accessible and Usefulfrog
 
(Design) Principles To Build By
(Design) Principles To Build By(Design) Principles To Build By
(Design) Principles To Build ByStephen Anderson
 
Competitive Analysis: Strategic UX Spy Games - Lyle Kantrovich
Competitive Analysis: Strategic UX Spy Games - Lyle KantrovichCompetitive Analysis: Strategic UX Spy Games - Lyle Kantrovich
Competitive Analysis: Strategic UX Spy Games - Lyle KantrovichUXPA International
 
Design Thinking is Killing Creativity
Design Thinking is Killing CreativityDesign Thinking is Killing Creativity
Design Thinking is Killing Creativitydesignsojourn
 
10 tips for a better UX survey
10 tips for a better UX survey10 tips for a better UX survey
10 tips for a better UX surveyCaroline Jarrett
 
Better Ideas Faster: How to Brainstorm More Effectively
Better Ideas Faster: How to Brainstorm More EffectivelyBetter Ideas Faster: How to Brainstorm More Effectively
Better Ideas Faster: How to Brainstorm More EffectivelyDavid Sherwin
 
Designing Structure Part II: Information Archtecture
Designing Structure Part II: Information ArchtectureDesigning Structure Part II: Information Archtecture
Designing Structure Part II: Information ArchtectureChristina Wodtke
 
10 Usability Heuristics explained
10 Usability Heuristics explained10 Usability Heuristics explained
10 Usability Heuristics explainedCraft Design
 
Content Strategy for Everything
Content Strategy for EverythingContent Strategy for Everything
Content Strategy for EverythingKristina Halvorson
 

Viewers also liked (14)

Design Synthesis
Design SynthesisDesign Synthesis
Design Synthesis
 
Writing surveysthatwork
Writing surveysthatworkWriting surveysthatwork
Writing surveysthatwork
 
Design Strategy: Aligning Business Goals and User Needs
Design Strategy: Aligning Business Goals and User NeedsDesign Strategy: Aligning Business Goals and User Needs
Design Strategy: Aligning Business Goals and User Needs
 
UX Flows - Skillshare
UX Flows - SkillshareUX Flows - Skillshare
UX Flows - Skillshare
 
Information Architecture: Making Information Accessible and Useful
Information Architecture: Making Information Accessible and UsefulInformation Architecture: Making Information Accessible and Useful
Information Architecture: Making Information Accessible and Useful
 
(Design) Principles To Build By
(Design) Principles To Build By(Design) Principles To Build By
(Design) Principles To Build By
 
Competitive Analysis: Strategic UX Spy Games - Lyle Kantrovich
Competitive Analysis: Strategic UX Spy Games - Lyle KantrovichCompetitive Analysis: Strategic UX Spy Games - Lyle Kantrovich
Competitive Analysis: Strategic UX Spy Games - Lyle Kantrovich
 
Task Analysis
Task AnalysisTask Analysis
Task Analysis
 
Design Thinking is Killing Creativity
Design Thinking is Killing CreativityDesign Thinking is Killing Creativity
Design Thinking is Killing Creativity
 
10 tips for a better UX survey
10 tips for a better UX survey10 tips for a better UX survey
10 tips for a better UX survey
 
Better Ideas Faster: How to Brainstorm More Effectively
Better Ideas Faster: How to Brainstorm More EffectivelyBetter Ideas Faster: How to Brainstorm More Effectively
Better Ideas Faster: How to Brainstorm More Effectively
 
Designing Structure Part II: Information Archtecture
Designing Structure Part II: Information ArchtectureDesigning Structure Part II: Information Archtecture
Designing Structure Part II: Information Archtecture
 
10 Usability Heuristics explained
10 Usability Heuristics explained10 Usability Heuristics explained
10 Usability Heuristics explained
 
Content Strategy for Everything
Content Strategy for EverythingContent Strategy for Everything
Content Strategy for Everything
 

Similar to Introduction to UX Research: Designing Surveys That Don't Suck!

How to create surveys that don't suck
How to create surveys that don't suckHow to create surveys that don't suck
How to create surveys that don't suckYana Alekseyevna
 
Giochi con Illusioni
Giochi con IllusioniGiochi con Illusioni
Giochi con IllusioniEva Zenith
 
Romeo And Juliet
Romeo And JulietRomeo And Juliet
Romeo And Julietcwozniak
 
[Share] communication style_en
[Share]  communication style_en [Share]  communication style_en
[Share] communication style_en Hisao Tsujimura
 
Yar! Thar be Sharks! Civility and Anonymity Online
Yar! Thar be Sharks! Civility and Anonymity OnlineYar! Thar be Sharks! Civility and Anonymity Online
Yar! Thar be Sharks! Civility and Anonymity OnlineJulie Strange
 
drafting bold dreams
drafting bold dreamsdrafting bold dreams
drafting bold dreamsmonika hardy
 
Interviewing Users
Interviewing UsersInterviewing Users
Interviewing UsersDevesh Desai
 
NCompass Live: Pioneers in Violence Prevention: Libraries & Rape Crisis Centers
NCompass Live: Pioneers in Violence Prevention: Libraries & Rape Crisis CentersNCompass Live: Pioneers in Violence Prevention: Libraries & Rape Crisis Centers
NCompass Live: Pioneers in Violence Prevention: Libraries & Rape Crisis CentersNebraska Library Commission
 
Is In Depth Journalism Dead
Is In Depth Journalism DeadIs In Depth Journalism Dead
Is In Depth Journalism DeadKarl Grubaugh
 
Qualitative methods in audience analysis
Qualitative methods in audience analysisQualitative methods in audience analysis
Qualitative methods in audience analysisJenna Condie
 
Simplify your future
Simplify your futureSimplify your future
Simplify your futureJamal Nazeer
 
The credibility problem
The credibility problemThe credibility problem
The credibility problemJason Oke
 
Is qualitative research holding us back?
Is qualitative research holding us back?Is qualitative research holding us back?
Is qualitative research holding us back?Jason Oke
 
Batteries Not Included: How to Supercharge Student Motivation
Batteries Not Included: How to Supercharge Student MotivationBatteries Not Included: How to Supercharge Student Motivation
Batteries Not Included: How to Supercharge Student MotivationECTutoring
 
12 PEER RESPONSES DUE IN 16 HOURS - EACH SET OF 2 HAS ITS OWN IN
12 PEER RESPONSES DUE IN 16 HOURS - EACH SET OF 2 HAS ITS OWN IN12 PEER RESPONSES DUE IN 16 HOURS - EACH SET OF 2 HAS ITS OWN IN
12 PEER RESPONSES DUE IN 16 HOURS - EACH SET OF 2 HAS ITS OWN INcargillfilberto
 
parenting_with_style_rev.ppt
parenting_with_style_rev.pptparenting_with_style_rev.ppt
parenting_with_style_rev.pptWmuia
 

Similar to Introduction to UX Research: Designing Surveys That Don't Suck! (20)

How to create surveys that don't suck
How to create surveys that don't suckHow to create surveys that don't suck
How to create surveys that don't suck
 
Giochi con Illusioni
Giochi con IllusioniGiochi con Illusioni
Giochi con Illusioni
 
Romeo And Juliet
Romeo And JulietRomeo And Juliet
Romeo And Juliet
 
Thanking
ThankingThanking
Thanking
 
[Share] communication style_en
[Share]  communication style_en [Share]  communication style_en
[Share] communication style_en
 
Yar! Thar be Sharks! Civility and Anonymity Online
Yar! Thar be Sharks! Civility and Anonymity OnlineYar! Thar be Sharks! Civility and Anonymity Online
Yar! Thar be Sharks! Civility and Anonymity Online
 
youth's wings
youth's wings youth's wings
youth's wings
 
drafting bold dreams
drafting bold dreamsdrafting bold dreams
drafting bold dreams
 
Interviewing Users
Interviewing UsersInterviewing Users
Interviewing Users
 
NCompass Live: Pioneers in Violence Prevention: Libraries & Rape Crisis Centers
NCompass Live: Pioneers in Violence Prevention: Libraries & Rape Crisis CentersNCompass Live: Pioneers in Violence Prevention: Libraries & Rape Crisis Centers
NCompass Live: Pioneers in Violence Prevention: Libraries & Rape Crisis Centers
 
Is In Depth Journalism Dead
Is In Depth Journalism DeadIs In Depth Journalism Dead
Is In Depth Journalism Dead
 
Qualitative methods in audience analysis
Qualitative methods in audience analysisQualitative methods in audience analysis
Qualitative methods in audience analysis
 
Simplify your future
Simplify your futureSimplify your future
Simplify your future
 
The credibility problem
The credibility problemThe credibility problem
The credibility problem
 
Is qualitative research holding us back?
Is qualitative research holding us back?Is qualitative research holding us back?
Is qualitative research holding us back?
 
Batteries Not Included: How to Supercharge Student Motivation
Batteries Not Included: How to Supercharge Student MotivationBatteries Not Included: How to Supercharge Student Motivation
Batteries Not Included: How to Supercharge Student Motivation
 
12 PEER RESPONSES DUE IN 16 HOURS - EACH SET OF 2 HAS ITS OWN IN
12 PEER RESPONSES DUE IN 16 HOURS - EACH SET OF 2 HAS ITS OWN IN12 PEER RESPONSES DUE IN 16 HOURS - EACH SET OF 2 HAS ITS OWN IN
12 PEER RESPONSES DUE IN 16 HOURS - EACH SET OF 2 HAS ITS OWN IN
 
parenting_with_style_rev.ppt
parenting_with_style_rev.pptparenting_with_style_rev.ppt
parenting_with_style_rev.ppt
 
jessica
jessicajessica
jessica
 
Life law #6
Life law #6Life law #6
Life law #6
 

More from William Evans

Empowering Agile Self-Organized Teams With Design Thinking
Empowering Agile Self-Organized Teams With Design ThinkingEmpowering Agile Self-Organized Teams With Design Thinking
Empowering Agile Self-Organized Teams With Design ThinkingWilliam Evans
 
Facilitating Complexity: Methods & Mindsets for Exploration
Facilitating Complexity: Methods & Mindsets for Exploration Facilitating Complexity: Methods & Mindsets for Exploration
Facilitating Complexity: Methods & Mindsets for Exploration William Evans
 
Leading Organizational Design and Transformation
Leading Organizational Design and TransformationLeading Organizational Design and Transformation
Leading Organizational Design and TransformationWilliam Evans
 
On Context: Methods and Mindsets for Situational Awareness
On Context: Methods and Mindsets for Situational AwarenessOn Context: Methods and Mindsets for Situational Awareness
On Context: Methods and Mindsets for Situational AwarenessWilliam Evans
 
On Branding: Turning Emotion into Connection
On Branding: Turning Emotion into ConnectionOn Branding: Turning Emotion into Connection
On Branding: Turning Emotion into ConnectionWilliam Evans
 
Dispositioning Advantage: A Pervert's Guide to Strategy Design
Dispositioning Advantage: A Pervert's Guide to Strategy DesignDispositioning Advantage: A Pervert's Guide to Strategy Design
Dispositioning Advantage: A Pervert's Guide to Strategy DesignWilliam Evans
 
New Models of Purpose-Driven Exploration in Knowledge Work
New Models of Purpose-Driven Exploration in Knowledge WorkNew Models of Purpose-Driven Exploration in Knowledge Work
New Models of Purpose-Driven Exploration in Knowledge WorkWilliam Evans
 
Facilitating Complexity: A Pervert's Guide to Exploration
Facilitating Complexity: A Pervert's Guide to ExplorationFacilitating Complexity: A Pervert's Guide to Exploration
Facilitating Complexity: A Pervert's Guide to ExplorationWilliam Evans
 
LeanUX: Problem Framing Using the 4 Ws
LeanUX: Problem Framing Using the 4 WsLeanUX: Problem Framing Using the 4 Ws
LeanUX: Problem Framing Using the 4 WsWilliam Evans
 
Good Design is Honest: Cognitive Science to UX Design Principles
Good Design is Honest: Cognitive Science to UX Design PrinciplesGood Design is Honest: Cognitive Science to UX Design Principles
Good Design is Honest: Cognitive Science to UX Design PrinciplesWilliam Evans
 
ReDesigned to Disrupt
ReDesigned to DisruptReDesigned to Disrupt
ReDesigned to DisruptWilliam Evans
 
Introduction to Lean Startup & Lean User Experience Design
Introduction to Lean Startup & Lean User Experience Design Introduction to Lean Startup & Lean User Experience Design
Introduction to Lean Startup & Lean User Experience Design William Evans
 
LeanUX is a Useful F*&king Lie
LeanUX is a Useful F*&king LieLeanUX is a Useful F*&king Lie
LeanUX is a Useful F*&king LieWilliam Evans
 
Introduction to Kanban for Creative Agencies
Introduction to Kanban for Creative AgenciesIntroduction to Kanban for Creative Agencies
Introduction to Kanban for Creative AgenciesWilliam Evans
 
Design Ethnography for Lean Teams
Design Ethnography for Lean TeamsDesign Ethnography for Lean Teams
Design Ethnography for Lean TeamsWilliam Evans
 
Modeling Leadership for #LeanUXNYC
Modeling Leadership for #LeanUXNYCModeling Leadership for #LeanUXNYC
Modeling Leadership for #LeanUXNYCWilliam Evans
 
Introduction to UX Research: Conducting Focus Groups
Introduction to UX Research: Conducting Focus GroupsIntroduction to UX Research: Conducting Focus Groups
Introduction to UX Research: Conducting Focus GroupsWilliam Evans
 
Customer Research & Persona Development
Customer Research & Persona DevelopmentCustomer Research & Persona Development
Customer Research & Persona DevelopmentWilliam Evans
 

More from William Evans (20)

Empowering Agile Self-Organized Teams With Design Thinking
Empowering Agile Self-Organized Teams With Design ThinkingEmpowering Agile Self-Organized Teams With Design Thinking
Empowering Agile Self-Organized Teams With Design Thinking
 
Facilitating Complexity: Methods & Mindsets for Exploration
Facilitating Complexity: Methods & Mindsets for Exploration Facilitating Complexity: Methods & Mindsets for Exploration
Facilitating Complexity: Methods & Mindsets for Exploration
 
Leading Organizational Design and Transformation
Leading Organizational Design and TransformationLeading Organizational Design and Transformation
Leading Organizational Design and Transformation
 
On Context: Methods and Mindsets for Situational Awareness
On Context: Methods and Mindsets for Situational AwarenessOn Context: Methods and Mindsets for Situational Awareness
On Context: Methods and Mindsets for Situational Awareness
 
On Branding: Turning Emotion into Connection
On Branding: Turning Emotion into ConnectionOn Branding: Turning Emotion into Connection
On Branding: Turning Emotion into Connection
 
Dispositioning Advantage: A Pervert's Guide to Strategy Design
Dispositioning Advantage: A Pervert's Guide to Strategy DesignDispositioning Advantage: A Pervert's Guide to Strategy Design
Dispositioning Advantage: A Pervert's Guide to Strategy Design
 
New Models of Purpose-Driven Exploration in Knowledge Work
New Models of Purpose-Driven Exploration in Knowledge WorkNew Models of Purpose-Driven Exploration in Knowledge Work
New Models of Purpose-Driven Exploration in Knowledge Work
 
Facilitating Complexity: A Pervert's Guide to Exploration
Facilitating Complexity: A Pervert's Guide to ExplorationFacilitating Complexity: A Pervert's Guide to Exploration
Facilitating Complexity: A Pervert's Guide to Exploration
 
LeanUX: Problem Framing Using the 4 Ws
LeanUX: Problem Framing Using the 4 WsLeanUX: Problem Framing Using the 4 Ws
LeanUX: Problem Framing Using the 4 Ws
 
Good Design is Honest: Cognitive Science to UX Design Principles
Good Design is Honest: Cognitive Science to UX Design PrinciplesGood Design is Honest: Cognitive Science to UX Design Principles
Good Design is Honest: Cognitive Science to UX Design Principles
 
LeanUX14
LeanUX14LeanUX14
LeanUX14
 
ReDesigned to Disrupt
ReDesigned to DisruptReDesigned to Disrupt
ReDesigned to Disrupt
 
Introduction to Lean Startup & Lean User Experience Design
Introduction to Lean Startup & Lean User Experience Design Introduction to Lean Startup & Lean User Experience Design
Introduction to Lean Startup & Lean User Experience Design
 
LeanUX is a Useful F*&king Lie
LeanUX is a Useful F*&king LieLeanUX is a Useful F*&king Lie
LeanUX is a Useful F*&king Lie
 
Introduction to Kanban for Creative Agencies
Introduction to Kanban for Creative AgenciesIntroduction to Kanban for Creative Agencies
Introduction to Kanban for Creative Agencies
 
Design Ethnography for Lean Teams
Design Ethnography for Lean TeamsDesign Ethnography for Lean Teams
Design Ethnography for Lean Teams
 
Promiscuous pairing
Promiscuous pairingPromiscuous pairing
Promiscuous pairing
 
Modeling Leadership for #LeanUXNYC
Modeling Leadership for #LeanUXNYCModeling Leadership for #LeanUXNYC
Modeling Leadership for #LeanUXNYC
 
Introduction to UX Research: Conducting Focus Groups
Introduction to UX Research: Conducting Focus GroupsIntroduction to UX Research: Conducting Focus Groups
Introduction to UX Research: Conducting Focus Groups
 
Customer Research & Persona Development
Customer Research & Persona DevelopmentCustomer Research & Persona Development
Customer Research & Persona Development
 

Recently uploaded

专业一比一美国亚利桑那大学毕业证成绩单pdf电子版制作修改#真实工艺展示#真实防伪#diploma#degree
专业一比一美国亚利桑那大学毕业证成绩单pdf电子版制作修改#真实工艺展示#真实防伪#diploma#degree专业一比一美国亚利桑那大学毕业证成绩单pdf电子版制作修改#真实工艺展示#真实防伪#diploma#degree
专业一比一美国亚利桑那大学毕业证成绩单pdf电子版制作修改#真实工艺展示#真实防伪#diploma#degreeyuu sss
 
原版1:1定制堪培拉大学毕业证(UC毕业证)#文凭成绩单#真实留信学历认证永久存档
原版1:1定制堪培拉大学毕业证(UC毕业证)#文凭成绩单#真实留信学历认证永久存档原版1:1定制堪培拉大学毕业证(UC毕业证)#文凭成绩单#真实留信学历认证永久存档
原版1:1定制堪培拉大学毕业证(UC毕业证)#文凭成绩单#真实留信学历认证永久存档208367051
 
MT. Marseille an Archipelago. Strategies for Integrating Residential Communit...
MT. Marseille an Archipelago. Strategies for Integrating Residential Communit...MT. Marseille an Archipelago. Strategies for Integrating Residential Communit...
MT. Marseille an Archipelago. Strategies for Integrating Residential Communit...katerynaivanenko1
 
Color Theory Explained for Noobs- Think360 Studio
Color Theory Explained for Noobs- Think360 StudioColor Theory Explained for Noobs- Think360 Studio
Color Theory Explained for Noobs- Think360 StudioThink360 Studio
 
10 must-have Chrome extensions for designers
10 must-have Chrome extensions for designers10 must-have Chrome extensions for designers
10 must-have Chrome extensions for designersPixeldarts
 
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10uasjlagroup
 
'CASE STUDY OF INDIRA PARYAVARAN BHAVAN DELHI ,
'CASE STUDY OF INDIRA PARYAVARAN BHAVAN DELHI ,'CASE STUDY OF INDIRA PARYAVARAN BHAVAN DELHI ,
'CASE STUDY OF INDIRA PARYAVARAN BHAVAN DELHI ,Aginakm1
 
昆士兰大学毕业证(UQ毕业证)#文凭成绩单#真实留信学历认证永久存档
昆士兰大学毕业证(UQ毕业证)#文凭成绩单#真实留信学历认证永久存档昆士兰大学毕业证(UQ毕业证)#文凭成绩单#真实留信学历认证永久存档
昆士兰大学毕业证(UQ毕业证)#文凭成绩单#真实留信学历认证永久存档208367051
 
cda.pptx critical discourse analysis ppt
cda.pptx critical discourse analysis pptcda.pptx critical discourse analysis ppt
cda.pptx critical discourse analysis pptMaryamAfzal41
 
General Knowledge Quiz Game C++ CODE.pptx
General Knowledge Quiz Game C++ CODE.pptxGeneral Knowledge Quiz Game C++ CODE.pptx
General Knowledge Quiz Game C++ CODE.pptxmarckustrevion
 
Design principles on typography in design
Design principles on typography in designDesign principles on typography in design
Design principles on typography in designnooreen17
 
guest bathroom white and blue ssssssssss
guest bathroom white and blue ssssssssssguest bathroom white and blue ssssssssss
guest bathroom white and blue ssssssssssNadaMohammed714321
 
Giulio Michelon, Founder di @Belka – “Oltre le Stime: Sviluppare una Mentalit...
Giulio Michelon, Founder di @Belka – “Oltre le Stime: Sviluppare una Mentalit...Giulio Michelon, Founder di @Belka – “Oltre le Stime: Sviluppare una Mentalit...
Giulio Michelon, Founder di @Belka – “Oltre le Stime: Sviluppare una Mentalit...Associazione Digital Days
 
DAKSHIN BIHAR GRAMIN BANK: REDEFINING THE DIGITAL BANKING EXPERIENCE WITH A U...
DAKSHIN BIHAR GRAMIN BANK: REDEFINING THE DIGITAL BANKING EXPERIENCE WITH A U...DAKSHIN BIHAR GRAMIN BANK: REDEFINING THE DIGITAL BANKING EXPERIENCE WITH A U...
DAKSHIN BIHAR GRAMIN BANK: REDEFINING THE DIGITAL BANKING EXPERIENCE WITH A U...Rishabh Aryan
 
AI and Design Vol. 2: Navigating the New Frontier - Morgenbooster
AI and Design Vol. 2: Navigating the New Frontier - MorgenboosterAI and Design Vol. 2: Navigating the New Frontier - Morgenbooster
AI and Design Vol. 2: Navigating the New Frontier - Morgenbooster1508 A/S
 
Making and Unmaking of Chandigarh - A City of Two Plans2-4-24.ppt
Making and Unmaking of Chandigarh - A City of Two Plans2-4-24.pptMaking and Unmaking of Chandigarh - A City of Two Plans2-4-24.ppt
Making and Unmaking of Chandigarh - A City of Two Plans2-4-24.pptJIT KUMAR GUPTA
 
Pearl Disrtrict urban analyusis study pptx
Pearl Disrtrict urban analyusis study pptxPearl Disrtrict urban analyusis study pptx
Pearl Disrtrict urban analyusis study pptxDanielTamiru4
 
FiveHypotheses_UIDMasterclass_18April2024.pdf
FiveHypotheses_UIDMasterclass_18April2024.pdfFiveHypotheses_UIDMasterclass_18April2024.pdf
FiveHypotheses_UIDMasterclass_18April2024.pdfShivakumar Viswanathan
 
TYPES-OF-VOLCANOES-AND-VOLCANIC-ERUPTIOn
TYPES-OF-VOLCANOES-AND-VOLCANIC-ERUPTIOnTYPES-OF-VOLCANOES-AND-VOLCANIC-ERUPTIOn
TYPES-OF-VOLCANOES-AND-VOLCANIC-ERUPTIOnbeetchunknown
 
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书zdzoqco
 

Recently uploaded (20)

专业一比一美国亚利桑那大学毕业证成绩单pdf电子版制作修改#真实工艺展示#真实防伪#diploma#degree
专业一比一美国亚利桑那大学毕业证成绩单pdf电子版制作修改#真实工艺展示#真实防伪#diploma#degree专业一比一美国亚利桑那大学毕业证成绩单pdf电子版制作修改#真实工艺展示#真实防伪#diploma#degree
专业一比一美国亚利桑那大学毕业证成绩单pdf电子版制作修改#真实工艺展示#真实防伪#diploma#degree
 
原版1:1定制堪培拉大学毕业证(UC毕业证)#文凭成绩单#真实留信学历认证永久存档
原版1:1定制堪培拉大学毕业证(UC毕业证)#文凭成绩单#真实留信学历认证永久存档原版1:1定制堪培拉大学毕业证(UC毕业证)#文凭成绩单#真实留信学历认证永久存档
原版1:1定制堪培拉大学毕业证(UC毕业证)#文凭成绩单#真实留信学历认证永久存档
 
MT. Marseille an Archipelago. Strategies for Integrating Residential Communit...
MT. Marseille an Archipelago. Strategies for Integrating Residential Communit...MT. Marseille an Archipelago. Strategies for Integrating Residential Communit...
MT. Marseille an Archipelago. Strategies for Integrating Residential Communit...
 
Color Theory Explained for Noobs- Think360 Studio
Color Theory Explained for Noobs- Think360 StudioColor Theory Explained for Noobs- Think360 Studio
Color Theory Explained for Noobs- Think360 Studio
 
10 must-have Chrome extensions for designers
10 must-have Chrome extensions for designers10 must-have Chrome extensions for designers
10 must-have Chrome extensions for designers
 
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10
CREATING A POSITIVE SCHOOL CULTURE CHAPTER 10
 
'CASE STUDY OF INDIRA PARYAVARAN BHAVAN DELHI ,
'CASE STUDY OF INDIRA PARYAVARAN BHAVAN DELHI ,'CASE STUDY OF INDIRA PARYAVARAN BHAVAN DELHI ,
'CASE STUDY OF INDIRA PARYAVARAN BHAVAN DELHI ,
 
昆士兰大学毕业证(UQ毕业证)#文凭成绩单#真实留信学历认证永久存档
昆士兰大学毕业证(UQ毕业证)#文凭成绩单#真实留信学历认证永久存档昆士兰大学毕业证(UQ毕业证)#文凭成绩单#真实留信学历认证永久存档
昆士兰大学毕业证(UQ毕业证)#文凭成绩单#真实留信学历认证永久存档
 
cda.pptx critical discourse analysis ppt
cda.pptx critical discourse analysis pptcda.pptx critical discourse analysis ppt
cda.pptx critical discourse analysis ppt
 
General Knowledge Quiz Game C++ CODE.pptx
General Knowledge Quiz Game C++ CODE.pptxGeneral Knowledge Quiz Game C++ CODE.pptx
General Knowledge Quiz Game C++ CODE.pptx
 
Design principles on typography in design
Design principles on typography in designDesign principles on typography in design
Design principles on typography in design
 
guest bathroom white and blue ssssssssss
guest bathroom white and blue ssssssssssguest bathroom white and blue ssssssssss
guest bathroom white and blue ssssssssss
 
Giulio Michelon, Founder di @Belka – “Oltre le Stime: Sviluppare una Mentalit...
Giulio Michelon, Founder di @Belka – “Oltre le Stime: Sviluppare una Mentalit...Giulio Michelon, Founder di @Belka – “Oltre le Stime: Sviluppare una Mentalit...
Giulio Michelon, Founder di @Belka – “Oltre le Stime: Sviluppare una Mentalit...
 
DAKSHIN BIHAR GRAMIN BANK: REDEFINING THE DIGITAL BANKING EXPERIENCE WITH A U...
DAKSHIN BIHAR GRAMIN BANK: REDEFINING THE DIGITAL BANKING EXPERIENCE WITH A U...DAKSHIN BIHAR GRAMIN BANK: REDEFINING THE DIGITAL BANKING EXPERIENCE WITH A U...
DAKSHIN BIHAR GRAMIN BANK: REDEFINING THE DIGITAL BANKING EXPERIENCE WITH A U...
 
AI and Design Vol. 2: Navigating the New Frontier - Morgenbooster
AI and Design Vol. 2: Navigating the New Frontier - MorgenboosterAI and Design Vol. 2: Navigating the New Frontier - Morgenbooster
AI and Design Vol. 2: Navigating the New Frontier - Morgenbooster
 
Making and Unmaking of Chandigarh - A City of Two Plans2-4-24.ppt
Making and Unmaking of Chandigarh - A City of Two Plans2-4-24.pptMaking and Unmaking of Chandigarh - A City of Two Plans2-4-24.ppt
Making and Unmaking of Chandigarh - A City of Two Plans2-4-24.ppt
 
Pearl Disrtrict urban analyusis study pptx
Pearl Disrtrict urban analyusis study pptxPearl Disrtrict urban analyusis study pptx
Pearl Disrtrict urban analyusis study pptx
 
FiveHypotheses_UIDMasterclass_18April2024.pdf
FiveHypotheses_UIDMasterclass_18April2024.pdfFiveHypotheses_UIDMasterclass_18April2024.pdf
FiveHypotheses_UIDMasterclass_18April2024.pdf
 
TYPES-OF-VOLCANOES-AND-VOLCANIC-ERUPTIOn
TYPES-OF-VOLCANOES-AND-VOLCANIC-ERUPTIOnTYPES-OF-VOLCANOES-AND-VOLCANIC-ERUPTIOn
TYPES-OF-VOLCANOES-AND-VOLCANIC-ERUPTIOn
 
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书
办理卡尔顿大学毕业证成绩单|购买加拿大文凭证书
 

Introduction to UX Research: Designing Surveys That Don't Suck!

  • 2. "They've done studies, you know. 60% of the time, it works every time. ” - Brian Fantana 06/16/12 2
  • 3. Surveys Aren’t Objective They are created by people who are biased and analyze their results based on their own (mis)perceptions. 06/16/12 3
  • 4. Epistemological Dead End? No. Awareness that the researcher plays an integral role in the process of measuring any given phenomena by deciding how to measure is key. There are ways to minimize researcher bias by creating better questions. 06/16/12 4
  • 5. Methods are not “just methods” How you measure what you are studying shapes what you find. 06/16/12 5
  • 6. Let’s say you are studying… How often teenagers use your website 06/16/12 6
  • 7. You might ask teenagers… “How often do you my website?” c.Very rarely d.Rarely e.Occasionally f.Frequently g.Very frequently Response options source: http://www.dataguru.org 06/16/12 7
  • 8. You might ask teenagers… “How often do you my website?” c.Very rarely d.Rarely And what’s wrong e.Occasionally with these???? f.Frequently g.Very frequently Response options source: http://www.dataguru.org 06/16/12 8
  • 9. You might ask teenagers… “How often do you my website?” ! !! VI TY EC TI c.Very rarely BJ d.Rarely SU And what’s wrong e.Occasionally with these???? f.Frequently g.Very frequently Response options source: http://www.dataguru.org 06/16/12 9
  • 10. “Hmmm, for me “frequently” really means….” A few times a week A few times a day So what are we really measuring? NOISE. 06/16/12 10
  • 11. What is “noise”? What we observe… 06/16/12 11
  • 12. What is “noise”? We WANT this! We DON’T WANT this! 06/16/12 12
  • 13. An imaginary scenario Let’s really stretch our thinking a bit here to provide a more concrete understanding of “noise” in your data. Imagine you have AT&T phone service and you’re trying to make a call. The signal isn’t clearly going through and you end up hearing everything but the other person’s voice. Frustrating, right? Thank God this is only a pretend scenario. 06/16/12 13
  • 14. How is AT&T related to “Noise”? When you develop bad questions, you don’t “hear” the message of your data clearly. 06/16/12 14
  • 15. So…why is “noise” bad??? • Badly structured and poorly worded questions that obfuscate meaning for participants provide bad data. • Bad data – Doesn’t answer your research question – Makes it difficult to interpret results – Is pretty much useless. 06/16/12 15
  • 16. More bad examples of frequently used response options a. Completely satisfied a. Totally like b. Very satisfied b. Very much like c. Fairly well satisfied c. Moderately like d. Somewhat dissatisfied d. Somewhat like e. Very dissatisfied e. Not like What’s the difference between What’s the difference between “fairly well” and “somewhat”? “moderately like” and “somewhat like”? 06/16/12 16
  • 17. More bad examples of frequently used response options a. Completely satisfied a. Totally like b. Very satisfied b. Very much like c. Fairly well satisfied c. Moderately like d. Somewhat dissatisfied d. Somewhat like e. Very dissatisfied e. Not like What’s the difference between What’s the difference between “fairly well” and “somewhat”? “moderately like” and “somewhat like”? Semantics 06/16/12 17
  • 18. Words like…. • Sometimes • Often • Moderately • Very • Not very much Are subjective = they mean different things for different people 06/16/12 18
  • 19. A better alternative? Actually asking people how often they do certain activities. 06/16/12 19
  • 20. In order to gather objective data… …your questions must mean the same thing for everyone. How often do you use my website? d.Never e.A few times a year f.Once a month g.2-3 times a month h.Once a week i.A few times a week j.Every day “Never” means never for everyone! 06/16/12 20
  • 21. Importance of “Anchors” Responses options serve as “anchors” for each question, determining the “location” of each responses as qualitatively distinct from the other. 06/16/12 21
  • 22. Importance of “Anchors” Responses options serve as “anchors” for each question, determining the “location” of each responses as qualitatively distinct from the other. Strongly Strongly disagree Disagree Neutral Agree Agree 06/16/12 22
  • 23. (Ideal) Response Symmetry • Good questions aim to have symmetrical quantitative/qualitative distance between anchors a. Never b. A few times a year The distance between “never” and c. Once a month “a few times a year” is proportionate to d. 2-3 times a month “every day” and “a few times a week”. e. Once a week f. A few times a week g. Every day 06/16/12 23
  • 24. Adherence to Logic & Linearity People are accustomed to paradigms that are intuitive, and often “linear”. Anchors should not be the exception. 06/16/12 24
  • 25. Intuitive Anchor Directions Never Always Disagree Agree Very poor Very good Not very Very important Important 06/16/12 25
  • 26. Value of Clarity If questions are clear and concise, participants would spend less time analyzing the questions themselves and more time on answering them. Mystic Arts, LLC 06/16/12 26
  • 27. So do good questions guarantee valid results? • No. But good questions offer more assurance that you are listening to signal and not the noise. • But answering your research question can be done in other ways • Behavioral Observations • Open ended qualitative questions • Many more… 06/16/12 27
  • 28. The End. So was this presentation simply awesome? a. Hell yeah! b. Definitely! c. Totally rocked! 06/16/12 28