Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Beyond
Measure
@mulegirl
!#? !#?
:-)
DEEP
THOUGHT,
TELL US THE
ANSWER…
OK
7,500,000
YEARS LATER…
???
42!
42!
!#!?&wtf!
¯_(ツ)_/¯
IT WOULD HAVE
BEEN SIMPLER TO
KNOW THE ACTUAL
QUESTION.
ONLY WHEN YOU
KNOW THE
QUESTION, WILL YOU
KNOW WHAT THE
ANSWER MEANS.
I am a rational biped.
All rational bipeds rule.
So, I rule.
:-)
:-(
We have to work together
to make better decisions.
Quantitative
approaches feel
better, but are often
actually worse.
We are
imperfect
humans
designing things
for other
imperfect
humans.
DON’T
PANIC
Fe
Kirk Siang/Flickr
60%
Kirk Siang/Flickr
70%
Kirk Siang/Flickr
$1<
Fe
Flickr/eskimo_jo
[dramatization]
[a dramatization]
The Nourishing Home / Kelly Smith
[dramatization]
Lucky!
luckyironfish.com
O
O
O
O
HO
HO
OH
OH
H2C
H2C
CH2
CH2
CH2C
C
CH2 C
C
:N
N:
Human problems
require illogical
approaches.
Stories
have
power
Flickr/ Billy Wilson
Only 12
people?!?
2,500,000
000,000,
000,000
bytes
90%?
011010000110010101101
100011100000010000001
10110101100101
010010010010000001100
001011011010010000001
1000100110010101101...
:-)
This is pretty
creepy when you
think about it.
Flickr/ thevlue
I am a rational biped.
Original image: Olly Moss
We are blind to
our minds.
Ease
Clear Display
Related Experience
Primed Idea
Good Mood
Feels True
Feels Familiar
Feels Good
Feels Effortless
Data doesn’t
change minds.
26%
Our access to data
has evolved,
our brains haven’t.
Data doesn’t
have meaning.
42
Obviously,
what I can
count
should
count more.
We have a lot of data.
We make a lot of decisions.
We like stories.
We aren’t good at math.
The hard decisions aren’t
math...
DON’T
PANIC
QuantitativeQualitative
What is going on?
How are people behaving?
Why?
What does it mean?
What is the problem?
Any ideas?
Where?
When?

How many?...
Usability Studies
Field Studies
Interviews
Clickstream Analysis
A/B Testing
Surveys*
Advantages of Qualitative Methods
In-depth
Quick revisions
Captures human experience
Powerful & compelling data
Perils of Qualitative Methods
Dependent on personal skills
Influenced by personal biases
Time-consuming data analysis
Conf...
The
Hawthorne
Effect
Advantages of Quantitative Methods
Control
Cause & effect
Ability to generalize
Comparisons
Distance
Perils of Quantitative Methods
Misses context
Inflexible
Less detail on attitudes & motivation
“Laboratory” results
Statis...
Are you a
statistician?
Flickr/ Billy Wilson
Tight
control.
THICK
DESCRIPTION
BIG
DATASET
No fishing trips.
Know your POV
first.
Flickr/gabbahey
The limit of
split testing
Your glorious
potential.
A B
Push Push
Traffic Quality
Safe Exciting
Attention $$$
New Familiar!
Mine Ours
More Enough
Yes
What is.
What
ought to
be
?
Questions
Practical
Questions
?
?
?
Specific
Actionable
A bad question
:-(How do we
get Millennials
to like us?
A better question
:-)How do recent college
graduates living in
cities decide what to
have for dinner?
The best question
The unknown that
carries the most risk.
? ? ?
Target customers don’t value product
Customers need something else
Business model doesn’t support it
Organization can’t pr...
Research activities
are simply ways to
answer questions.
InterviewsInterviews
Usability
Testing
A/B
Testing
Contextual
Inquiry
Literature
Review
SWOT
Analysis
Brand
Audit
Usabilit...
InterviewsInterviews
Usability
Testing
A/B
Testing
Contextual
Inquiry
Literature
Review
SWOT
Analysis
Brand
Audit
Usabilit...
What do we
need to know
about?
What kind of
decision will it
inform?
How long do
we have?
What is our
budget?
Research
Act...
What do we
need to know
about?
What kind of
decision will it
inform?
How long do
we have?
What is our
budget?
Phone
Interv...
Dogma
Surveys
Surveys are the most
dangerous research tool.
Strongly disagree
Disagree
Neither agree nor disagree
Agree
Strongly agree
It is too easy
to run a
survey.
Easy feels
true.
Bad surveys don’t
smell bad.
Lies Poppycock Irrelevancies
Direct interaction with
users is prohibited by my
organization, but I have
been allowed to conduct a
simple survey by emai...
[a dramatization]
The Nourishing Home / Kelly Smith
My boss is a convert to Foresee.
She was apparently very skeptical
of it at first, but she's a very
analytical person and ...
Customer satisfaction
is a lie.
Will the people I’m
surveying be willing and
able to provide a truthful
answer to my question?
Never ask people
what they like.
Never ask people
to remember.
Never ask people
to predict.
How likely are you to
attend the opera?
Very!
How concerned are you
about looking smart?
Very!
How likely is it you are
telling the truth?
Not Very!
ONLY WHEN YOU
KNOW THE
QUESTION, WILL YOU
KNOW WHAT THE
ANSWER MEANS.
“At its core, all business is
about making bets on
human behavior.”
—Ben Wiseman, WSJ
Selecting Metrics
Define
success
first.
The most
measurable data
is not the most
valuable.
Vanity Metrics
Get
actionable
insights.
Get
useful
information.
Make better
choices.
We are
imperfect
humans
designing things
for other
imperfect
humans.
The most
measurable data
may not be
valuable.
Know your
reason.
What and Why
Flickr/johnwiechecki
Thank you!
erika@muledesign.com
@mulegirl
www.muledesign.com
Brief books for people who make websites No.
9
JUSTENOUGH
RES...
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Akkurat nok brukerinnsikt - Erika Hall
Upcoming SlideShare
Loading in …5
×

Akkurat nok brukerinnsikt - Erika Hall

181 views

Published on

Brukertestlab med avansert kamerautstyr? Store undersøkelser med 175 variabler og multiple regresjoner? Nope. Gjør det enkelt. Hent inn akkurat det du trenger av innsikt og bruk enkle metoder. Erika viser deg hvordan.

Published in: Design
  • Be the first to comment

  • Be the first to like this

Akkurat nok brukerinnsikt - Erika Hall

  1. 1. Beyond Measure @mulegirl
  2. 2. !#? !#?
  3. 3. :-)
  4. 4. DEEP THOUGHT, TELL US THE ANSWER… OK
  5. 5. 7,500,000 YEARS LATER…
  6. 6. ??? 42!
  7. 7. 42! !#!?&wtf! ¯_(ツ)_/¯
  8. 8. IT WOULD HAVE BEEN SIMPLER TO KNOW THE ACTUAL QUESTION.
  9. 9. ONLY WHEN YOU KNOW THE QUESTION, WILL YOU KNOW WHAT THE ANSWER MEANS.
  10. 10. I am a rational biped. All rational bipeds rule. So, I rule.
  11. 11. :-)
  12. 12. :-(
  13. 13. We have to work together to make better decisions.
  14. 14. Quantitative approaches feel better, but are often actually worse.
  15. 15. We are imperfect humans designing things for other imperfect humans.
  16. 16. DON’T PANIC
  17. 17. Fe
  18. 18. Kirk Siang/Flickr 60%
  19. 19. Kirk Siang/Flickr 70%
  20. 20. Kirk Siang/Flickr $1<
  21. 21. Fe
  22. 22. Flickr/eskimo_jo [dramatization]
  23. 23. [a dramatization] The Nourishing Home / Kelly Smith
  24. 24. [dramatization]
  25. 25. Lucky!
  26. 26. luckyironfish.com
  27. 27. O O O O HO HO OH OH H2C H2C CH2 CH2 CH2C C CH2 C C :N N:
  28. 28. Human problems require illogical approaches.
  29. 29. Stories have power
  30. 30. Flickr/ Billy Wilson Only 12 people?!?
  31. 31. 2,500,000 000,000, 000,000 bytes
  32. 32. 90%?
  33. 33. 011010000110010101101 100011100000010000001 10110101100101 010010010010000001100 001011011010010000001 100010011001010110100 101101110011001110010 000001110111011000010 111010001100011011010 000110010101100100 011010000110010101101 100011100000010000001 10110101100101
  34. 34. :-)
  35. 35. This is pretty creepy when you think about it. Flickr/ thevlue
  36. 36. I am a rational biped.
  37. 37. Original image: Olly Moss
  38. 38. We are blind to our minds.
  39. 39. Ease Clear Display Related Experience Primed Idea Good Mood Feels True Feels Familiar Feels Good Feels Effortless
  40. 40. Data doesn’t change minds.
  41. 41. 26%
  42. 42. Our access to data has evolved, our brains haven’t.
  43. 43. Data doesn’t have meaning.
  44. 44. 42
  45. 45. Obviously, what I can count should count more.
  46. 46. We have a lot of data. We make a lot of decisions. We like stories. We aren’t good at math. The hard decisions aren’t math problems, anyway,
  47. 47. DON’T PANIC
  48. 48. QuantitativeQualitative
  49. 49. What is going on? How are people behaving? Why? What does it mean? What is the problem? Any ideas? Where? When?
 How many? How much? How often? Which yields more?
  50. 50. Usability Studies Field Studies Interviews Clickstream Analysis A/B Testing Surveys*
  51. 51. Advantages of Qualitative Methods In-depth Quick revisions Captures human experience Powerful & compelling data
  52. 52. Perils of Qualitative Methods Dependent on personal skills Influenced by personal biases Time-consuming data analysis Confidentiality issues Hard to visualize
  53. 53. The Hawthorne Effect
  54. 54. Advantages of Quantitative Methods Control Cause & effect Ability to generalize Comparisons Distance
  55. 55. Perils of Quantitative Methods Misses context Inflexible Less detail on attitudes & motivation “Laboratory” results Statistical significance
  56. 56. Are you a statistician?
  57. 57. Flickr/ Billy Wilson Tight control.
  58. 58. THICK DESCRIPTION BIG DATASET
  59. 59. No fishing trips.
  60. 60. Know your POV first.
  61. 61. Flickr/gabbahey
  62. 62. The limit of split testing Your glorious potential.
  63. 63. A B
  64. 64. Push Push
  65. 65. Traffic Quality
  66. 66. Safe Exciting
  67. 67. Attention $$$
  68. 68. New Familiar!
  69. 69. Mine Ours
  70. 70. More Enough
  71. 71. Yes
  72. 72. What is. What ought to be
  73. 73. ?
  74. 74. Questions
  75. 75. Practical Questions ? ? ? Specific Actionable
  76. 76. A bad question :-(How do we get Millennials to like us?
  77. 77. A better question :-)How do recent college graduates living in cities decide what to have for dinner?
  78. 78. The best question The unknown that carries the most risk. ? ? ?
  79. 79. Target customers don’t value product Customers need something else Business model doesn’t support it Organization can’t produce it Someone else is doing it better Project Risks !#? !#?
  80. 80. Research activities are simply ways to answer questions.
  81. 81. InterviewsInterviews Usability Testing A/B Testing Contextual Inquiry Literature Review SWOT Analysis Brand Audit Usability Testing Competitive Analysis Heuristic Analysis Descriptive Evaluative Evaluative Evaluative Analytic Analytic Generative Descriptive Competition Users ProductQuestions About Organization
  82. 82. InterviewsInterviews Usability Testing A/B Testing Contextual Inquiry Literature Review SWOT Analysis Brand Audit Usability Testing Competitive Analysis Heuristic Analysis Descriptive Evaluative Evaluative Evaluative Analytic Analytic Generative Descriptive Competition Users ProductQuestions About Organization Topic Purpose Time Money
  83. 83. What do we need to know about? What kind of decision will it inform? How long do we have? What is our budget? Research Activity
  84. 84. What do we need to know about? What kind of decision will it inform? How long do we have? What is our budget? Phone Interviews In-Person Interviews Contextual Inquiry Usability Testing Competitive Analysis
  85. 85. Dogma
  86. 86. Surveys
  87. 87. Surveys are the most dangerous research tool. Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree
  88. 88. It is too easy to run a survey.
  89. 89. Easy feels true.
  90. 90. Bad surveys don’t smell bad.
  91. 91. Lies Poppycock Irrelevancies
  92. 92. Direct interaction with users is prohibited by my organization, but I have been allowed to conduct a simple survey by email to identify usability issues.
  93. 93. [a dramatization] The Nourishing Home / Kelly Smith
  94. 94. My boss is a convert to Foresee. She was apparently very skeptical of it at first, but she's a very analytical person and was converted by its promise of being able to quantify unquantifiable data —like "satisfaction".
  95. 95. Customer satisfaction is a lie.
  96. 96. Will the people I’m surveying be willing and able to provide a truthful answer to my question?
  97. 97. Never ask people what they like.
  98. 98. Never ask people to remember.
  99. 99. Never ask people to predict.
  100. 100. How likely are you to attend the opera? Very!
  101. 101. How concerned are you about looking smart? Very!
  102. 102. How likely is it you are telling the truth? Not Very!
  103. 103. ONLY WHEN YOU KNOW THE QUESTION, WILL YOU KNOW WHAT THE ANSWER MEANS.
  104. 104. “At its core, all business is about making bets on human behavior.” —Ben Wiseman, WSJ
  105. 105. Selecting Metrics
  106. 106. Define success first.
  107. 107. The most measurable data is not the most valuable.
  108. 108. Vanity Metrics
  109. 109. Get actionable insights.
  110. 110. Get useful information.
  111. 111. Make better choices.
  112. 112. We are imperfect humans designing things for other imperfect humans.
  113. 113. The most measurable data may not be valuable.
  114. 114. Know your reason.
  115. 115. What and Why Flickr/johnwiechecki
  116. 116. Thank you! erika@muledesign.com @mulegirl www.muledesign.com Brief books for people who make websites No. 9 JUSTENOUGH RESEARCH Erika Hall

×