This document contains a collection of quotes, images, and musings on the topics of data, decision making, and human behavior. Some key ideas expressed are: - Quantitative data does not always yield the best decisions and qualitative approaches are often better at understanding problems. - Surveys are a limited tool for understanding users as people are not always truthful or able to recall accurately. - Customer satisfaction metrics explain little and most-hated companies can still perform well, so other factors beyond surveys should inform decisions. - As imperfect beings, we must design with an understanding of human flaws and limitations rather than assume perfect rationality.

1. Beyond
Measure
@mulegirl

2. !#? !#?

3. :-)

4. DEEP
THOUGHT,
TELL US THE
ANSWER…
OK

5. 7,500,000
YEARS LATER…

6. ???
42!

7. 42!
!#!?&wtf!
¯_(ツ)_/¯

8. IT WOULD HAVE
BEEN SIMPLER TO
KNOW THE ACTUAL
QUESTION.

9. ONLY WHEN YOU
KNOW THE
QUESTION, WILL YOU
KNOW WHAT THE
ANSWER MEANS.

11. I am a rational biped.
All rational bipeds rule.
So, I rule.

13. :-)

14. :-(

15. We have to work together
to make better decisions.

18. Quantitative
approaches feel
better, but are often
actually worse.

19. We are
imperfect
humans
designing things
for other
imperfect
humans.

20. DON’T
PANIC

22. Fe

23. Flickr/fungleo
Kirk Siang/Flickr

24. Fe

25. Flickr/eskimo_jo
[dramatization]

27. [a dramatization]
The Nourishing Home / Kelly Smith

28. [dramatization]

30. Lucky!

31. luckyironfish.com

32. oo
O
O
O
O
HO
HO
OH
OH
H2C
H2C
CH2
CH2
CH2C
C
CH2 C
C
:N
N:

34. Human problems
require illogical
approaches.

35. Stories
have
power

36. Flickr/ Billy Wilson
Only 12
people?!?

37. 2,500,000
000,000,
000,000
bytes

40. 90%?

42. 011010000110010101101
100011100000010000001
10110101100101
010010010010000001100
001011011010010000001
100010011001010110100
101101110011001110010
000001110111011000010
111010001100011011010
000110010101100100
011010000110010101101
100011100000010000001
10110101100101

43. :-)

44. 0110100001100101011
0110001110000001000
000110110101100101
0100100100100000011
0000101101101001000
0001100010011001010
1101001011011100110
0111001000000111011
1011000010111010001
1000110110100001100
10101100100
$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$
$$$$$$$$$$$$$$$$$$$

45. This is pretty
creepy when you
think about it.
Flickr/ thevlue

51. Flickr/ thevlue
This is pretty
creepy when you
think about it.

53. I am a rational biped.

55. one line is definitely longer

57. Original image: Olly Moss

58. We are blind to
our minds.

59. 1 2
System System
1
System
2
System

60. Fast
Intuitive
Emotional1
System
System
1

61. 2
SystemSlow
Deliberative
Effortful

62. Data doesn’t
change minds.

64. Ease
Clear Display
Related Experience
Primed Idea
Good Mood
Feels True
Feels Familiar
Feels Good
Feels Effortless

65. 26%

66. Feeling confident?

67. You might have a case of
Dunning-Kruger

68. “The incompetent are often
blessed with an inappropriate
confidence, buoyed by
something that feels to them like
knowledge.”
—David Dunning

69. Our access to data
has evolved,
our brains haven’t.

70. Data doesn’t
have meaning.

71. 42

72. Obviously,
what I can
count
should
count more.

73. We have a lot of data.
We make a lot of decisions.
We like stories.
We aren’t good at math.
The hard decisions aren’t
math problems, anyway,

74. DON’T
PANIC

76. QuantitativeQualitative

77. QuantityQuality

79. What is going on?
How are people behaving?
Why?
What does it mean?
What is the problem?
Any ideas?
Where?
When?
How many?
How much?
How often?
Which yields more?

80. Usability Studies
Field Studies
Interviews
Clickstream Analysis
A/B Testing
Surveys*

81. Are you a
statistician?

82. Flickr/ Billy Wilson
Tight
control.

83. 42!
Hi!
37banana672

84. No fishing trips.

85. Know your POV
first.

86. Flickr/gabbahey

88. The limit of
split testing
Your glorious
potential.

89. A B

90. Push Push

91. Traffic Quality

92. Safe Exciting

93. Attention $$$

94. New Familiar!

95. Mine Ours

96. More Enough

97. Yes

98. What is.
What
ought to
be

99. ?

102. DecisionsDecisions

104. EVIDENCE

105. EVIDENCE

106. What should we do?

107. Surveys

108. Surveys are the most
dangerous research tool.
Strongly disagree
Disagree
Neither agree nor disagree
Agree
Strongly agree

109. It is too easy
to run a
survey.

110. Easy feels
true.

111. Bad surveys don’t
smell bad.

113. Direct interaction with
users is prohibited by my
organization, but I have
been allowed to conduct a
simple survey by email to
identify usability issues.

115. [a dramatization]
The Nourishing Home / Kelly Smith

116. My boss is a convert to Foresee.
She was apparently very skeptical
of it at first, but she's a very
analytical person and was
converted by its promise of being
able to quantify unquantifiable data
—like "satisfaction".

117. Customer satisfaction
is a lie.

118. “Changes in customers’
satisfaction levels explain less
than 1% of the variation in
changes in their share of
category spending.”
—MIT Sloan Management Review

119. “Customer-service scores have
no relevance to stock market
returns… the most-hated
companies perform better than
their beloved peers.”
—Bloomberg Businessweek

120. “Over a one year period 6% of
‘extremely satisfied customers’
closed their accounts whilst 5.8%
of ‘unsatisfied customers’ closed
their accounts.”
—HBR, 1995

125. Will the people I’m
surveying be willing and
able to provide a truthful
answer to my question?

126. Never ask
people
what they like.

127. Never ask
people
to remember.

128. Never ask
people
to predict.

129. How likely are you to
attend the opera?
Very!

130. How concerned are you
about looking smart?
Very!

131. How likely is it you are
telling the truth?
Not Very!

132. Lies Poppycock Irrelevancies

133. Be Better

134. ONLY WHEN YOU
KNOW THE
QUESTION, WILL YOU
KNOW WHAT THE
ANSWER MEANS.

135. “At its core, all business is about
making bets on human
behavior.”
—Ben Wiseman, WSJ

136. Define
success
first.

137. The most
measurable data
is not the most
valuable.

138. Get
actionable
insights.

139. Get
useful
information.

140. Make better
choices.

141. We are
imperfect
humans
designing things
for other
imperfect
humans.

142. Know your
reason.

143. What and Why
Flickr/johnwiechecki

144. Thank you.