Angus Carbarns, Director of Strategy at UX consultancy We Are Engines, explores key psychological biases underpinning consumer decision-making and their impact on market research and strategy. Rich in case studies and actionable take-aways, this talk should be of interest to brand managers, marketers, researchers and designers alike.

1. Perfectly Irrational:
The importance of psychological
validity in market research

2. Housekeeping.
Please Tweet using #CIMEvents and tag @CIMinfo_Scot
Post questions at Sli.do/PerfI and we’ll get to them at the end.

3. A little bit about me.
•Hi, I’m Angus Carbarns
•I run We Are Engines - a digital strategy, research and
marketing consultancy
•My goal is to make strategy and research more
accessible, so companies of all sizes can be more
effective
•I’m also a social science and qualitative research nerd

4. WORKSHOP
FACILITATION
DIGITAL
ANALYTICS
IN-HOUSE
TRAINING
UX
RESEARCH &
DESIGN
We Are Engines
MARKETING
STRATEGY
www.weareengines.com

5. Some of the brands I work with.

6. So, here’s what we’ll be covering.

7. 1. Why validity in market research, and why now?
2. What we can learn from social science about bias and behaviour
3. Making behavioural science more relevant for marketers
4. How to do research that’s more valid and useful

8. Why market research,
and why now?

9. Marketing has always been about
changing behaviour.

10. Globally, the market
research industry
generated an estimated
$47.3 Billion in 2018
Statistia 2020

11. 95% of new products fail.
Harvard Business Review
Most organisations aren’t
very strong at evaluating
whether the research they’re
doing is actually any good.

12. The opposite can also happen.

13. “No other product had ever performed so poorly in
consumer testing; the look, the taste, and the mouth-
feel were regarded as ‘disgusting’ and the idea that it
‘stimulates mind and body’ didn’t persuade anyone
the taste was worth tolerating.”
From Consumer.ology by Philip Graves

14. Demand for new insights comes at a cost.
•Increased pressure on research provers to differentiate and innovate services
•This has led to an explosion of ‘new’ methods and techniques
•But not all data is good data
•In the wrong hands, we can run into issues of validity and bias very quickly

15. How research plays out in most organisations.
Someone
identifies a complex
‘problem’
We conduct or
commission
research to find out
how to solve it
Re-assurance
comes from
‘significance’ or
‘depth’
Insights are
disseminated and
acted upon

16. But two key questions are often missing.
Someone
identifies a complex
‘problem’
We conduct or
commission
research to find out
how to solve it
Re-assurance
comes from
‘significance’ or
‘depth’
Insights are
disseminated and
acted upon
Are we measuring what we set out to measure?
Are we accounting for bias?

17. So where are we going wrong?

18. It’s often how we use research.

19. “There is a basic misunderstanding about
the forces that shape shopping behaviour.
Consumers believe that they act
rationally to maximise their personal best
interests when making purchase
decisions. Marketers assume that
consumer shopping decisions are
based on rational evaluation of the
features and attributes of the products
they sell”
Consumer psychologist, Peter Noel Murray

20. We tend to forget two key things.
1. No one has access the mental traits that drive decision-making
2. The process of asking people what they think affects what they say

21. What this means for market research
1. Asking people what they are likely to do may reveal intention, but
this often doesn’t correlate with action
2. We must be careful about how we ask, observe and design
research to minimise bias

22. What we can learn from social science
Recent thinking on bias and decision-making

23. Psychologists from various
schools of thought have
explored human decision-
making and behaviour from
a wide range of angles.
Freud’s therapeutic
approach
Theory of reasoned action
The theory of planned
behaviour

24. Freud’s greatest
contribution to psychology
was the unconscious.

25. •fMRI scanning and cognitive
neuropsychology
•Reveal unconscious prowesses
guiding behaviour
•This new field is sometimes referred
to as ‘the new subconscious’
We can now measure subconscious

26. “Our subliminal brain is invisible to us,
yet it influences our conscious
experience of the world in the most
fundamental of ways: how we view
ourselves and others, the meanings we
attach to the everyday events of our
lives, our ability to make the quick
judgement calls”
Leonard Mlodinaow, Subliminal

27. If the unconscious controls most of our decision-
making, what are the implications for marketing?
How do we research the inaccessible?

28. Behavioural economics
bridges theory and action.

29. ‘Thinking, Fast and Slow’
was key to bringing this
idea to the mainstream.

30. The central
thesis to the
book.
SYSTEM 1 SYSTEM 2
Dominant Mode of thought
>95% of brain activity
Secondary mode of thought
<5% of brain activity
(Binet & Carter)

31. This has big
implications for
marketers, as
we’ll explore.
SYSTEM 1 SYSTEM 2
Dominant Mode of thought
>95% of brain activity
Secondary mode of thought
<5% of brain activity
Fast and powerful
Honed by millions of years of evolution
Slow and limited
A more recent addition
Scans all sensory inputs
But can be primed by System 2 to watch
out for things
Selective attention
Guided by System 1 - feelings
associations and institutions
Unconscious and automatic
Associative and heuristic processing
Conscious and deliberate
Can follow learned rules of thought,
e.g. maths and logic
Primary decision-making
system
Strong influence on System 2
Secondary cross-checking
system
Mostly post-rationalises System 1
decisions, can sometimes overrule
(Binet & Carter)

32. Making behavioural science more
relevant for marketers.

33. RESEARCH COMMUNICATION
CONSUMER There are three ‘lenses’ we can
use to better understand
decision-making and bias in
marketing: the consumer, the
researcher, the marketer.

34. SYSTEM 1 SYSTEM 2
Does most brand choice work
Not logical or rational
Some brands just ‘feel’ more
attractive
Only kicks in close to the point
of purchase
More likely to prevent buying
than stimulate it
The consumer implications
(Binet & Carter)

35. The Nike
smell test.
Two identical pairs of Nike shoes
were placed in two identical
rooms, with one difference: one
was scented with floral tones and
the other wasn’t.

36. The Nike
smell test.
84% of people exposed to the
shoes in the scented room
expressed higher levels of
preference for the shoes and
were willing to pay $10 more for
them.
(Smell & Taste Foundation, 2014)

37. Where does
rationality factor
in to this?
Our self-perception of rationality
is powerful and takes over when we
describe our own behaviour.

38. This jars with how marketers talk
about consumer choice.

39. We still use models like AIDA
to rationalise behaviour, even
though there’s little evidence
this is how the consumer
decision-making process
works.
(Cogent Psychology)

40. Decision-making actually looks more like this.
“During the buying process, people oscillate long periods
of low interest, when they’re guided by feelings. Then
short bursts of more intense interest, when rational
thinking can play a role.”
(Binet & Carter)

41. SYSTEM 1 SYSTEM 2
Brand building is about creating
associations, feelings and habits
through exposure
Influenced by messages,
arguments & information, but
only late in the decision-making
process
The communications implications
(Binet & Carter)

42. We’re obsessed
with ‘rational’
messaging.
Brand marketing is awash ‘reasons
to believe’, propositions and
positioning statements.

43. Is the obsession
warranted?
While they’re certainly important,
their effectiveness in driving
behaviour in marketing and
advertising is questionable.

44. “Advertisements with high levels of
emotional content enhanced how
people felt about brands… however,
advertisements which were low on
emotional content had no effect on
how favourable the public were
towards brand”
Robert Heath, University of Bath

45. The reality is, all marketing needs a mix of emotional
and rational messaging in the right proportions to
persuade customers to take action.

46. SYSTEM 1 SYSTEM 2
Hard to research.
System 1 dominates, but we’re
mostly unaware of its influence
Research exaggerates
importance.
We mistakenly attribute actions to
System 2 because it’s what we’re
aware of.
The research implications
(Binet & Carter)

47. In 2009, Walmart asked their
customers:
“Would you like Walmart
aisles to be less
cluttered?”
Their customers said:
“Yes!”

48. $100’s of millions were
spent ‘decluttering’ stores
15% of store inventory cut

49. Walmart lost $2 Billion
in revenue in one year.

50. Why did this happen?
Walmart took the consumers’ stated preference at face value
without considering whether the research design was valid.

51. How to do research that’s more valid.

52. 1. Define clear research questions &
objectives.

53. 1. What do we want to know and why?
2. What do we already know about this issue/group/product?
3. What’s the hypothesis for our research?
4. What’s an appropriate way of measuring this? (qual or quant)
5. What will this data be used for?
Defining clear research objectives

54. Choose the right method for the job
Qualitative or quantitive
methods have to be
chosen carefully.
Start with what you
want to know and work
back from there.

55. 2. Observe actual behaviour
whenever possible.

56. To really understand what
people do and why, we
can’t just ask them, we
have to observe them.
Observation provides deep
insight into people’s needs,
habits and pain points with a
product or service.

57. Selecting the right type of observation.
Method Context of use Pros Cons
Controlled
observation
Controlled observation tends
to take place in a laboratory
environment (usability lab).
Usually focused on quant data
Easy to replicate
Easy to analyse
Quick to conduct
Can be too restrictive
Observation may
alter behaviour
Naturalistic
Observation
A naturalistic observation
involves studying the user “in
the wild” (field study)
Improved validity -
closer representation
to real life
Hard to replicate
Hard to adjust for
external variables

58. Covert observation
• Is conducted without the person(s) knowing
• Not affected by researcher bias
• Ethically questionable
Overt observation
• The person(s) knows they are being observed
• Affected by researcher bias
• Ethically sound and almost always the right option
Covert or overt observation?

59. 3. Be prepared to go off-script.

60. Use interviews as a means to go
deep into a customer or users’
thoughts, emotions and attitudes.
Use a script to guide, but always
follow up on anything that piques
your interest.
Yes it’s biased, but it’s likely
you’ll get to insights you won’t
find any other way.

61. Try ‘Five Why’s’ to get deeper insights
Ask a broad question about your participant’s habits or behaviours1.
2.
3.
In response to their answer, ask a ‘why’ question (e.g. ‘why do you think that is?’)
Allow the participant to respond to your first “why”, then repeat 4 times

62. 4. Use data that’s less subject to bias.

63. Mine digital data for insight into behaviours, preferences and trends:
• Google Analytics - website behaviour
• Google Trends - Search behaviour
• Google Shopping Insights - Category behaviour
• Facebook Insights - Social behaviour
• Google Keyword Planner - Search / Shopping behaviour
These provide an unfiltered, unbiased snapshot of audience needs
Aggregated digital data = less bias.

65. 5. Triangulate your findings, ideally
with others.

66. Triangulation involves
assimilating findings from
a variety of sources to
identify patterns and
insights.

67. Inviting others into the
process can help
overcome our own
confirmation bias;
our tendency to look
for data that fits with
our existing
assumptions.

68. One of the best methods
of triangulating a broad
range of findings is
through Affinity
Diagrams.

69. Space, saturate and group.
Use a physical space
Saturate it with data/
findings
Group findings into
themes

70. Map your findings in a
single place with a
customer journey map.
This enables insights to
be easily shared, edited
and added to.

71. 6. If you’re researching customers,
have an opinion.

72. Focus on being useful.
“Good researchers are usually those who can form a
strong opinion after research, and act like thought
partners, as opposed to those who are only craft in
conducting perfectly rigorous research”.
Elsa Ho, Facebook

73. Be bold, have a perspective.
1. Who else will draw out the key insights?
2. Who else is representing the customer?
3. What do I think we should do about this?
4. What value does this research bring to the table?

74. To wrap things up.

75. Bias affects all of us: marketers, consumers and researchers.
We need to know where, how and when it can impact us.
Once we do, we can work with it.
We can even learn to leverage it in our favour.

76. Thanks for listening.

77. Come and say hello.
Twitter: @WAEConsultancy
LinkedIn: /company/we-are-engines
Website: www.weareengines.com