2. Today’s Agenda:
• reveal how prediction markets work;
• reveal why prediction markets are more cost effective than traditional
methods;
• help you to determine when to utilize a prediction market for your
research; and
• demonstrate why prediction markets deliver accurate concept tests
more quickly than traditional methods.
3. John Barrett
• Joined Infosurv in 2010 and brings over 20 years of experience in
marketing research, consultative sales, and business development.
• Has extensive experience in the Marketing Research Industry in providing
strategic insights and unique solutions to business challenges for his
clients.
• John’s experience spans multiple verticals and industries including, CPG,
pharmaceuticals, tel-com, retail, hospitality, financial services, consumer
electronics, technology, and apparel.
• John holds a B.S. from Northeastern University in Business Administration
and has done Master’s work at Rutgers University.
• He lives in the Atlanta area with his wife and three children and is active
in the community as a Little League Coach and a volunteer with Junior
Achievement.
4. Tapping the Wisdom of
Crowds
A Comparison Case Study on
Prediction Markets for Concept Testing
5. Infosurv Overview
• Full-service online research firm focused on
innovation
• Core service is online survey design, administration, &
analysis
• Diverse and cutting-edge research services
• Employee, customer, & B2B/B2C market research
surveys
• Infosurv Concept Exchange (iCE) prediction market
system
• Strong industry reputation
• Founded in 1998
• 300+ clients worldwide including many Fortune 500's & major
governmental agencies
6. What if you could predict the future?
6
• New Product Concepts
• Advertising Concepts
• Packaging & Design Concepts
• SKU Optimization
7. 7
large groups of people are
smarter than an elite few, no
matter how brilliant—better
at solving problems,
fostering innovation, coming
to wise decisions, even
predicting the future.
The Wisdom of Crowds
8. What is a Prediction Market?
8
• Market for predicting future outcomes
– Traders invest in outcomes
– Weight their decision with currency allocation
– Rewarded for accurate predictions
• Think
10. How Do They Compare?
10
Survey
•Targeted
•What would you do?
•Dozens of metrics
•Flat incentive
•3-4 weeks
Prediction Market
•General Population
•What would they do?
•Most important number
•Variable incentive based on
performance
•3-4 days
11. Client Screening Program
• Methodology
– Mall Intercept
– Screen for target population
•Consider sufferer or user
– Screen for 2 – 4 categories per quarter
•Each respondent evaluates a MAXIMUM of 8 concepts across
2 categories
– Sample size is approximately 250 per category
11
12. Client Screening Program
• Question Areas
– Purchase Intent
– Uniqueness
– Value
– For Top 3 Ideas
• Competitive Strength
• Immediacy of purchase
• Feature/Benefit most interested in
• Uniqueness of feature/benefit above
12
13. Prediction Markets for Concept Testing:
How They Work
13
• Traders Get Virtual Dollars
• Each Concept Has a Starting Price
• Prices Fluctuate as Trading Occurs
• Price Movements are Analyzed
14. 14
• Methodology
– Tested 9 concepts in two categories; 4 in health, 5 in beauty
– Separate prediction market for each category
– One control concept used per category to gauge absolute concept strength
– General population sample; self-selection for market participation
– Total of ~200 traders per market
iCE Methodology
15. 15
Results for Health Category
Volume Weighted Average Price
$15.00
$20.00
$25.00
$30.00
$35.00
$40.00
1
6
11
16
21
26
31
36
41
46
51
56
61
66
71
76
81
86
91
96
101
106
111
116
121
126
131
136
141
146
151
156
161
166
171
176
181
186
191
Concept A
Concept B
Concept C
Control Concept
VWAP
# of traders
16. CPG Prediction Market Results
Compared to Survey Data - Health*
16
Concepts VWAP**
iCE
Index SI
iCE
Rank
Strength
Index Rank
Concept A $36.41 146 102 1 1
Control Concept $22.94 92 100 2 2
Concept B $20.72 83 75 3 4
Concept C $19.04 76 85 4 3
**Volume-Weighted Average Price
17. Open-Ended Comments:
Winning Concepts - Health
Concept A
(Winning Concept)
# of
Responses
% of
Traders*
Long lasting 68 53%
Would sell better/appeal to
consumers
16 13%
Works quickly 9 7%
Good concept/product/idea 9 7%
I would use it 9 7%
Price/Value 7 5%
Good/Popular/Well-known brand 6 5%
Flavor/taste 6 5%
Packaging 6 5%
17
Concept CONTROL
(Second Place Concept)
# of
Responses
% of
Traders*
Effectiveness 60 58%
Good concept/product/idea 7 7%
Works quickly 7 7%
Would sell better/appeal to
consumers
7 7%
Long lasting 6 6%
Price/Value 6 6%
Not specified 5 5%
* Note: Traders could provide more than one answer. Therefore, percentages may add up to more than 100%.
18. Demographics: Health Market
18
• The iCE Health Market skews younger than the survey—40% of survey
respondents were under age 45, while 56% of iCE participants were under 40.
• The iCE market also relied more heavily on female respondents (62%), while
the survey was half male/half female.
19. CPG Prediction Market Results
Compared to Survey Data - Beauty*
19
Concepts VWAP**
iCE
Index SI
iCE
Rank
Strength
Index Rank
Concept D $33.48 167 158 1 1
Concept A $23.87 119 74 2 3
Control Concept $17.49 87 100 3 2
Concept B $12.66 63 58 4 5
Concept C $11.97 60 68 5 4
*Relative Study Investment: Prediction Market = 50% of traditional survey
**Volume-Weighted Average Price
20. Demographics: Beauty Market
20
• The iCE market included female respondents (46%), while the survey was
exclusively males (100%).
21. 21
“Excellent way to judge a product. Makes one think of the best logic and
back it up since money is on the line”
“I loved it. You should have more of these to do. They’re fun, and helps
you to learn more about stock marketing trading.”
“I loved the idea! It was a super fun way to preview new products and get
feedback on what other people think of the idea of those products”
Trader Quotes
22. 22
• In both categories, the iCE prediction market successfully predicted
the most preferred concept and the same directional rank order of the
other concepts, with minor exceptions.
• The study was completed in less than two business days and utilized
a general population sample. By comparison, the traditional survey
data was obtained and analyzed over a period of weeks and relied
upon a targeted sample with incidence rates as low as 7%.
• Over 70% of traders described the process using words such as “fun,
interesting, cool, love it”.
• The client estimates that using prediction markets as a substitute for
traditional methods would cut concept screening costs. .
Conclusions & Project Notes
23. What does this mean for researchers?
• Prediction markets are powerful alternatives to surveys
for concept screening
• Incentivizing thoughtful, measured responses is possible
• Your customers are not always the best predictor of their
own behavior
• Other applications for prediction markets and
crowdsourcing in MR?
23
27. Thank you for joining us!
The slide deck along with a recording of today’s
presentation will be available for download via our
website. We will be sending all attendees a link to the
slide deck as soon as it is available.
