Predictive Markets & Experimental Results

Predictive Markets & the Wisdom of Crowds Experiments A Market Research Project by BrainJuicer ® Produced December 2005 The Mind Reading Agency

Extraordinary delusions and the Madness of Crowds, Charles MacKay (1841)

The Wisdom of Crowds – How the Many Can Be Smarter than the Few, James Surowiecki (2004)

Diversity
Independence
Faithful aggregation
Enables a crowd to be wise

Galton’s surprise findings. Collective Guess = 1197 lbs Actual Weight = 1198 lbs

How did the market know? Rockwell – Down 3% Lockheed – Down 3% Martin Marietta – Down 3% Morton Thiokol – Down 12%

Qu. Is there a more efficient alternative?

IEM - The original decision market X 596 Polls (1988 - 2000)  ¾

French – Referendum Non 53.6% Oui 46.4% Non 54.9% Oui 45.1% 29 th May – Betting BetFair 30 th May - Result (70% turn out) Non 51.0% Oui 49.0% 28 th May – Polls IPSOS

A random collection of 500 individuals will be as accurate as a targeted sample
Predictive Markets will be as accurate as monadics
The ‘crowd’ answering for the market will be as accurate as people answering for themselves
3 hypotheses to test on… Predictive Markets’ & Wisdom of Crowds

The Experiments Monadic Tests of 3, 10, 15 & 20 ideas, with matched targeted samples of 100 Predictive Markets of 3, 10, 15 & 20 ideas, with diverse ‘crowds’ of 500

Predictive Market 1 - UK  A diverse crowd will be as accurate as a targeted sample  A ‘Predictive Market’ will be as accurate as a monadic test  Answering for the market as accurate as answering for oneself Respondent Base Sizes: Monadic = 100 per cell / Predictive Market *No Betting = 507 *** 16 43 J 45 49 I *** 35 54 H *** 28 64 G *** 28 64 F 67 67 UK Norms (top 2 box) 70 74 E 86 78 D 80 81 C + 76 83 B 85 85 A Significant Differences Predictive Market 2* With diverse group of 500 people - *no betting (Top 2 Box Purchase Intent) Monadic Test with matched samples of 100 in the target market (Top 2 Box Purchase Intent) New Product Concepts Top Two Box Purchase Intention Results – Predictive Market 2* Respondent Base Sizes: Monadic = 100 per cell / Predictive Market *No Betting = 507 *** 16 43 J 45 49 I *** 35 54 H *** 28 64 G *** 28 64 F 67 67 UK Norms (top 2 box) 70 74 E 86 78 D 80 81 C + 76 83 B 85 85 A Significant Differences Predictive Market 2* With diverse group of 500 people - *no betting (Top 2 Box Purchase Intent) Monadic Test with matched samples of 100 in the target market (Top 2 Box Purchase Intent) New Product Concepts Top Two Box Purchase Intention Results – Predictive Market 2*

Predictive Market Experiment 2 - USA   *** 7 21 28 Snow Drapery 21 ** 18 20 40 Plug-in Lighting 20 * 27 19 45 Revitalight 19 + 62 11 45 Rise & Gently Shine 18 56 14 45 Sleep-ease 17 63 10 48 Bath & Spa Candle 16 45 16 50 Sky Scaper 15 * 32 17 52 Color Creator 14 64 9 52 Atmosphere Creator 13 48 15 54 Energizing Light 12 * 71 6 54 Permanent Outdoor Lighting System 11 ** 31 18 58 Light Stems 10 60 12 58 Cleaning Light 9 69 7 68 Voice Light 8 74 4 69 Solar Powered Holiday Lights 7 56 13 70 Two in One 6 71 5 72 Nightglow 5 76 3 74 Exit Alert 4 67 8 74 Garden Lights 3 78 2 76 Odor Eliminator 2 87 1 83 Fade Out 1 Monadic vs. Research Mkt Sig. Difference Predictive Market Def + Prob Buy Crowd of 500 Rank R.Mkt Monadic Def + Prob Buy 100 tgt per concept Concept Rank Mon.
R.Market; 7/7 top, + 1 with sig. difference
The same no. of av. ideas but much clearer on which ideas are weaker
Juicy Bits

Predictive Market Experiment 3 - UK x = Only tested in the 1 st screener, so no comparative data available   *** 61 36 150 61 89 52 Marmite Barrel Nuts + 46 52 150 69 93 47 *** Marmite Cheese Bites 45 62 150 69 96 40 *** McCain Marmite Wedges 70 80 150 80 92 62 ** Marmite Mini Toasts 129 84 150 79 89 63 ** Marmite Squeezy ** 49 41 150 64 91 49 *** Marmite Seasoning + 62 50 150 64 x x Marmite Soldiers 48 50 150 64 x x Marmite Mini Rice Snacks *** *** *** + Significance Screener vs. Monadic 11 x 26 9 x x x 63 84 x 95 81 x Marmite R.Market Def + Prob Buy 152 x 82 107 x x x 87 31 x 40 57 x Base Crowd 500 *** *** *** *** Significance Monadic vs. R.Market 150 51 95 22 Marmite Brunch Bar x x 92 28 Marmite Rusk Soldiers 150 55 89 31 McCain Marmite Home Fries 150 59 91 38 Marmite Cream Cheese x x 99 39 Marmite Buttery Spread x x 96 43 Marmite Squeezable Portion Pack x x 93 55 Marmite Quavers 150 64 88 57 Marmite Marbled Cheddar 150 72 97 60 Marmite Crackers x x 90 61 Marmite Hula Hoops 150 73 88 65 Marmite Bites 150 73 87 71 Marmite Spreadable x x 95 72 Marmite Walkers Base Tgt 150 Marmite Monadic Def + Prob Buy Base Tgt 87-95 Marmite Screener Def + Prob Buy
R.Market; 5/5 top, only 1 with sig. difference & clearer on weaker ideas
Screener; 2/5 top, 3 with sig. diff. / Av. ideas; Mon=7, Screener=5, R.M=3
Juicy Bits

Predictive Market Experiment 4 - Poland 55.3 53.6 73.3 R.Market (Sell / Buy most shares ) Def + Prob Buy 235 194 225 Base Crowd 500 *** *** *** Significance Monadic vs. R.Market ( All Shares ) 61.5 58.5 75.7 R.Market (Sell / Buy any shares ) Def + Prob Buy 366 313 341 Base Crowd 500 ** *** *** Significance Monadic vs. R.Market ( Any shares ) + 65 63.1 158 75 New Plus 91 69.2 150 73 New Sink & Pipe Unblocker 86.7 R.Market (Sell / Buy All Shares ) Def + Prob Buy 75 Base Crowd 500 Significance Monadic vs. R.Market (All Shares) 151 87 New Cling-Gel Base Tgt 150-158 Bases: Monadic Def + Prob Buy
In this experiment, only three concepts were tested. Therefore, to add a degree of sensitivity having asked which concept they’d buy / sell shares in, we asked how many shares they’d buy / sell (limited to 1000). The options were:
- 250 shares
- 500 shares
- 750 shares
- 1000 shares
Respondents who chose to buy / sell all 1,000 of their shares give the most accurate results when compared to the Bases Monadic Test.
Juicy Bits

Predictive Markets provides a wider spread in PI scores through aggregation of scores between buyers and sellers in a market place.
The PM scores rewards strong ideas and punishes weak concepts; making it easier, combined with other quant diagnostics and qualitative insights to identify progressive (highly potent and “on message” ideas).
Predictive Market Experiment 5 - Guatemala 99 92 92 92 89 Purchase Intention 22 29 29 38 49 56 65 69 83 Purchase Intention Monadic Predictive Markets 1 2 2 2 5 Rank 9 7 7 6 5 4 3 2 1 Rank *** 101 55 Be Light *** 101 97 7UP H2Oh! *** 100 100 Lipton Ice Tea 219 Pepsi Cappuccino 109 Flavour Splash 76 Juicy Punch *** 100 92 Propel *** 100 90 Tropicana Twister 166 7UP Bite Significant Difference Base Base

Possibilities for Market Research?

The Size of the Prize
Monadic Test of 10 ideas
Difficult to differentiate
Difficult to compare samples
Costly targeted samples
Costly monadic approach
Very time consuming process
Predictive Markets
Clarity on strong & weak ideas
All results are comparable
Cheap convenience sample
Cheaper ‘markets’ approach
Incredibly quick results

Predictive Markets & Experimental Results

by fred325i on Nov 23, 2006

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 11

Statistics

Predictive Markets & Experimental Results — Presentation Transcript