Predictive Markets & Experimental Results

Predictive Markets & the Wisdom of Crowds Experiments A Market Research Project by BrainJuicer ® Produced December 2005 The Mind Reading Agency

Extraordinary delusions and the Madness of Crowds, Charles MacKay (1841)

The Wisdom of Crowds – How the Many Can Be Smarter than the Few, James Surowiecki (2004)

Diversity

Independence

Faithful aggregation

Enables a crowd to be wise

Galton’s surprise findings. Collective Guess = 1197 lbs Actual Weight = 1198 lbs

How did the market know? Rockwell – Down 3% Lockheed – Down 3% Martin Marietta – Down 3% Morton Thiokol – Down 12%

Qu. Is there a more efficient alternative?

IEM - The original decision market X 596 Polls (1988 - 2000)  ¾

French – Referendum Non 53.6% Oui 46.4% Non 54.9% Oui 45.1% 29 th May – Betting BetFair 30 th May - Result (70% turn out) Non 51.0% Oui 49.0% 28 th May – Polls IPSOS

A random collection of 500 individuals will be as accurate as a targeted sample

Predictive Markets will be as accurate as monadics

The ‘crowd’ answering for the market will be as accurate as people answering for themselves

3 hypotheses to test on… Predictive Markets’ & Wisdom of Crowds

The Experiments Monadic Tests of 3, 10, 15 & 20 ideas, with matched targeted samples of 100 Predictive Markets of 3, 10, 15 & 20 ideas, with diverse ‘crowds’ of 500

Predictive Market 1 - UK  A diverse crowd will be as accurate as a targeted sample  A ‘Predictive Market’ will be as accurate as a monadic test  Answering for the market as accurate as answering for oneself Respondent Base Sizes: Monadic = 100 per cell / Predictive Market *No Betting = 507 *** 16 43 J 45 49 I *** 35 54 H *** 28 64 G *** 28 64 F 67 67 UK Norms (top 2 box) 70 74 E 86 78 D 80 81 C + 76 83 B 85 85 A Significant Differences Predictive Market 2* With diverse group of 500 people - *no betting (Top 2 Box Purchase Intent) Monadic Test with matched samples of 100 in the target market (Top 2 Box Purchase Intent) New Product Concepts Top Two Box Purchase Intention Results – Predictive Market 2* Respondent Base Sizes: Monadic = 100 per cell / Predictive Market *No Betting = 507 *** 16 43 J 45 49 I *** 35 54 H *** 28 64 G *** 28 64 F 67 67 UK Norms (top 2 box) 70 74 E 86 78 D 80 81 C + 76 83 B 85 85 A Significant Differences Predictive Market 2* With diverse group of 500 people - *no betting (Top 2 Box Purchase Intent) Monadic Test with matched samples of 100 in the target market (Top 2 Box Purchase Intent) New Product Concepts Top Two Box Purchase Intention Results – Predictive Market 2*

Predictive Market Experiment 2 - USA   *** 7 21 28 Snow Drapery 21 ** 18 20 40 Plug-in Lighting 20 * 27 19 45 Revitalight 19 + 62 11 45 Rise & Gently Shine 18 56 14 45 Sleep-ease 17 63 10 48 Bath & Spa Candle 16 45 16 50 Sky Scaper 15 * 32 17 52 Color Creator 14 64 9 52 Atmosphere Creator 13 48 15 54 Energizing Light 12 * 71 6 54 Permanent Outdoor Lighting System 11 ** 31 18 58 Light Stems 10 60 12 58 Cleaning Light 9 69 7 68 Voice Light 8 74 4 69 Solar Powered Holiday Lights 7 56 13 70 Two in One 6 71 5 72 Nightglow 5 76 3 74 Exit Alert 4 67 8 74 Garden Lights 3 78 2 76 Odor Eliminator 2 87 1 83 Fade Out 1 Monadic vs. Research Mkt Sig. Difference Predictive Market Def + Prob Buy Crowd of 500 Rank R.Mkt Monadic Def + Prob Buy 100 tgt per concept Concept Rank Mon.

R.Market; 7/7 top, + 1 with sig. difference

The same no. of av. ideas but much clearer on which ideas are weaker

Juicy Bits

Predictive Market Experiment 3 - UK x = Only tested in the 1 st screener, so no comparative data available   *** 61 36 150 61 89 52 Marmite Barrel Nuts + 46 52 150 69 93 47 *** Marmite Cheese Bites 45 62 150 69 96 40 *** McCain Marmite Wedges 70 80 150 80 92 62 ** Marmite Mini Toasts 129 84 150 79 89 63 ** Marmite Squeezy ** 49 41 150 64 91 49 *** Marmite Seasoning + 62 50 150 64 x x Marmite Soldiers 48 50 150 64 x x Marmite Mini Rice Snacks *** *** *** + Significance Screener vs. Monadic 11 x 26 9 x x x 63 84 x 95 81 x Marmite R.Market Def + Prob Buy 152 x 82 107 x x x 87 31 x 40 57 x Base Crowd 500 *** *** *** *** Significance Monadic vs. R.Market 150 51 95 22 Marmite Brunch Bar x x 92 28 Marmite Rusk Soldiers 150 55 89 31 McCain Marmite Home Fries 150 59 91 38 Marmite Cream Cheese x x 99 39 Marmite Buttery Spread x x 96 43 Marmite Squeezable Portion Pack x x 93 55 Marmite Quavers 150 64 88 57 Marmite Marbled Cheddar 150 72 97 60 Marmite Crackers x x 90 61 Marmite Hula Hoops 150 73 88 65 Marmite Bites 150 73 87 71 Marmite Spreadable x x 95 72 Marmite Walkers Base Tgt 150 Marmite Monadic Def + Prob Buy Base Tgt 87-95 Marmite Screener Def + Prob Buy

R.Market; 5/5 top, only 1 with sig. difference & clearer on weaker ideas

Screener; 2/5 top, 3 with sig. diff. / Av. ideas; Mon=7, Screener=5, R.M=3

Juicy Bits

Predictive Market Experiment 4 - Poland 55.3 53.6 73.3 R.Market (Sell / Buy most shares ) Def + Prob Buy 235 194 225 Base Crowd 500 *** *** *** Significance Monadic vs. R.Market ( All Shares ) 61.5 58.5 75.7 R.Market (Sell / Buy any shares ) Def + Prob Buy 366 313 341 Base Crowd 500 ** *** *** Significance Monadic vs. R.Market ( Any shares ) + 65 63.1 158 75 New Plus 91 69.2 150 73 New Sink & Pipe Unblocker 86.7 R.Market (Sell / Buy All Shares ) Def + Prob Buy 75 Base Crowd 500 Significance Monadic vs. R.Market (All Shares) 151 87 New Cling-Gel Base Tgt 150-158 Bases: Monadic Def + Prob Buy

In this experiment, only three concepts were tested. Therefore, to add a degree of sensitivity having asked which concept they’d buy / sell shares in, we asked how many shares they’d buy / sell (limited to 1000). The options were:

- 250 shares

- 500 shares

- 750 shares

- 1000 shares

Respondents who chose to buy / sell all 1,000 of their shares give the most accurate results when compared to the Bases Monadic Test.

Juicy Bits

Predictive Markets provides a wider spread in PI scores through aggregation of scores between buyers and sellers in a market place.

The PM scores rewards strong ideas and punishes weak concepts; making it easier, combined with other quant diagnostics and qualitative insights to identify progressive (highly potent and “on message” ideas).

Predictive Market Experiment 5 - Guatemala 99 92 92 92 89 Purchase Intention 22 29 29 38 49 56 65 69 83 Purchase Intention Monadic Predictive Markets 1 2 2 2 5 Rank 9 7 7 6 5 4 3 2 1 Rank *** 101 55 Be Light *** 101 97 7UP H2Oh! *** 100 100 Lipton Ice Tea 219 Pepsi Cappuccino 109 Flavour Splash 76 Juicy Punch *** 100 92 Propel *** 100 90 Tropicana Twister 166 7UP Bite Significant Difference Base Base