Online Information Aggregation Markets

Experimental Results of Online Information Aggregation Markets for Sales Forecasting Eric van Heck Rotterdam School of Management Erasmus University evanheck@rsm.nl www.rsm.nl/evanheck INSEAD Presentation Fontainebleau, 20 June 2008 © Eric van Heck, 2008.

Menu 1. What are information aggregation markets (or also called prediction markets)? 2. State-of-the-art in practice IOWA Political Markets Hollywood Stock Exchange 3. State-of-the-art in theory Project 1 with Mathijs van der Vlis 4. State-of-the-art in theory and practice Project 2 with Annie Yang et al. and anonymous company 5. Conclusions

Introduction How many passengers can travel with the Silja Symphony? Helsinki – Stockholm v.v.

Aggregation and Averaging Bo: 2,700 Heli: 2,640 Virpi: 3,005 Ari: 3,050 Helsinki – Stockholm v.v. Pekka: 2,502 Mika: 2,600 Jyrki: 2,845 Szymon: 2,777 Ralph: 2,799 Esko: 3,592

Aggregation and Averaging Bo: 2,700 Heli: 2,640 Virpi: 3,005 Ari: 3,050 Helsinki – Stockholm v.v. Pekka: 2,502 Mika: 2,600 Jyrki: 2,845 Szymon: 2,777 Ralph: 2,799 Esko: 3,592 Average: 2,851

Aggregation and Averaging Bo: 2,700 Heli: 2,640 Virpi: 3,005 Ari: 3,050 Helsinki – Stockholm v.v. Pekka: 2,502 Mika: 2,600 Jyrki: 2,845 Szymon: 2,777 Ralph: 2,799 Esko: 3,592 Average: 2,851 Correct answer: 2,852

Aggregation and Averaging Bo: 2,700 Heli: 2,640 Virpi: 3,005 Ari: 3,050 Helsinki – Stockholm v.v. Pekka: 2,502 Mika: 2,600 Jyrki: 2,845 Szymon: 2,777 Ralph: 2,799 Esko: 3,592 Average: 2,851 Correct answer: 2,852 The average is a very good predictor – wisdom of crowds. Jyrki is closest to the correct answer!

What are information markets? 1. A group of people that buy and sell stocks. 2. Stocks represent the potential outcome of the subject to be forecasted (number of Silja passengers, future demand of mobile telephones, winner soccer game, etc). 3. Market mechanism is a double auction. 4. Market price of a particular stock represent the probability that that potential outcome will happen – for example: stock Italy (in the game Italy – NL) is 0,80 cent (range 0 – 100 cents) = probability that Italy wins is 80%. 5. The market aggregates information by the aggregation of the individual beliefs of the players.

State-of-the-art in practice Some applications in practice: IOWA Political Markets Hollywood Stock Exchange Internal Information Markets for example by HP, Google, and external Information Markets such as NewsFutures, Foresight Exchange.

IOWA Political Markets

Lessons Learned (Berg et al, 1996, 2000) • IOWA political markets perform better than polling results • Presidential election markets perform better than (lower profile) congressional, state, or local elections • Markets with more volume near the election perform better • Markets with fewer contracts (i.e. fewer candidates or parties) predict better

Hollywood Stock Exchange

Trading in MovieStocks

Trading in StarBonds

Lessons Learned • Prices of securities in Oscar, Emmy, and Grammy awards correlate well with actual award outcome frequencies, and prices of movie stocks accurately predict real box office results (Pennock, 2001).

Hype Cycle for Emerging Technologies 2006

State-of-the-art in theory Market Characteristics Market Efficiency Incentive Mechanism Transaction Costs Trader Anonymity Market Information/Signals Prediction Metric (last trading price, avg price) Trading Mechanism Contract Type (binary, spread, index) Liquidity/Market Size Selling short/portfolios Information Cascades/Market Bubbles Frequency of information update Trader Characteristics TraderType Biases/Bounded Rationality Trader Demographics Characteristics of the to- be-predicted event Information Source Homogeneity/Heterogeneity Inherent Predictability Trading Experience/Knowledge Wealth Risk Attitude Aggregate Certainty Private Information Cheating/Collaboration/Manipulation Information Availability/Costs Time Scope

Main Theories • Mechanism Design Theory (Hayek 1945) Markets are an appropriate mechanism for the purpose of efficient information aggregation and decision making due to the incentives for information discovery. • Double Auction Theory (Plott and Sunder, 1982, 1988) Prediction markets have the ability to aggregate dispersed private information held by individuals as the double auction mechanism has the ability to disseminate private information among traders. • Rational Expectation Theory (Lucas 1972, Grossman 1981) The price observed in a prediction market is a sufficient statistic for all information available to traders • The Wisdom of Crowds (Surowiecki 2004) Small and large groups of people seem to do better at decision making than individuals.

Mechanism Design Theory

Project 1 - with Mathijs van der Vlis : What is the impact of the number of traders, the distribution of wisdom, and monetary incentives to the outcome of information markets?

Hypotheses 1. Number of traders (Surowiecki, 2004) More traders will increase the level of aggregation and the level of prediction accuracy 2. Distribution of wisdom (Anderson and Holt, 1997; Hansen, Smith, and Strober, 2001; Hanson and Oprea, 2004) Uneven distribution among traders will increase the level of aggregation and the level of prediction accuracy 3. Monetary incentives (Servan-Schreiber, Wolfers, Pennock, and Galebach, 2004) Monetary incentives will not increase the level of aggregation and the level of prediction accuracy

128 Laboratory Experiments to forecast future demand of mobile telephones

Results Experiments (N = 128)

Hypotheses 1. Number of traders (Surowiecki, 2004) More traders will increase the level of aggregation and level of No prediction accuracy Yes 2. Distribution of wisdom (Anderson and Holt, 1997; Hansen, Smith, and Strober, 2001; Hanson and Oprea, 2004) Uneven distribution among traders will increase the level of Yes aggregation and the level of prediction accuracy No 3. Monetary incentives (Servan-Schreiber, Wolfers, Pennock, and Galebach, 2004) Monetary incentives will not increase the level of aggregation Yes and the level of prediction accuracy Yes

Lessons Learned • Results indicate that even in the presence of a small number of traders there tends to be aggregation, while only in the presence of a large number of traders are accurate predictions generated. • When wisdom is unequally distributed there is aggregation (wise people lead markets), yet the markets do not produce more accurate predictions (wise people can potentially mislead markets). • Monetary incentives impact neither the level of aggregation nor the level of accuracy.

State-of-the-art in theory and practice Some applications: Internal Information Markets for example at a financial company Helsinki – Stockholm v.v.

Project 2 - with Annie Yang, Maarten Colijn, Willem Verbeke, Mathijs van der Vlis and anonymous company What is the performance of information markets in forecasting the overall sales of a product over several regions in the Netherlands?

Hypotheses • Market Size – Number of Traders (Surowiecki 2004, Hansen 2003) H1a: A prediction market with more traders is likely to aggregate sooner and more significantly. H1b: A prediction market with more traders is likely to forecast more accurately. • Monetary Incentives (Servan-Schreiber et al. 2004) H2: An offer of monetary incentives does not affect the activeness of traders’ participation in a prediction market. • Time Horizon (Berg et al. 2003) H3: A prediction market forecasts more accurately in a short run than in a long run.

Trading Web Page

1st Prediction Market 2nd Prediction Market Subject to be predicted Annual sales of a financial product Periodical sales of a financial product Contracts Spread contracts (in million euro) Spread contracts (in million euro) Description of Prediction Markets Description of traders Regional sales managers Regional sales managers Number of stocks 10 9 Number of traders 34 34 Number of active traders 31 18 Number of very active 8 3 traders Total number of bids (incl. 604 461 demand and sell) Total number of completed 368 275 bids (buy and sell) Time of markets 24 hrs / 7 days 24 hrs / 7 days Market duration 12 calendar days (Feb 2007) 12 calendar days (June 2007)

Aggregation and Forecasting Results Historical Stock Prices in 1st Prediction Market 80 110-120 Actual sales 70 121-130 i.e. 133 60 131-140 Stock Price (in point) 141-150 Market forecast 50 151-160 40 161-170 Top-down 171-180 30 forecast 181-190 20 191-200 201-210 10 0 Trading Day Trade 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th 11th 12th

Aggregation and Forecasting Results Historical Stock Prices in 2nd Prediction Market 120 Market forecast 19 - 22 100 22 - 25 Top-down 25 - 28 forecast Stock Price (in point) 80 28 - 31 31 - 34 Actual sales 60 34 - 37 i.e. 28.6 37 - 40 40 40 - 43 43 - 46 20 0 Trading Day Trade Day 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th 11th 12th

Forecasting Accuracy 1st Prediction Market 2nd Prediction Market % error % error Actual results 133 28.6 Prediction market forecast 141-150 +6% - 13% 19-22 -23% - 34% Top-down forecast 150-160 +13% - 20% 27 -6%

Hypotheses • Market Size – Number of Traders (Surowiecki 2004, Hansen 2003) H1a: A prediction market with more traders is likely to No aggregate sooner and more significantly. H1b: A prediction market with more traders is likely to forecast Yes more accurately. • Monetary Incentives (Servan-Schreiber et al. 2004) H2: An offer of monetary incentives does not affect the Yes activeness of traders’ participation in a prediction market. • Time Horizon (Berg et al. 2003) H3: A prediction market forecasts more accurately in a short No run than in a long run.

Lessons Learned 1. Market size, in terms of the number of traders, does not necessarily influence market aggregations but the accuracy of predictions. A thicker market is more likely to forecast accurately. 2. Monetary incentives are not effective to motivate traders to trade in internal prediction markets – time for trading is a constraint. 3. Markets predict more accurately in a long run than in a short run. Interesting because the impact of the worldwide mortgage crises was predicted very well 4. Traders are sensitive to the prices of contracts, learning from signals and constantly updating their beliefs. However, this yields that traders could be easily misled, particularly in a thin market.

Conclusions 1. “Information Aggregation” is a Key Critical Component for Firms - online markets can improve the information aggregation capability of a firm! Helsinki – Stockholm v.v. 2. Several issues need to be solved for example: details of the market design incentive structure of players 3. Do you want to know more: please join!

Online Information Aggregation Markets

by Eric van Heck on Nov 17, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

2 Embeds 64

Statistics

Online Information Aggregation Markets — Presentation Transcript

http://presentationsericvanheck.blogspot.com	63
http://www.slideshare.net	1