Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Norris Clark PM Cluster SFO 2009


Published on

Published in: Business
  • Hi there! Get Your Professional Job-Winning Resume Here - Check our website!
    Are you sure you want to  Yes  No
    Your message goes here

Norris Clark PM Cluster SFO 2009

  1. 1. Collective Intelligence @ Work Stories from the Field Norris Clark
  2. 2. Thanks to John and Sarah!
  3. 3. Leading companies have adopted prediction markets to improve forecasting and innovation ” “ 2000 2004 2008 Leaders deprive themselves of information that could enrich their analysis and reduce the risk of ivory tower decision making. Some executives understand that valuable information lies scattered around the organization but don’t know how to retrieve it. Prediction markets might solve these problems. The McKinsey Quarterly, April 2008
  4. 4. Workers, place your bets! <ul><li>Prediction markets let employees bet on various corporate KPIs, sales forecasts, the success of new products, etc. </li></ul><ul><li>This process aggregates the collective intelligence of entire pools of selected employees to generate better forecasts than polls, delphi, or individual experts. </li></ul><ul><li>The «  wisdom of crowds  » principle: None of us is as smart as all of us. </li></ul>(c) New York Times, Betting to improve the odds , April 9 ,2008
  5. 5. Main Business Applications <ul><li>Forecasting Sales, KPIs... Selecting New Ideas </li></ul><ul><li>Measuring Risks Monitoring Project Delivery </li></ul>Decision Markets
  6. 6. NewsFutures has worked for leading companies and media groups around the world 2002 2003 2004 2005 2006 2007 Newsfutures invents Competitive Forecasting for Arcelor and Mars Yahoo! buys NewsFutures prediction market software Launch of Global Risks Prediction Market at Davos 2006 U.S. Military adopts NewsFutures technology NewsFutures invents Idea Pageant for managing innovation 1 2 3 4 5 1 2 3 4 5
  7. 7. <ul><li>1) The Power of Diversity </li></ul><ul><ul><ul><li>Everyone contributes biased knowledge; </li></ul></ul></ul><ul><ul><ul><li>Knowledge adds up; </li></ul></ul></ul><ul><ul><ul><li>Biases cancel each other. </li></ul></ul></ul><ul><li>2) Rewarding timely and truthful revelation </li></ul><ul><li>of information </li></ul><ul><ul><ul><li>It’s not enough to be right ; </li></ul></ul></ul><ul><ul><ul><li>You must be right before others . </li></ul></ul></ul><ul><ul><ul><li>Encouraging contrarian opinions rather than conformity </li></ul></ul></ul>What makes market predictions accurate? <ul><li>« A true Englishman doesn’t joke when he is talking about so serious a thing as a wager. » – Phileas Fogg </li></ul><ul><li>3) Leveraging more brain power </li></ul><ul><ul><ul><li>When betting, the brain recruits specific neural modules that are specialized for evaluating risks and rewards. </li></ul></ul></ul><ul><ul><ul><li>The brain thinks harder when it is making bets. </li></ul></ul></ul>
  8. 8. Some Challenges to Enterprise Markets <ul><li>Perceived threat to traditional management roles -- Accenture </li></ul><ul><li>Changing the way things are usually done – American Insurance </li></ul><ul><li>Feeling that some questions are better left unanswered </li></ul><ul><li>Feeling that some questions are better left unasked </li></ul><ul><li>Decision makers that do not really care about forecasting - DMX </li></ul><ul><li>Engaging busy professionals - Dow </li></ul>
  9. 9. Choosing the right tool(s) for the task <ul><li>Prediction Markets . For measuring risks and opportunities as event probabilities. It can also be used to forecast continuous variables.   </li></ul><ul><li>Competitive Forecasting . For forecasting specific business variables, such as quarterly sales, project completion dates, KPIs, etc, especially under several conditional scenarios, or with a small pool of senior-level participants.   </li></ul><ul><li>Idea Pageant . For harvesting and filtering a large number of new ideas that require evaluation and prioritization.   </li></ul><ul><li>Impact Matrix . To gauge the likelihood and business impact of various trends in the long term. </li></ul>
  10. 10. Design: More than Software <ul><li>I nformation </li></ul><ul><ul><ul><li>What variable or event should we be forecasting? </li></ul></ul></ul><ul><ul><ul><li>What existing data might inform or bias the participants? </li></ul></ul></ul><ul><li>I ntegration </li></ul><ul><ul><ul><li>Which business process should the market be part of? </li></ul></ul></ul><ul><ul><ul><li>How is the market coordinated with related business processes (eg, other forecasts)? </li></ul></ul></ul><ul><li>I nclusion </li></ul><ul><ul><ul><li>How many should participate? </li></ul></ul></ul><ul><ul><ul><li>How knowledgeable should participants be? </li></ul></ul></ul><ul><li>I nterface </li></ul><ul><ul><ul><li>How is knowledge translated into data? </li></ul></ul></ul><ul><ul><ul><li>How is this data aggregated into a forecast? </li></ul></ul></ul><ul><li>I ncentives </li></ul><ul><ul><ul><li>How to reward participation and performance? (see next slide) </li></ul></ul></ul><ul><ul><ul><li>How to avoid conflicts of interest? </li></ul></ul></ul>
  11. 11. Incentives Calibration The 4 best r easons to participate R ewards : What can I win? R elevance : Are the questions relevant to my job? Am I knowledgeable? R ecognition : Will my performance be recognized by my bosses and peers? R elationships : Does the market help me interact meaningfully with bosses and peers?
  12. 12. Keys to Success <ul><li>Strong commitment from management </li></ul><ul><ul><ul><ul><li>Market participation is part of everyone’s job description </li></ul></ul></ul></ul><ul><li>Important questions </li></ul><ul><ul><ul><ul><li>Don’t play it safe – seek the truth </li></ul></ul></ul></ul><ul><li>Actionable results </li></ul><ul><ul><ul><ul><li>Input to actual decisions </li></ul></ul></ul></ul><ul><li>Appropriate rewards </li></ul><ul><ul><ul><ul><li>Recognize performance – this is not just a game </li></ul></ul></ul></ul><ul><li>Engaging tool </li></ul><ul><ul><ul><ul><li>Minimize demand on brain power – choose the right tool/platform for the job </li></ul></ul></ul></ul>
  13. 13. Customer Praise <ul><li>«I selected NewsFutures after a comprehensive and competitive RFP process. The team is very customer-oriented and is able to listen to me and translate my thoughts and requests into tangible results. They are very responsive; they have consistently flexed and adapted to any new parameters and ideas that I may bring to them.» - Kristin Eilenberg - Manager, Global Customer Solutions, Eli Lilly </li></ul><ul><li>«The study team was impressed by NewsFutures' different designs ( Prediction Market and Competitive Forecasting ) and its Web content and display. The pilot program benefited by employing NewsFutures, the largest and most established organization currently hosting prediction markets for commercial clients.» - From a report by the RAND Corporation </li></ul><ul><li>«NewsFutrures' competitive forecasts provide a valuable input in defining the commercial policy in the coming quarters.» - Javier Llera Fueyo, Senior Manager of Market Intelligence, Arcelor Mittal </li></ul><ul><li>«Compared to polling people, NewsFutures' Idea Pageant significantly improves the process of harvesting and prioritizing ideas.» - Zubin Dowlaty, vice president of emerging technologies, InterContinental Hotels Group </li></ul><ul><li>«NewsFutures' Impact Matrix tool proved very useful to my team.» - Cathy Middaugh - Manager, Corporate Strategic Planning, Eli Lilly </li></ul><ul><li>«In the case of Corning , the exercise was well thought out [...] The market had very good accuracy.» - From a report by Forrester Research </li></ul>
  14. 14. ” “ NewsFutures has done some of the most innovative work in the area for longer than most. They were in there making it happen for real while others were clueless on the fringes. Ajit Kambil Gobal Director, Deloitte Research
  15. 15. Corning <ul><li>Forrester Research: While the market had very good accuracy, the metrics that Corning chose to examine were ultimately too far removed from Corning’s core business to be of value. </li></ul>Application: Forecasting sales of LCD TVs Case Study
  16. 16. Forecasting Product Prices
  17. 17. Company : Intercontinental Hotels Date : Sept 2007 - … Duration : 4 weeks Application : Innovation Design : Idea Pageant Participants : Emerging Technlogies Dept. # participants : 200 ” “ We wanted to tap the creative class that may not be able to voice their ideas Zubin Dowlaty, VP Emerging Technologies, Intercontinental Hotels – NYT, 09/04/08 InterContinental Hotels Group was looking for new ideas to improve customer experience. Case Study
  18. 18. ” “ <ul><li>Process </li></ul><ul><li>All employees generate and describe new ideas </li></ul><ul><li>Administrator enters new ideas into the pageant </li></ul><ul><li>All employees bet on which ideas will get the green light, which will be rejected </li></ul><ul><li>This quickly creates a ranking of the ideas </li></ul><ul><li>Top-ranked ideas are taken to decision makers </li></ul><ul><li>Decision makers choose which ideas get the green light </li></ul><ul><li>Those who invented the winning ideas, and those who bet on them, are both rewarded. </li></ul>Case Study We had a top time! I had the feeling it was the first time I could really show my creativity. Idea Pageant participant
  19. 19. Case Study Describing a new idea
  20. 20. Case Study Describing a new idea
  21. 21. Case Study Betting on the ideas
  22. 22. “ <ul><li>More than 200 people participated, submitting 85 ideas </li></ul><ul><li>2 projects have been implemented as a result, one of which is so innovative that Google is writing a case study about it (it involves Google Maps) </li></ul><ul><li>Authors of winning ideas received great recognition </li></ul><ul><li>3 best bettors were also recognized: $500 for 1st, $250 each for 2nd and 3rd </li></ul><ul><li>Winners were engineers, analysts, contractors, not managers </li></ul>Results ” Compared to polling people, the Idea Pageant significantly improves the process of harvesting and prioritizing ideas. Zubin Dowlaty, VP Emerging Technologies, Intercontinental Hotels – NYT, 09/04/08 Case Study
  23. 23. Partnering with the New England Consulting Group Client: A large ($3B) off-web info company in disrupted market. Goal: New CEO wants actionable ideas to transform business Design: Three phase process using various tools Duration: 1 week, 92 participants Phase One – 644 proposals in seven categories Phase Two - 371 moved to Idea Pageant Phase Three - 120 Ideas evaluated by senior management “ There were many, many compliments on the software. Very easy to use, intuitive, and extremely useful. It would have taken us weeks or months to get this much information otherwise. Very successful project.” - EVP in charge of project Case Study
  24. 24. <ul><li>Collaboration between MarketTools & NewsFutures </li></ul><ul><li>Client: Director of Innovation </li></ul><ul><li>Year-long project </li></ul><ul><li>NewsFutures application: Predicting the success of various product platforms being considered by an advisory board. </li></ul><ul><li>Conditional predictions – NewsFutures ’ Idea Pageant technology </li></ul><ul><li>Panel: two dozen FritoLay executives at first, then hundreds of employees. </li></ul>Fostering Innovation