• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Online Scenario Planning
 

Online Scenario Planning

on

  • 5,004 views

This is the presentation of my research I recently gave at the MIT Department of Urban Studies and Planning, to members of the MIT Center for Collective Intelligence and others. ...

This is the presentation of my research I recently gave at the MIT Department of Urban Studies and Planning, to members of the MIT Center for Collective Intelligence and others.

It covers the basic principles of scenario planning, of crowdsourcing and collective intelligence, and then proposes a way to bring them together into an effective online system for futures work.

Statistics

Views

Total Views
5,004
Views on SlideShare
4,429
Embed Views
575

Actions

Likes
9
Downloads
313
Comments
0

6 Embeds 575

http://news.noahraford.com 505
http://noahraford.com 48
http://www.slideshare.net 16
http://futuramb.tumblr.com 4
https://lms.kku.edu.sa 1
https://twitter.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

Online Scenario Planning Online Scenario Planning Presentation Transcript

  • Large-Scale Participatory Futures Systems Harnessing Collective Intelligence for Crowdsourced Scenario Planning Noah Raford PhD Candidate, UIS/CDD Massachusetts Institute of Technology (MIT) Department of Urban Studies and Planning nraford@mit.edu http://news.noahraford.com/ Friday, April 9, 2010 Wednesday, 21 April 2010
  • Purpose An informal crowdsourcing charrette to harness your collective intelligence and thereby scenario plan my PhD. Wednesday, 21 April 2010
  • Purpose An informal crowdsourcing charrette to harness your collective intelligence and thereby scenario plan my PhD. Me Wednesday, 21 April 2010
  • Purpose An informal crowdsourcing charrette to harness your collective intelligence and thereby scenario plan my PhD. Me Your brains Wednesday, 21 April 2010
  • Purpose An informal crowdsourcing charrette to harness your collective intelligence and thereby scenario plan my PhD. Ideas Feedback Me Your brains Wednesday, 21 April 2010
  • Purpose An informal crowdsourcing charrette to harness your collective intelligence and thereby scenario plan my PhD. Ideas Ph D Feedback Me Your brains Wednesday, 21 April 2010
  • Outline Wednesday, 21 April 2010
  • Outline 1. De nitions & overview Wednesday, 21 April 2010
  • Outline 1. De nitions & overview 2. Research questions Wednesday, 21 April 2010
  • Outline 1. De nitions & overview 2. Research questions 3. An example, “The Future of Cities” Wednesday, 21 April 2010
  • Outline 1. De nitions & overview 2. Research questions 3. An example, “The Future of Cities” 4. Theoretical foundations Wednesday, 21 April 2010
  • Outline 1. De nitions & overview 2. Research questions 3. An example, “The Future of Cities” 4. Theoretical foundations 5. Schema for an online approach Wednesday, 21 April 2010
  • Outline 1. De nitions & overview 2. Research questions 3. An example, “The Future of Cities” 4. Theoretical foundations 5. Schema for an online approach 6. Case studies Wednesday, 21 April 2010
  • Outline 1. De nitions & overview 2. Research questions 3. An example, “The Future of Cities” 4. Theoretical foundations 5. Schema for an online approach 6. Case studies 7. Hypotheses Wednesday, 21 April 2010
  • Outline 1. De nitions & overview 2. Research questions 3. An example, “The Future of Cities” 4. Theoretical foundations 5. Schema for an online approach 6. Case studies 7. Hypotheses 8. Evaluation criteria Wednesday, 21 April 2010
  • Outline 1. De nitions & overview 2. Research questions 3. An example, “The Future of Cities” 4. Theoretical foundations 5. Schema for an online approach 6. Case studies 7. Hypotheses 8. Evaluation criteria 9. Limitations & future work Wednesday, 21 April 2010
  • De nitions & overview Collective Intelligence for Crowdsourced Scenario Planning Wednesday, 21 April 2010
  • De nitions & overview Collective Intelligence for Crowdsourced Scenario Planning “Creation, aggregation and interpretation of strategically relevant information for decision-making [through distributed means]” (Por, 2008) Wednesday, 21 April 2010
  • De nitions & overview Collective Intelligence for Crowdsourced Scenario Planning “Creation, aggregation and interpretation of strategically relevant information for decision-making [through distributed means]” (Por, 2008) “The act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call.” (Howe, 2006) Wednesday, 21 April 2010
  • De nitions & overview Collective Intelligence for Crowdsourced Scenario Planning “Creation, aggregation and interpretation of strategically relevant information for decision-making [through distributed means]” (Por, 2008) “The act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call.” (Howe, 2006) “Tools for foresight discussions... whose purpose is not a prediction or a plan, but a change in the mindset of the people who use them.” (de Gues, 1997) Wednesday, 21 April 2010
  • De nitions & overview Collective Intelligence for Crowdsourced Scenario Planning “Creation, aggregation and interpretation of strategically relevant information for decision-making [through distributed means]” (Por, 2008) “The act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call.” (Howe, 2006) “Tools for foresight discussions... whose purpose is not a prediction or a plan, but a change in the mindset of the people who use them.” (de Gues, 1997) Wednesday, 21 April 2010
  • De nitions & overview Collective Intelligence for Crowdsourced Scenario Planning “Creation, aggregation and interpretation of strategically relevant information for decision-making [through distributed means]” (Por, 2008) “The act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call.” (Howe, 2006) “Tools for foresight discussions... whose purpose is not a prediction or a plan, but a change in the mindset of the people who use them.” (de Geus, 1997) Gues, Wednesday, 21 April 2010
  • Research questions Wednesday, 21 April 2010
  • Research questions Can the bene ts of face-to-face scenario planning (SP) be had using online, collective intelligence approaches? Wednesday, 21 April 2010
  • Research questions Can the bene ts of face-to-face scenario planning (SP) be had using online, collective intelligence approaches? Wednesday, 21 April 2010
  • Research questions Can the bene ts of face-to-face scenario planning (SP) be had using online, collective intelligence approaches? What do online approaches add to traditional SP and what do they take away? Wednesday, 21 April 2010
  • Research questions Can the bene ts of face-to-face scenario planning (SP) be had using online, collective intelligence approaches? What do online approaches add to traditional SP and what do they take away? Wednesday, 21 April 2010
  • Research questions Can the bene ts of face-to-face scenario planning (SP) be had using online, collective intelligence approaches? What do online approaches add to traditional SP and what do they take away? Which aspects of online, Web 2.0 approaches are most in uential to the stated goals of SP, in what way and why? Wednesday, 21 April 2010
  • Research questions Can the bene ts of face-to-face scenario planning (SP) be had using online, collective intelligence approaches? What do online approaches add to traditional SP and what do they take away? Which aspects of online, Web 2.0 approaches are most in uential to the stated goals of SP, in what way and why? Wednesday, 21 April 2010
  • Research questions Can the bene ts of face-to-face scenario planning (SP) be had using online, collective intelligence approaches? What do online approaches add to traditional SP and what do they take away? Which aspects of online, Web 2.0 approaches are most in uential to the stated goals of SP, in what way and why? Can the design & testing of such a system provide more rigourous data for understanding the effects of SP on group process in general? Wednesday, 21 April 2010
  • Typical scenario method* * i.e., The most common, Shell-style, Client defines key questions through initial Meetings, “inductive logics” approach, distinct from a ID Issues range of other rich approaches including conversations & meetings conversations Causal Layered Analysis (Inayatullah, 2004), La Prospective (Berger, 1964), the Generate Expert interviews, brainstorm with client, F2F & phone Manoa Method (Schultz, 1994), or the key themes desktop research interviews Three Horizons framework (Hodgson & Curry, 2008). ID driving Extract key themes, create trends and Group forces timelines, key events workshop Select key uncertainties and forces, list by Rank factors uncertainty / impact, predetermined drivers Develop draft Create scenario snippets, draft systems scenario logic diagrams, mix and match trends, 2x2 grids Create draft Integrate themes from draft scenarios, create Consultant final scenarios headlines and scenario narratives report Finalise Get client feedback, refine, detail, elaborate Group scenarios narrative to final form workshop Identify key strategic themes, reflect on Consider strategic questions in the context of each implications scenario Identify ID key indicators in each scenario for Consultant (Raford, 2010 after indicators strategic concerns report Schwartz, 1991; van der Heijden, 1997) Wednesday, 21 April 2010
  • Example, “Future of Cities” Wednesday, 21 April 2010
  • Example, “Future of Cities” 23 interviews world wide: • Architecture • Commercial tenants • Entrepreneurship • Environment • Governance • Infrastructure • Non-governmental organisations • Planning • Real estate • Technology Wednesday, 21 April 2010
  • Political • Most city governments will lack the resources to meet increasing citizens demands • Weaken central government and open room for other players • Civil society and community based organisations will be the rst to ll this role • Bottom-up participatory approaches to development and management will become important • Local government will need to shift from regulation to enabling and facilitating • Boundaries of where city authorities ends will blur, administrative implications are unclear • Grassroots' innovation could lead to transformational change • Insecurity more important factor, with high unemployment and economic, political and environmental migration Wednesday, 21 April 2010
  • Economic • Increased division of wealth between rich and poor • The world’s poor are an increasingly powerful force in urban development • Current urban development models not t for their emergent needs • Global warming will disproportionately effect the poor • International nance will become more important, domestic capital less • International nance will become more selective, comparing between cities • Taxation will continue to be a strong determinant of capital ows • New ecological accounting mechanisms will play an increasing role in real estate nance and development • Building obsolescence will become an increasingly important factor Wednesday, 21 April 2010
  • Technological • ICT will enable acceleration of social dynamics already in place • Could have signi cant destabilising effects through asymmetric warfare, etc. • May allow for breakthroughs in decentralised infrastructure and governance • ICT enables relocation of activities, such as public administration • ICT will enable more social surveillance and government control Wednesday, 21 April 2010
  • Drivers of change Growing income inequality Lack of capital availability Role of centralized governance Infrastructure decay Lifestyle change & value shifts Resource shortages Wednesday, 21 April 2010
  • Ranking key drivers Wednesday, 21 April 2010
  • Wednesday, 21 April 2010
  • Gulliver’s World Wednesday, 21 April 2010
  • Gulliver’s Massive socio- World technical revolution Wednesday, 21 April 2010
  • Gulliver’s Massive socio- Triumph of the World technical revolution Triads Wednesday, 21 April 2010
  • Scenario 1: Gulliver’s World Wednesday, 21 April 2010
  • Scenario 2: Massive socio-technical revolution... Or bust Wednesday, 21 April 2010
  • Scenario 3: Triumph of the Triad’s Wednesday, 21 April 2010
  • Components of scenarios Contextual Focused on the uncertainty & discontinuities over which we have limited or no control Narrative, with actors, motives, chronology & events control Materially different along key strategic dimensions some control Primarily qualitative but often with quantitative support no control Use rich media & diverse presentation styles (van der Heijden, 1997) Wednesday, 21 April 2010
  • Components of scenarios Contextual Focused on the uncertainty & discontinuities over which we have limited or no control Narrative, with actors, motives, chronology & events control Materially different along key strategic dimensions some control Primarily qualitative but often with quantitative support no control Use rich media & diverse presentation styles (van der Heijden, 1997) Wednesday, 21 April 2010
  • Theoretical foundations Studies of group decision-making reveal important shortcomings which limit our ability to make effective decisions under conditions of dynamic uncertainty (Dorner, 1997). These include the “availability bias”, whereby people estimate the future probability of events based on easily remembered experiences from their past (Tversky & Kahneman, 1974) “Experimenter bias”, whereby people look for and select data that con rms pre- existing expectations (Rosenthal, 1966) “Ambiguity effect”, whereby subjects are ignored or discounted for which we have partial or incomplete information (Frisch & Baron, 1988) “Groupthink biases”, whereby groups seek to minimize con ict and reach consensus without critically testing, analyzing, or evaluating ideas (Janis, 1972). Wednesday, 21 April 2010
  • Theoretical foundations As a result we build operational theories of the way the world works based on past experience and data, then are reluctant to revise them in the face of change. Wednesday, 21 April 2010
  • Theoretical foundations Wednesday, 21 April 2010
  • Theoretical foundations "We will not have any more crashes in our time." John Maynard Keynes, 1927 Wednesday, 21 April 2010
  • Theoretical foundations "There will be no interruption of our permanent prosperity." Myron E. Forbes, President, Pierce Arrow Motor Car Co., January 12, 1928 Wednesday, 21 April 2010
  • Theoretical foundations "Stock prices have reached what looks like a permanently high plateau.” Irving Fisher, Ph.D, Economist, Oct. 17, 1929 Wednesday, 21 April 2010
  • Theoretical foundations "This crash is not going to have much effect on business." Arthur Reynolds, Chairman of Continental Illinois Bank of Chicago, October 24, 1929 Wednesday, 21 April 2010
  • Theoretical foundations "... the present depression has about spent its force..." HES, Aug 30, 1930 Wednesday, 21 April 2010
  • Theoretical foundations "Stabilization at [present] levels is clearly possible." HES Oct 31, 1931 Wednesday, 21 April 2010
  • Theoretical foundations "All safe deposit boxes in banks or financial institutions have been sealed... and may only be opened in the presence of an agent of the I.R.S." President F.D. Roosevelt, 1933 Wednesday, 21 April 2010
  • Theoretical foundations Scenarios focus on decision-makers’ internal landscape “A company’s perception of its business environment is as important as its investment infrastructure because its strategy comes from this perception. I cannot over-emphasize this point: unless the corporate microcosm changes, managerial behavior will not change; the internal compass must be re-calibrated.” (Wack, in Chermack, 2003) Wednesday, 21 April 2010
  • Theoretical foundations They produce learning & strategic awareness Scenarios are a game board of the future, designed to break down people’s perceptions of the present & their assumptions of the future, in a way that allows them to better understand changing contexts and see new insights & opportunities, today. “Leading from the future, as it emerges in the present” Scharmer, 2009 Wednesday, 21 April 2010
  • Theoretical foundations Proposition 1: If scenarios are positively associated Scenarios with learning, then learning will increase as a result of participation in scenario planning. Proposition 2: If learning is positively associated with Increased the alteration of mental models, then mental models learning change as a result of learning. Proposition 3: If a chance in More accurate mental models alters decision mental models structure, then a change in mental models implies a change in the approach to decision making. Better decisions Proposition 4: If changes in decision making are positively associated with Proposition 5: If scenarios are positively rm performance, then rm associated with learning, learning is positively Improved performance will increase associated with altered mental models, altered performance as a result of altered mental models are positively associated with decision making strategies. rm performance, then scenarios can be positively associated with rm performance. (Chermack, 2003) Wednesday, 21 April 2010
  • Theoretical foundations How does SP do this? Includes diverse, contrary and non-traditional viewpoints and evidence Focuses explicitly on “game-changing” events and trends Uses creative workshops & methods to understand organizational strategy making, “get inside”, then creatively disrupt it (Innes & Booher, 2001) Wednesday, 21 April 2010
  • Theoretical foundations Why does it work? A focuses on organizantional learning, sensemaking and decision-making: Argys & Schon (1974) Double loop organizational learning Piaget (1977) Constructivist & social learning theory Weick (1979) Sensemaking & organizational awareness Klein (1999) Recognition-primed decision making Jarzabkowski (2005), Orlikowski (1992) Activity- & practice-based strategizing Boyd (1976) Competitive advantages of perception management Wednesday, 21 April 2010
  • Theoretical foundations Why does it work? Wednesday, 21 April 2010
  • Theoretical foundations Why does it work? Wednesday, 21 April 2010
  • Theoretical foundations Why does it work? Wednesday, 21 April 2010
  • Theoretical foundations Collaborative sensemaking in the public realm “Urban planning has lost sight of the future... creating increasingly feeble, myopic, degenerate frameworks that are more likely to react to yesterday's events than to prepare the way from here to the future.” (Isserman, 1985) Innes & Booher (1999) - Critique of public participation Healy (2001) - Role and need for community visioning Wednesday, 21 April 2010
  • Theoretical foundations Collaborative sensemaking in the public realm “Scenarios are developed collectively to build shared images of possible futures… scenarios nurture openness to change by allowing more complexity in futures states of a system and environment to be taken into account.” (Van der Heijden, 1997) Wednesday, 21 April 2010
  • Schema for online approach Wednesday, 21 April 2010
  • Schema for online approach Current challenges (Raford, 2010; Pang, 2010) Wednesday, 21 April 2010
  • Schema for online approach Current challenges (Raford, 2010; Pang, 2010) Labor intensive & expensive Wednesday, 21 April 2010
  • Schema for online approach Current challenges (Raford, 2010; Pang, 2010) Labor intensive & expensive Bene ts poorly documented (no veri cation or reputation systems) Wednesday, 21 April 2010
  • Schema for online approach Current challenges (Raford, 2010; Pang, 2010) Labor intensive & expensive Bene ts poorly documented (no veri cation or reputation systems) Limited participation (time, space & numbers) Wednesday, 21 April 2010
  • Schema for online approach Current challenges (Raford, 2010; Pang, 2010) Labor intensive & expensive Bene ts poorly documented (no veri cation or reputation systems) Limited participation (time, space & numbers) Predominance of senior decision-making elite (participant bias) Wednesday, 21 April 2010
  • Schema for online approach Current challenges (Raford, 2010; Pang, 2010) Labor intensive & expensive Bene ts poorly documented (no veri cation or reputation systems) Limited participation (time, space & numbers) Predominance of senior decision-making elite (participant bias) Highly dependent on facilitation skills & consultant synthesis (facilitator & author bias) Wednesday, 21 April 2010
  • Schema for online approach (Malone et al., 2010) Wednesday, 21 April 2010
  • Schema for online approach Who is performing the task? Why are they doing it? (Malone et al., 2010) Wednesday, 21 April 2010
  • Schema for online approach Who is performing the task? Why are they doing it? What is being accomplished? How is it being done? (Malone et al., 2010) Wednesday, 21 April 2010
  • Schema for online approach (Malone et al., 2010) Wednesday, 21 April 2010
  • Schema for online approach The problem Can activities be divided into pieces? Are necessary resources widely distributed or in unknown locations? Yes No Crowd Hiearchy Are there adequate incentives to participate? Direct Influence / Love, Learning compenstion self-promotion friendship (Raford, 2010, after Yes Malone et al., 2010) Wednesday, 21 April 2010
  • Yes Schema for online approach What kind of activity needs to be done? Create Decide Can the activity be divided into Does the entire group need to small, independent pieces? abide by the same decision? Yes No No Collect Collaborate Individual decisions Mechanisms for collaboration - wikis - blogs - discussion forums Are money or resources required to Are only a few good (best) - collaborative solutions needed? exchange hands or motivate workspaces, etc. decision? Voting Averaging No Yes Yes No (Raford, 2010, after Market Trust Malone et al., 2010) Finished exchange networks Wednesday, 21 April 2010
  • What kind of activity needs to be Schema for online approach done? Create Can the activity be divided into small, independent pieces? Yes No No Collect Collaborate Individual decisions Mechanisms for collaboration - wikis - blogs - discussion forums Are money or resources required to Are only a few good (best) - collaborative solutions needed? exchange hands or motivate workspaces, etc. decision? No Yes Yes No Market Trust Finished exchange networks (Raford, 2010, after Malone et al., 2010) Finished Finished Wednesday, 21 April 2010
  • Schema for online approach Decide Does the entire group need to abide by the same decision? No Yes Individual Group decisions decision r ums Are money or resources required to exchange hands or motivate c. Prediction decision? Voting Averaging Concensus markets Yes No Finished Market Trust exchange networks Finished Finished (Raford, 2010, after Malone et al., 2010) Wednesday, 21 April 2010
  • Schema for online approach The problem Can activities be divided into pieces? Are necessary resources widely distributed or in unknown locations? Yes No Crowd Hiearchy Are there adequate incentives to participate? Direct Influence / Love, Learning compenstion self-promotion friendship Yes What kind of activity needs to be done? Create Decide Can the activity be divided into Does the entire group need to small, independent pieces? abide by the same decision? Yes No Yes No Individual Group Collect Collaborate decisions decision Mechanisms for collaboration - wikis - blogs - discussion forums Are money or resources required to Are only a few good (best) - collaborative solutions needed? exchange hands or motivate workspaces, etc. Prediction decision? Voting Averaging Concensus markets No Yes Yes No Finished Market Trust Finished exchange networks Finished Finished Wednesday, 21 April 2010
  • Schema for online approach The problem Client defines key questions through initial Meetings, Can activities be divided into ID Issues pieces? Are necessary resources widely distributed or in unknown conversations & meetings conversations locations? Yes No Generate Expert interviews, brainstorm with client, F2F & phone Crowd Hiearchy key themes desktop research interviews Are there adequate incentives to ID driving Extract key themes, create trends and Group participate? forces timelines, key events workshop Direct Influence / Love, Learning Select key uncertainties and forces, list by compenstion self-promotion friendship Rank factors uncertainty / impact, predetermined drivers Develop draft scenario logic Create draft Create scenario snippets, draft systems diagrams, mix and match trends, 2x2 grids Integrate themes from draft scenarios, create headlines and scenario narratives Consultant report + Yes What kind of activity needs to be done? final scenarios Create Decide Can the activity be divided into Does the entire group need to Finalise Get client feedback, refine, detail, elaborate Group small, independent pieces? abide by the same decision? scenarios narrative to final form workshop Yes No Yes No Identify key strategic themes, reflect on Consider strategic questions in the context of each implications Collect Collaborate Individual Group scenario Mechanisms for decisions decision collaboration - wikis - blogs - discussion forums Are money or resources required to Are only a few good (best) - collaborative exchange hands or motivate Identify ID key indicators in each scenario for Consultant solutions needed? workspaces, etc. decision? Voting Averaging Concensus Prediction markets indicators strategic concerns report No Yes Yes No Finished Market Trust Finished exchange networks Finished Finished Wednesday, 21 April 2010
  • Schema for online approach The problem Client defines key questions through initial Meetings, Can activities be divided into ID Issues pieces? Are necessary resources widely distributed or in unknown conversations & meetings conversations locations? Yes No Generate Expert interviews, brainstorm with client, F2F & phone Crowd Hiearchy key themes desktop research interviews Are there adequate incentives to ID driving Extract key themes, create trends and Group participate? forces timelines, key events workshop Direct Influence / Love, Learning Select key uncertainties and forces, list by compenstion self-promotion friendship Rank factors uncertainty / impact, predetermined drivers Develop draft scenario logic Create draft Create scenario snippets, draft systems diagrams, mix and match trends, 2x2 grids Integrate themes from draft scenarios, create headlines and scenario narratives Consultant report + Yes What kind of activity needs to be done? final scenarios Create Decide Can the activity be divided into Does the entire group need to Finalise Get client feedback, refine, detail, elaborate Group small, independent pieces? abide by the same decision? scenarios narrative to final form workshop Yes No Yes No Identify key strategic themes, reflect on Consider strategic questions in the context of each implications Collect Collaborate Individual Group scenario Mechanisms for decisions decision collaboration - wikis - blogs - discussion forums Are money or resources required to Are only a few good (best) - collaborative exchange hands or motivate Identify ID key indicators in each scenario for Consultant solutions needed? workspaces, etc. decision? Voting Averaging Concensus Prediction markets indicators strategic concerns report No Yes Yes No Finished Market Trust Finished exchange networks Finished Finished Schema for online scenario planning Wednesday, 21 April 2010
  • Schema for online approach The problem Client defines key questions through initial Meetings, Can activities be divided into ID Issues pieces? Are necessary resources widely distributed or in unknown conversations & meetings conversations locations? Yes No Generate Expert interviews, brainstorm with client, F2F & phone Crowd Hiearchy key themes desktop research interviews Are there adequate incentives to ID driving Extract key themes, create trends and Group participate? forces timelines, key events workshop Direct Influence / Love, Learning Select key uncertainties and forces, list by compenstion self-promotion friendship Rank factors uncertainty / impact, predetermined drivers Develop draft scenario logic Create draft Create scenario snippets, draft systems diagrams, mix and match trends, 2x2 grids Integrate themes from draft scenarios, create headlines and scenario narratives Consultant report + Yes What kind of activity needs to be done? final scenarios Create Decide Can the activity be divided into Does the entire group need to Finalise Get client feedback, refine, detail, elaborate Group small, independent pieces? abide by the same decision? scenarios narrative to final form workshop Yes No Yes No Identify key strategic themes, reflect on Consider strategic questions in the context of each implications Collect Collaborate Individual Group scenario Mechanisms for decisions decision collaboration - wikis - blogs - discussion forums Are money or resources required to Are only a few good (best) - collaborative exchange hands or motivate Identify ID key indicators in each scenario for Consultant solutions needed? workspaces, etc. decision? Voting Averaging Concensus Prediction markets indicators strategic concerns report No Yes Yes No Finished Market Trust Finished exchange networks Finished Finished Schema for online scenario planning Wednesday, 21 April 2010
  • Schema for online approach The problem ID Issues Client defines key questions through initial conversations & meetings Meetings, conversations Can activities be divided into pieces? Are necessary resources widely distributed or in unknown locations? = Key Generate Expert interviews, brainstorm with client, F2F & phone Yes Crowd No Hiearchy decision key themes desktop research interviews Are there adequate incentives to points for Group system participate? ID driving Extract key themes, create trends and forces timelines, key events workshop Rank factors Select key uncertainties and forces, list by uncertainty / impact, predetermined drivers Direct compenstion Learning Influence / self-promotion Love, friendship design Develop draft scenario logic Create draft Create scenario snippets, draft systems diagrams, mix and match trends, 2x2 grids Integrate themes from draft scenarios, create headlines and scenario narratives Consultant report + Yes What kind of activity needs to be done? final scenarios Create Decide Can the activity be divided into Does the entire group need to Finalise Get client feedback, refine, detail, elaborate Group small, independent pieces? abide by the same decision? scenarios narrative to final form workshop Yes No Yes No Identify key strategic themes, reflect on Consider strategic questions in the context of each implications Collect Collaborate Individual Group scenario Mechanisms for decisions decision collaboration - wikis - blogs - discussion forums Are money or resources required to Are only a few good (best) - collaborative exchange hands or motivate Identify ID key indicators in each scenario for Consultant solutions needed? workspaces, etc. decision? Voting Averaging Concensus Prediction markets indicators strategic concerns report No Yes Yes No Finished Market Trust Finished exchange networks Finished Finished Schema for online scenario planning Wednesday, 21 April 2010
  • Schema for online approach The problem ID Issues Client defines key questions through initial conversations & meetings Meetings, conversations Can activities be divided into pieces? Are necessary resources widely distributed or in unknown locations? = Key Generate Expert interviews, brainstorm with client, F2F & phone Yes Crowd No Hiearchy decision key themes desktop research interviews Are there adequate incentives to points for Group system participate? ID driving Extract key themes, create trends and forces timelines, key events workshop Rank factors Select key uncertainties and forces, list by uncertainty / impact, predetermined drivers Direct compenstion Learning Influence / self-promotion Love, friendship design Develop draft scenario logic Create draft Create scenario snippets, draft systems diagrams, mix and match trends, 2x2 grids Integrate themes from draft scenarios, create headlines and scenario narratives Consultant report + Yes What kind of activity needs to be done? final scenarios Create Decide Can the activity be divided into Does the entire group need to Finalise Get client feedback, refine, detail, elaborate Group small, independent pieces? abide by the same decision? scenarios narrative to final form workshop Yes No Yes No Identify key strategic themes, reflect on Consider strategic questions in the context of each implications Collect Collaborate Individual Group scenario Mechanisms for decisions decision collaboration - wikis - blogs - discussion forums Are money or resources required to Are only a few good (best) - collaborative exchange hands or motivate Identify ID key indicators in each scenario for Consultant solutions needed? workspaces, etc. decision? Voting Averaging Concensus Prediction markets indicators strategic concerns report No Yes Yes No Finished Market Trust Finished exchange networks Finished Finished Schema for online scenario planning Wednesday, 21 April 2010
  • Case studies Differentiating dimensions of parameter space Type of information collected Opinion Degree of collaboration required Anecdote Who is involved & what is their in uence? Level of transparency Analysis Level of computer assistance (automation) Facts Presence & type of reputation accounting User interface design Forecasts Wednesday, 21 April 2010
  • Case studies Differentiating dimensions of parameter space Type of information collected Solo Degree of collaboration required Teams Who is involved & what is their in uence? Level of transparency Entire group Level of computer assistance (automation) Single round Presence & type of reputation accounting User interface design Multiple rounds Wednesday, 21 April 2010
  • Case studies Differentiating dimensions of parameter space Type of information collected General public Degree of collaboration required Who is involved & what is their in uence? Stake holders Level of transparency Level of computer assistance (automation) Experts Presence & type of reputation accounting User interface design Moderator Wednesday, 21 April 2010
  • Case studies Differentiating dimensions of parameter space Type of information collected Everyone sees & can use everything Degree of collaboration required Information shared Who is involved & what is their in uence? only with team mates Level of transparency Private information Level of computer assistance (automation) Anonymous Presence & type of reputation accounting User interface design Administrators only ones w/ access Wednesday, 21 April 2010
  • Case studies Differentiating dimensions of parameter space Type of information collected None Degree of collaboration required Who is involved & what is their in uence? Recommendations Level of transparency / ltering Level of computer assistance (automation) Presence & type of reputation accounting Pattern matching & User interface design algorithms Wednesday, 21 April 2010
  • Case studies Differentiating dimensions of parameter space Type of information collected None Degree of collaboration required Who is involved & what is their in uence? Level of transparency “Liked / Not Liked” Level of computer assistance (automation) Presence & type of reputation accounting Scoring, levels & User interface design points Wednesday, 21 April 2010
  • Case studies Differentiating dimensions of parameter space Type of information collected Web-form “bare bones” Degree of collaboration required Who is involved & what is their in uence? Level of transparency Functional, wiki-style Level of computer assistance (automation) Presence & type of reputation accounting User interface design Designed for UX Wednesday, 21 April 2010
  • Case studies Clustering into representative system types “Future “Expert Game” System” Signti c Labs Futurescaper IFTF IFF “Human Sensor Net” “Futurepedia” Sensemaker Vegas Online Suite System Wednesday, 21 April 2010
  • Case studies Future Game (Signti c, Institute for the Future) Wednesday, 21 April 2010
  • Case studies Expert System (Futurescaper, International Futures Forum) Wednesday, 21 April 2010
  • Case studies Human Sensor Net (Sensemaker Suite) Wednesday, 21 April 2010
  • Case studies Futurepedia (Vegas Online Scenario System) Wednesday, 21 April 2010
  • Evaluation approach Hypotheses 1. Individual learning will increase with greater participation in the system 2. Greater numbers of participants will increase the: a. number of variables and information considered b. amount of discussion around divergent viewpoints c. number of scenarios produced d. quality of nal scenarios 3. This will produce: a. a decrease in the probability estimation of individual future events b. an increase in spread of probability estimates for a range of future events c. better understanding of divergent viewpoints d. increased consensus around key issues Wednesday, 21 April 2010
  • Evaluation approach Empirical measures for hypothesis testing Number of participants Time spent participating (when & where) Participant demographics Number of variables created / used Number of scenarios created / shared / revised Number of comments / discussion threads Total time taken to complete exercise Wednesday, 21 April 2010
  • Evaluation approach Qualitative measures for hypothesis testing Expert panel review: quality of scenarios Self-reported review: quality of scenarios Self-reported review: quality of process - Appreciation of uncertainty - Understanding of divergent viewpoints - Consideration of alternative sources of evidence - Level of surprise / strategic relevance - Learning & attitude change Interviews & observation Wednesday, 21 April 2010
  • Evaluation approach Evaluation example, SenseMaker Suite Wednesday, 21 April 2010
  • Evaluation approach Evaluation example, SenseMaker Suite Wednesday, 21 April 2010
  • Evaluation approach Evaluation example, SenseMaker Suite Wednesday, 21 April 2010
  • Evaluation approach Evaluation example, SenseMaker Suite Wednesday, 21 April 2010
  • Limitations & next steps Lack of empirical evidence for “learning” (Chermack, 2005) Hard to isolate cause / effect relationships Dif cult to untangle UI effects, audience effects & “scenario effects”, i.e., Hawthorne effect? Dif culty attracting & curating the right audience Dif cult retaining interest & investment Wednesday, 21 April 2010
  • Large-Scale Participatory Futures Systems Thank you! Noah Raford PhD Candidate, UIS/CDD Massachusetts Institute of Technology (MIT) Department of Urban Studies and Planning nraford@mit.edu http://news.noahraford.com/ Friday, April 9, 2010 Wednesday, 21 April 2010