• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Raford PhD defense final
 

Raford PhD defense final

on

  • 3,751 views

 

Statistics

Views

Total Views
3,751
Views on SlideShare
1,627
Embed Views
2,124

Actions

Likes
6
Downloads
81
Comments
0

11 Embeds 2,124

http://news.noahraford.com 1701
http://www.gwen-dolen.com 277
http://noahraford.com 71
http://pmoscenarios.wordpress.com 64
http://us-w1.rockmelt.com 3
https://twitter.com 2
http://tweetree.com 2
http://ven-ex.eu 1
http://localhost 1
http://dwarf.guruj.net 1
http://translate.googleusercontent.com 1
More...

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

Raford PhD defense final Raford PhD defense final Presentation Transcript

  • Large-Scale Participatory Futures Systems A Comparative Study of Online Scenario Planning ApproachesCandidate: Noah Raford PhD Candidate, Urban Information Systems Group, City Design and Development Group, Department of Urban Studies and Planning, MITCommittee: Michael Flaxman (Chair) Assistant Professor, Urban Information Systems Group, MIT Joseph Ferreira Professor of Urban Planning and Operations Research, Associate Department Head and Head of Urban Information Systems Group, MIT Andres Sevtsuk Lecturer, Department of Urban Studies and Planning, MIT
  • Outline1. Introduction2. Review & Synthesis of the Literature3. Study Design & Methodology4. Findings & Discussion5. Conclusion Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 2
  • IntroductionQualitative Scenario PlanningHumans have important shortcomings that Client defines key questions through initial Meetings, ID Issueslimit our ability to make effective decisions conversations & meetings conversationsunder conditions of dynamic uncertainty Generate key themes Expert interviews, brainstorm with client, desktop research F2F & phone interviews(Dorner, 1997) ID driving Extract key themes, create trends and Group workshop“A disciplined methodology for imaging forces timelines, key eventspossible futures in which organizational Rank factors Select key uncertainties and forces, list by uncertainty / impact, predetermined driversdecisions may be played out” (Shoemaker, Develop draft Create scenario snippets, draft systems1995) scenario logic diagrams, mix and match trends, 2x2 grids Create draft Integrate themes from draft scenarios, create Consultant“Tools for foresight discussions... final scenarios headlines and scenario narratives reportwhose purpose is not a prediction or a Finalise Get client feedback, refine, detail, elaborate Group workshopplan, but a change in the mindset of the scenarios narrative to final form Identify key strategic themes, reflect onpeople who use them” (de Gues, 1997) Consider implications strategic questions in the context of each scenario Identify ID key indicators in each scenario for Consultant indicators strategic concerns report Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 3
  • IntroductionPurported Bene ts Reduced individual and group decision bias Scenarios (Tetlock, 2006) Increased Enhanced awareness of environmental change, learning future risks & opportunities (Weick, 1999) Gain appreciation of different More accurate mental models stakeholders’ positions and attitudes (Chermack, 2003) Better Greater exibility and better decisions decision-making (Schwartz, 1997) Improved performance (Chermack, 2003) Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 4
  • IntroductionChallenges to Scenario Planning in the Public Realm Labor intensive & expensive Bene ts poorly documented (no veri cation or reputation systems) Limited participation (time, space & numbers) Predominance of senior decision-making elite (participant bias) Highly dependent on facilitator skills & consultant synthesis (facilitator & author bias) Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 5
  • IntroductionResearch Questions Do web-based participatory approaches add value to the traditional scenario planning process? If so, where and in what ways? If not, where do they fall short, in what ways, and why? Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 6
  • Outline1. Introduction2. Review & Synthesis of the Literature3. Study Design & Methodology4. Findings & Discussion5. Conclusion Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 7
  • Literature Review & Synthesis Planning Support Urban Planning Role of the Future Systems (PSS) & Policy Policy ICT Platforms Scenario & Web 2.0 Planning ? Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 8
  • Literature Review & SynthesisUrban Planning & Public Policy “The future orientation of planning is unique to the fields identity... The very substance of urban planning is founded in time (Myers and Kitsuse, 2000) Four planning traditions (Freidman, 1987): • Social Reform • Policy Analysis (Simon, 1945; Forrester, 1968; Stokey and Zeckhauser, 1978) • Social Learning (Majone, 1989; Scott, 1998; Schon, 1983) • Social Mobilization (Davidoff, 1965; Forester, 1989; Castells, 1977; Healey, 1992; Innes, 1996) Growing demand for public participation (Arnstein, 1969; Hulchanski, 1977; APA, 1990) “Urban planning has retreated from strategic, future-oriented topics to become absorbed in operational and managerial activities characterized by short time horizons and value choices likely to be equally short-sighted and ad hoc” (Coucelis, 2005) Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 9
  • Literature Review & SynthesisICT Platforms Planning Support Systems (PSS) “Loosely coupled assemblages of computer- based techniques”, forming a mixed toolbox of techniques to help decision- makers in their daily tasks (Britton Harris, 1989; Brail and Klosterman, 2001; Batty, 2003) • PPGIS (Warnecke, Beatie, & Lyday, 1998; Craig & Elwood, 1998; Geertman & Stillwell, 2003) • Alternative Futures Analysis (Steinitz, 2003; Lagigno & Reed, 2003; Hopkins & Zapata, 2007) • Participatory Agent Based Modeling (Bousquet & Le Page, 2004; Barnaud et al., 2007; Castella et al, 2005) “Modelling as negotiation” (Guhathakurta, 1993) “Complicated, convoluted, time-consuming, and intimidating... that do not achieve genuine participation in planning or other decisions” (Innes & Booher, 2004; Cooke & Kotari, 2001) Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 10
  • Literature Review & SynthesisCrowdsourcing & Web 2.0 Web 2.0 (O’Rielly; 2005; Anderson; 2007) • Crowdsourcing (Howe, 2006) • Collective Intelligence (Levy, 1994; Por, 2008; Malone et al., 2010) • Human Computation (Quinn and Bederson, 2010; Sakamoto et al., 2010) “The creation, aggregation and interpretation of strategically relevant information for decision-making through distributed means” (Por, 2008) Wikipedia, Innocentive, Threadless, CrowdFlower, IdeaScale, Reddit, etc. Have been studied but rarely used as research instruments themselves (Malone, 2010) Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 11
  • Literature Review & SynthesisScenario Planning Creative, narrative, group-based processes for engaging with uncertainty and change (Wack, 1985; Van der Heijden, 1997) • Double loop organizational learning (Argys & Schon; 1974) • Constructivist & social learning theory (Piaget, 1977) • Sensemaking & organizational awareness (Weick, 1979; Kleine,1999) • Activity- & practice-based strategizing (Jarzabkowski, 2005; Orlikowski, 1992) • Competitive advantages of perception management (Boyd, 1976) Labor intensive & expensive, bene ts poorly documented (no veri cation or reputation systems), limited participation (time, space & numbers), predominance of senior decision-making elite (participant bias), dependent on facilitator skills & consultant synthesis (facilitator & author bias) Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 12
  • IntroductionContribution of This Study 1. Operational: Help to understand the role that online systems might play in enhancing multi-stakeholder policy creation, speci cally in the context of the challenges of future-focused, public planning initiatives 2. Methodological: Help to generate new analytical frameworks that can improve our understanding of how such systems may be used for measurement instruments and data analysis platforms Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 13
  • Outline1. Introduction2. Review & Synthesis of the Literature3. Study Design & Methodology4. Findings & Discussion5. Conclusion Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 14
  • Study Design & MethodologyResearch Questions• Do web-based participatory approaches add value to the traditional scenario planning process? If so, where and in what ways?• If not, where do they fall short, in what ways, and why?Participation Interaction• The number and type of • The number of variables and participants involved, and in opinions incorporated? what phases? • The mechanism of analysis,• The geographic scope of ranking and clustering? participation enabled? • The time spent on data• The range of expert professional collection and analysis? disciplines consulted? • The amount of user debate and re ection? Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 15
  • Study Design & MethodologyAn Exploratory Case Study ApproachA three-tiered, mixed method, case-study based approach, including: • Informant interviews to identify key Interviews themes and constructs (n=46) • Creation of two novel, prototypical In-depth data generation platforms and cases application on in-depth cases • Pair-wise comparison of case Base case studies to a base case • Evaluation of three additional Comparative comparative examples from examples secondary sources Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 16
  • Study Design & MethodologyCase 1: Futurescaper: The Impact of Climate ChangeImpacts on the UK 186 drivers, ranked, analyzed and visualized as system maps Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 17
  • Study Design & MethodologyCase 2: SenseMaker Scenarios: Future of Public ServicesUnder Financial Uncertainty • 265 participants, micro- scenarios • Aggregated to three sketch scenarios based on pre-de ned archetypes Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 18
  • Study Design & MethodologyBase Case: Future of a Northern Region in Spain • Face-to-face scenario method • Expert scenario consultancy • 15 in-depth interviews • Two day workshop, 20 participants • Four regional scenarios Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 19
  • Study Design & MethodologyComparative Examples Institute for the Future’s Foresight Engine • 700 participants • 81 countries • 5,000 submissions in 24 hours WikiStrat Collaborative Strategy Platform • 30 teams • 13 countries • ~35,000 words of high-quality content created in 4 weeks The Future of Facebook Project • 25 video interviews • 109 Quora interactions • ~50 Facebook participants Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 20
  • Study Design & MethodologyData Constructs MeasuredParticipant Characteristics Interaction Characteristics • Degree of public openness • Tasks performed (including promotion & recruitment • Amount and types of input efforts) considered • Amount of preparation required • Amount and types of visualization • The number of participants involved tools used • Reasons for participation • Amount and types of analytical • Degree of user anonymity tools used • Type of participants involved • Amount and kinds of socialization • Level of Education enabled • Professional Experience • Amount and kinds of feedback • Professional Discipline provided • Age • Geographic Origin • Supplementary interviews Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 21
  • Study Design & MethodologyChallenges 1) The relevant categories and variables for measurement were unknown in advance 2) There was little empirical evidence for, or agreement on, the key outcome variables for scenario planning 3) There were no standard measurement instruments or protocols available that could be readily appliedBoth dependent and independent variables were unknown and no standardmethod of comparison could be established.An exploratory, or “revelatory” case study design (Yin, 1994) was appropriate. Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 22
  • Outline1. Introduction2. Review & Synthesis of the Literature3. Study Design & Methodology4. Findings & Discussion5. Conclusion Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 23
  • Findings & DiscussionFinding 1: Greater Number and Diversity of Participants Number of ParticipantsMore participants were involved 700 700 525 350 265 175 Base Case (166) SenseMaker 150 125 Foresight Engine WikiStrat FoFB 35 0 Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 24
  • Findings & DiscussionFinding 1: Greater Number and Diversity of Participants Number of Countries RepresentedMore participants were involved 90 82From more diverse locations 68 45 30 23 Base Case SenseMaker 18 Foresight Engine WikiStrat 5 0 Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 25
  • Findings & DiscussionFinding 1: Greater Number and Diversity of ParticipantsMore participants were involved Base Case: ~20 different disciplines Case 1: 35 different disciplinesFrom more diverse locations Case 2: Signi cant experienceWider range of experts &professional disciplines WikiStrat: Mixed teams of highly trained inter-disciplinary contributors Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 26
  • Findings & DiscussionFinding 2: Most Participation Was Light, Skewed Towards aFew Heavy Users Base Case: ~4.5 contributions per user, more extensive involvement & conversation through-out workshop Case 1: ~ 1 contribution per user # of Case 2: ~ 1 contribution per user Cards Forecastin #Name Location Occupation Played g Points SI Awards!"#$%&$"(")*+ ,-./-. 012.3* 456 447857 8 9*2&*.:*( IFTF: ~6 contributions per user (1.5;"/";& <"&12.($-.=+ ?-;#@$"$2-."A+C5 5DE6 7 F*G.;". >? B2-A-(2&$ original contributions, 4.5 responses to&*!*$*.(2.** ?-A-"/- H.(2.** ID4 47J7 7 9*2&*.:*( 0H>+0"A3 others), 20% of users = 70% of content;"$1#@.3 K-$A"./=+LM N"$1*;"$2!2". I46 E84 7 N"!O$1@ 9*2&*.:*($1*#A-$P$12!3*.&B?=+?"."/" HA*!$2!"A+ H.(2.** II5 5II 7 WikiStrat: Intensive contribution through-out process, ~7,000 words per team Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 27
  • Findings & DiscussionFinding 3: Rapid Driver Generation & Exploration Driver Generation: Base Case: 80 hours + 120 minutes in workshop (5 hours per driver) Case 1: ~5 minutes per driver Case 2: ~10 minutes per driver IFTF: ~90 seconds per driver Clustering & Ranking: Base Case: ~2 hours in workshop, “not enough time to discuss” Case 1: Instantly sortable along number of dimensions Case 2: Instantlysortable along number of dimensions IFTF: N/A WikiStrat: N/A FoFB: Unknown, but “signi cant and more than we thought” Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 28
  • Findings & DiscussionFinding 4: Most In uential at Early Stages Detailed Case Comparative Studies Examples Increases the likelihood Scenario Planning Case 1: Case 2: Foresight WikiStrat Open Futurescaper SenseMaker Engine Foresight Steps that a wide variety of ID Issues forces and factors will be included Generate key themes Increases likelihood that ID driving forces a diversity of perspectives will be Rank factors achieved Develop draft scenario logic Implies that individual Create draft and group biases may final scenarios be less dominant at the Finalise early drivers exploration scenarios stage Consider implications Identify Strong scaling potential indicators Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 29
  • Findings & DiscussionFinding 5: “The Hourglass Effect” Tension between structured / unstructured interfaces and analysis approaches Case 1: Highly structured interface, open-ended analysis Case 2: Open-ended interface, highly structured analysis More data = greater analytical burden IFTF: Largest number of drivers and social interaction, but very dif cult to make sense of FoFB: “None of us had any idea it would take this long to complete.” Trade off between ease of use & level of participation IFTF: Simple, game-like engaging interface, very light analytic power WikiStrat: High barrier of entry, rich analytic input and deep participation Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 30
  • Findings & DiscussionFinding 5: “The Hourglass Effect” Tension between structured / unstructured interfaces and analysis approaches Case 1: Highly structured interface, open-ended analysis Case 2: Open-ended interface, highly structured analysis “People enter these activities with little More data = background experience. Part of your job is to greater analytical burden help model the thinking process that they IFTF: Largest number of drivers and social interaction, but very dif cult to should undergo.” make sense of FoFB: “None of us had any idea it would take this long to complete.” Trade off between ease of use & level of participation IFTF: Simple, game-like engaging interface, very light analytic power WikiStrat: High barrier of entry, rich analytic input and deep participation Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 30
  • Findings & DiscussionFinding 6: Role of Visuals & Multimedia Increasing hydrological imbalance Decreasing water quality Increasing malaria Decreasing water availability Increasing toxic algal blooms Increasing hardships for women Increasing improved water and sanitation Increasing food prices Increasing diarrhea Increasing population displacement Decreasing agricultural productivity Decreasing crop yields Increasing pollution Decreasing sustainablilty of crop production Decreasing deaths from cold temperatures Increasing demand Increasing migration Increasing water shortages Increasing market Increasing flooding Increasing air pollution Increasing droughts Increasing contamination of water supply Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 31
  • Findings & DiscussionFinding 6: Role of Visuals & Multimedia Increasing uncertainty in food production Decreasing agricultural productivity Increasing potency of airborne diseases Increasing droughts Increasing food prices Increasing diarrhea Increasing hardships for women Decreasing deaths from cold temperatures Increasing malaria Increasing air pollution Decreasing water quality Increasing hydrological imbalance Increasing flooding Increasing improved water and sanitation Decreasing water availability Increasing toxic algal blooms Decreasing crop yields Increasing water shortages Increasing migration Increasing contamination of water supply Decreasing sustainablilty of crop production Increasing population displacement Increasing pollution Increasing market Increasing demand Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 31
  • Findings & DiscussionFinding 6: Role of Visuals & Multimedia Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 31
  • Findings & DiscussionFinding 6: Role of Visuals & Multimedia Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 31
  • Findings & DiscussionFinding 7: Social Experience of Online Scenario Building Base Case was far more effective at producing active socialization and interaction between participants “People need feedback in order to stay involved. You can provide automated feedback, but other people are the best kind of feedback you can possibly ask for.” Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 32
  • Findings & DiscussionFinding 7: Social Experience of Online Scenario Building Base Case was far more effective at producing active socialization and interaction between participants “People need feedback in order to stay involved. You can provide automated feedback, but other people are the best kind of feedback you can possibly ask for.” Different kinds of experience were possible with IFTF and WikiStrat • Ranks and Roles • “Coopetition” Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 32
  • Findings & DiscussionSpeculative Finding 1: Better Outcomes? The evidence suggests that the use of such systems on their own will not produce the desired outcome of the scenario process Augment early-stages • Transparency • Speed • Ef ciency • Larger scale engagement Suggests may be effective analytically, but is it psychologically? A hybrid approach is worth exploring to get the full bene ts Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 33
  • Findings & DiscussionSpeculative Finding 2: Impact on Professional Standards Greater transparency could facilitate reputation systems (eBay, Amazon) “The futures profession is decentralized, eclectic and intellectually varied: there are no schools that train its elite, few barriers to entry, no certi cation or regulatory body.” (Pang, 2009) Commoditize the scenarios market, split between “fast & cheap” or “slow & bespoke” Trade-off between quality (qualitative) aspects & quantity / speed Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 34
  • Findings & DiscussionSpeculative Finding 3: Impact on Scholarly Method Continuous, self-re ective and emergent Allow for user re ection on, and modi cation of, research constructs “Moderators... sometimes have the feeling that they’re barely holding on for dear life, because sometimes the carriage tries to run away without them.” Requires post-hoc and real-time evaluation, dif cult to determine what to study in advance Signi cantly enhanced potential for creativity, but signi cant challenges for research design and rigor Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 35
  • Outline1. Introduction2. Review & Synthesis of the Literature3. Study Design & Methodology4. Findings & Discussion5. Conclusion Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 36
  • ConclusionContribution 1) Creating under-explored connections between urban planning, public participation, online tools and scenario planning 2) The creation and evaluation of two unique online platforms for participatory scenario planning in urban planning and public policy 2) The creation of an intellectual framework for measuring and evaluating their role in the qualitative scenario planning process. Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 37
  • ConclusionLimitations Lack of a more rigorous experimental design, more controlled cases or a peer- reviewed evaluation framework Lack of a controlled, standardized recruitment process for participation Differences in de nitions, processes and goals between cases and comparative examples Strongly dissenting views and participants self-selected out of being interviewed, thereby biasing the results and discussion towards those available and interested in the subject Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 38
  • ConclusionPossible Evolution of These Approaches Personal Futures Systems Real-time Horizon Scanning & Scenario Generation Systems Crowdsourced Think Tank Policy Review Mass Media Speculation Engines Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 39
  • ConclusionAreas for Future Research Continue to develop more rigorous measures for evaluating the scenario process and its outcomes Conduct more controlled research on the impacts of speci c design and interaction features Explore the impact of various forms of socialization systems (chat, commenting, voting, etc.) on the process and outcomes Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 40
  • Thank You Noah Raford PhD Candidate, Urban Information Systems Group, City Design and Development Group, Department of Urban Studies and Planning, MIT nraford@mit.edu August 29, 2011 Questions? Noah Raford, PhD Defence, MIT DUSP, Large Scale Participatory Futures Systems: A Comparative Study of Online Scenario Planning Approaches 41