Ted Gordon Senior Research Fellow The Millennium Project World Federation of United Nations Associations
<ul><li>First developed at RAND in the 1960’s </li></ul><ul><li>A means of eliciting and combining expert judgments while ...
From Gordon and Helmer,  A Long Range Forecasting Study,  RAND, 1964 Economically useful desalination of sea water 1970 19...
Sociologist Economist Technologist Environment Person 1 XXX XXX Person 2 XXX Person 3 XXX Person 4 XXX Person 5 XXX XXX Pe...
<ul><li>First developed in a Defense Advanced Research Projects Agency (DARPA) study by Gordon and Pease (2004). </li></ul...
<ul><li>An on-screen form presents a series of questions to each participant, and for each: </li></ul><ul><ul><li>The aver...
<ul><li>There is only one round; no explicit second round is necessary as in a conventional Delphi. </li></ul><ul><li>When...
<ul><li>If an answer is beyond a pre-specified distance from the average an  indicator “flag” can be used to show the resp...
<ul><li>The answers can be submitted cell by cell or as an entire document. </li></ul><ul><li>To avoid crowding, the reaso...
<ul><li>Extensive hyperlink reference material can be included to permit the respondents to retrieve data on line while co...
<ul><li>Collection of judgments about when certain developments might occur, their importance, and impacts and reasons for...
<ul><li>Decision models:  </li></ul><ul><ul><li>Real Time Decision Making.   (DARPA)   </li></ul></ul><ul><ul><li>Choosing...
<ul><li>Substantive studies:  </li></ul><ul><ul><li>Future education and learning possibilities </li></ul></ul><ul><ul><li...
Possible Development  Likelihood by 2030 (%)  1. National programs for improving collective intelligence   Some richer as ...
1. What might encourage this possibility? What positive consequences might follow?  Please enter positive developments and...
<ul><li>Average, median, standard deviation </li></ul><ul><li>Distribution of opinions </li></ul><ul><li>Summaries by regi...
Upcoming SlideShare
Loading in …5
×

Real Time Delphi Briefing 8/08

2,169 views

Published on

A Briefing on the Real Time Delphi software tool developed by Ted Gordon of the WFUNA Millennium Project

Published in: Education, Economy & Finance
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,169
On SlideShare
0
From Embeds
0
Number of Embeds
13
Actions
Shares
0
Downloads
61
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Real Time Delphi Briefing 8/08

  1. 1. Ted Gordon Senior Research Fellow The Millennium Project World Federation of United Nations Associations
  2. 2. <ul><li>First developed at RAND in the 1960’s </li></ul><ul><li>A means of eliciting and combining expert judgments while avoiding the pitfalls of conference room confrontations. </li></ul><ul><li>Used a sequential questionnaires, each building on the previous round. </li></ul><ul><li>Required features: </li></ul><ul><ul><li>Participants guaranteed anonymity </li></ul></ul><ul><ul><li>Feedback of reason for extreme opinions </li></ul></ul><ul><li>Has been used in thousands of studies </li></ul>
  3. 3. From Gordon and Helmer, A Long Range Forecasting Study, RAND, 1964 Economically useful desalination of sea water 1970 1964-1980 Effective fertility control by oral contraceptive 1970 1970-1983 Development of new synthetic ultra light materials 1971 1970-1978 Automated language translators 1972 1968-1976 New organs through transplanting or prosthesis 1972 1968-1982 Reliable weather forecasts 1975 1972-1988 Operation of a central data storage facility 1980 1971-1991 Reformation of quantum relativity 1980 1975-1993 Implanted artificial organs made of plastic and electronic components 1981 1975-1988 Widespread and socially acceptable use of psychotropic drugs 1983 1980-2000 Creation of a form of artificial life (at least in the form of self replicating molecules) 1989 1979-2000 Economically useful exploitation of the ocean bottom 1980 1980-2000 Feasibility of limited weather control 1990 1987-2000 Commercial generation of protein for food 1990 1985-2003 Drug treatment for psychotic cases 1992 1983-2017 Biochemical general immunization 1994 1983-2000
  4. 4. Sociologist Economist Technologist Environment Person 1 XXX XXX Person 2 XXX Person 3 XXX Person 4 XXX Person 5 XXX XXX Person 6 XXX Person 7 XXX XXX Person 8 XXX TOTAL 3 2 2 3
  5. 5. <ul><li>First developed in a Defense Advanced Research Projects Agency (DARPA) study by Gordon and Pease (2004). </li></ul><ul><li>The object was to invent a technique by which a commander could rapidly collect and synthesize opinions of field personnel, using Delphi, to help him reach a decision. </li></ul><ul><li>Work was published in Technological Forecasting and Social Change . </li></ul><ul><li>Since then, many improvements and adaptations. </li></ul>
  6. 6. <ul><li>An on-screen form presents a series of questions to each participant, and for each: </li></ul><ul><ul><li>The average (or median) response of the group so far (and possibly the distribution of responses) </li></ul></ul><ul><ul><li>The number of responses made so far </li></ul></ul><ul><ul><li>A button that opens a window showing reasons that others have given for their responses. </li></ul></ul><ul><ul><li>A button that opens a window that allows the respondent to type in justifications for their own answer. </li></ul></ul><ul><ul><li>A space for the new numerical estimate, answering the question. </li></ul></ul>
  7. 7. <ul><li>There is only one round; no explicit second round is necessary as in a conventional Delphi. </li></ul><ul><li>When a respondent revisits the study in a minute or a day, the original input form is presented with their prior answers in place and the new average or median of the group, and if desired, the distribution of responses. </li></ul><ul><li>By then others may have contributed judgments, so the averages or medians may have changed since the original answers were provided. </li></ul>
  8. 8. <ul><li>If an answer is beyond a pre-specified distance from the average an indicator “flag” can be used to show the respondents the questions in which their answers differ significantly from the group. </li></ul><ul><li>When the flag is “up” reasons become particularly important </li></ul><ul><li>The respondents are not shown the group average until after they have inserted an answer and a selected number of responses have been received or a given time has elapse since the study has started. </li></ul>
  9. 9. <ul><li>The answers can be submitted cell by cell or as an entire document. </li></ul><ul><li>To avoid crowding, the reasons can be submitted and appear on a separate page. </li></ul><ul><li>A notation appears after the respondent has submitted an answer indicating that the response has been received and entered into the database. </li></ul><ul><li>Notations also appear if individual responses are outside of predetermined plausible ranges. </li></ul>
  10. 10. <ul><li>Extensive hyperlink reference material can be included to permit the respondents to retrieve data on line while completing the questionnaire. </li></ul><ul><li>Reference material is included directly on the questionnaire </li></ul><ul><li>Split samples can be used to reduce the effect of bias introduced by question placement on the questionnaire. </li></ul><ul><li>A permanent record is maintained of responses including both quantitative and narrative responses. </li></ul>
  11. 11. <ul><li>Collection of judgments about when certain developments might occur, their importance, and impacts and reasons for extreme opinions </li></ul><ul><li>Choosing from among alternative decisions based on clear cut criteria </li></ul><ul><li>Providing information for and sharpening of scenarios </li></ul><ul><li>Input/output and cross impact matrixes </li></ul><ul><li>Environmental monitoring </li></ul>
  12. 12. <ul><li>Decision models:  </li></ul><ul><ul><li>Real Time Decision Making. (DARPA)   </li></ul></ul><ul><ul><li>Choosing a Configuration. (auto mfg) </li></ul></ul><ul><li>  Priority studies:   </li></ul><ul><ul><li>Assessing Priorities of Proposed Millennium Project Study Topic, 2006 . </li></ul></ul><ul><ul><li>Assessing Priorities of Proposed Millennium Project Study Topic. 2007 </li></ul></ul><ul><ul><li>Assessing Priorities of Proposed Millennium Project Study Topic. 2008 </li></ul></ul><ul><li>SOFI studies:   </li></ul><ul><ul><li>Variables for the Global State of the Future Index </li></ul></ul><ul><ul><li>Variables for the Republic of Korea State of the Future Index </li></ul></ul><ul><ul><li>Variables for the South Africa Global State of the Future Index </li></ul></ul>
  13. 13. <ul><li>Substantive studies:  </li></ul><ul><ul><li>Future education and learning possibilities </li></ul></ul><ul><ul><li>Future Energy Developments </li></ul></ul><ul><li>  </li></ul><ul><li>Studies in support of UNESCO: </li></ul><ul><ul><li>Judgments on acceptability of a Delphi process . </li></ul></ul><ul><ul><li>  Scenario Drivers. </li></ul></ul><ul><ul><li>Report construction </li></ul></ul><ul><li>Resource allocation (WB subcontractor) </li></ul><ul><ul><li>Performance </li></ul></ul><ul><ul><li>Biodiversity </li></ul></ul><ul><ul><li>Climate Change </li></ul></ul>
  14. 14. Possible Development Likelihood by 2030 (%) 1. National programs for improving collective intelligence Some richer as well as lower income countries have (by this year of 2030) made improving collective intelligence a national goal; this includes improving individual as well as intelligence for their nations-as-whole. Click here to see references Likelihood by 2030 The average group answer: 61.7 Respondents: 195 Comments on this possibility (page 2) click here 2. Just in time knowledge and learning Rote learning has diminished in importance. With ubiquitous computing and education for life-long learning, 'just in time knowledge' has become the norm. Reasoning, problem solving, and learning strategies form the core focus of public educational systems. Click here to see references Likelihood by 2030 The average group answer: 71.1 Respondents: 185 Comments on this possibility (page 2) click here
  15. 15. 1. What might encourage this possibility? What positive consequences might follow? Please enter positive developments and consequences. If you wish to change your prior input, type it here:  
  16. 16. <ul><li>Average, median, standard deviation </li></ul><ul><li>Distribution of opinions </li></ul><ul><li>Summaries by region, occupation, gender </li></ul><ul><li>Correlation of qualitative responses with quantitative responses (e.g. “those who answered with a low estimate, had these reasons…”) </li></ul>

×