SlideShare a Scribd company logo

Judgement: Managing the Cognitive Factor to Improve the accuracy of Technology Foresight

IFIS_org
IFIS_org

Presentation made at Military Technology University (Warsaw) for PYTHIA project. The PYTHIA project aims to devise an innovative methodology for strategic technology foresight, able to deliver frequent “predictions” on technology-related matters. Project is granted by European Defence Agency.

1 of 13
Download to read offline
© INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED.
JUDGEMENT: MANAGING THE COGNITIVE
FACTOR TO IMPROVE THE ACCURACY OF
TECHNOLOGY FORESIGHT
4th Workshop
Military University of Technology
04 - 2019
Pawel Fleischer
President
Institute for Forecasting and International Studies
Contents
© INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED.
1. BIBLIOGRAPHY
2. THE ABSOLUTE WORST TECHNOLOGY PREDICTIONS
3. COGNITIVE BIASES
4. GOOD JUDGEMENT PROJECT
1. 4.1 GJP – EXAMPLES
2. 4.3 GJP - RESULTS
3. 4.4 BIASES IN FORECASTING
5. LL'S FOR TECHNOLOGY FORESIGHT?
© INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED.
Bibliography
• Exploring the cognitive value of technology foresight: The case of the Cisco Technology Radar, Siri
Boe-Lillegraven, Stephan Monterde,
• Expert forecast and realized outcomes in technology foresight, Riccardo Apredaa, Andrea
Bonaccorsib, Felice dell'Orlettac, Gualtiero Fantonib,
• Strategic foresight in corporate organizations: Handling the effect and response uncertainty of
technology and social drivers of change, Riccardo Vecchiato, Claudio Roveda
• Pitfalls in and success factors of corporate foresight projects, M. Atilla Öner, Senem Göl, Int. J.
Foresight and Innovation Policy, Vol. 3, No. 4, 2007
• UNIDO TECHNOLOGY FORESIGHT MANUAL, Volume 1 - Organization and Methods
• Evidence on good forecasting practices from the Good Judgment Project: an accompanying blog
post
• Making the Best Use of Judgmental Forecasting, Shayne Kavanagh, Daneil Willaims, Government
Finance Review
• Superforecasting: The Art and Science of Prediction, Philip E. Tetlock, Dan Gardner
© INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED.
The absolute worst technology predictions
1936: “A rocket will never be able to leave the Earth’s
atmosphere.” —New York Times.
1977: “There is no reason for any individual to have a computer
in his home.” —Ken Olsen, founder of Digital Equipment Corp.
1989: “We will never make a 32-bit operating system.” 
—Bill Gates, co-founder and chairman of Microsoft.
2003: “The subscription model of buying music is bankrupt. I think you
could make available the Second Coming in a subscription model, and it
might not be successful.” —Steve Jobs, in Rolling Stone
© INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED.
Cognitive Biases
© INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED.

Recommended

Introduction to Futures Studies
Introduction to Futures StudiesIntroduction to Futures Studies
Introduction to Futures StudiesVahid Shamekhi
 
TCFD Workshop: Practical steps for implementation – Ian Edwards
 TCFD Workshop: Practical steps for implementation – Ian Edwards TCFD Workshop: Practical steps for implementation – Ian Edwards
TCFD Workshop: Practical steps for implementation – Ian EdwardsMcGuinness Institute
 
Reference Class Forecasting - useful method, or random number generator? webinar
Reference Class Forecasting - useful method, or random number generator? webinarReference Class Forecasting - useful method, or random number generator? webinar
Reference Class Forecasting - useful method, or random number generator? webinarAPMDonotuse
 
Technology Forecasting - Monitoring and Intelligence Methods
Technology Forecasting - Monitoring and Intelligence MethodsTechnology Forecasting - Monitoring and Intelligence Methods
Technology Forecasting - Monitoring and Intelligence MethodsMufaddal Nullwala
 
Foresight; The Springboard to maintaining pace and achieving success
Foresight; The Springboard to maintaining pace and achieving successForesight; The Springboard to maintaining pace and achieving success
Foresight; The Springboard to maintaining pace and achieving successChiomaChigozieOkwum
 
Cross impact-analysis
Cross impact-analysisCross impact-analysis
Cross impact-analysisA P
 
Strategy under uncertainty by
Strategy under uncertainty byStrategy under uncertainty by
Strategy under uncertainty bysunil prasad
 
Some New Directions in the Economics of AI
Some New Directions in the Economics of AISome New Directions in the Economics of AI
Some New Directions in the Economics of AIJuan Mateos-Garcia
 

More Related Content

Similar to Judgement: Managing the Cognitive Factor to Improve the accuracy of Technology Foresight

Getsmarter Survey: Disruptive tech report 2018
Getsmarter Survey: Disruptive tech report 2018Getsmarter Survey: Disruptive tech report 2018
Getsmarter Survey: Disruptive tech report 2018Richard Kleiner
 
Getsmarter Survey: Disruptive tech report 2018
Getsmarter Survey: Disruptive tech report 2018Getsmarter Survey: Disruptive tech report 2018
Getsmarter Survey: Disruptive tech report 2018Susan Kench
 
Experiment of the Assessment of Societal and Economic Impacts by Policy Simul...
Experiment of the Assessment of Societal and Economic Impacts by Policy Simul...Experiment of the Assessment of Societal and Economic Impacts by Policy Simul...
Experiment of the Assessment of Societal and Economic Impacts by Policy Simul...scirexcenter
 
Orbit Foundations of Responsible Research and Innovation
Orbit Foundations of Responsible Research and InnovationOrbit Foundations of Responsible Research and Innovation
Orbit Foundations of Responsible Research and InnovationLab Hackaton Zimbabwe
 
Claus Kjeldsen - Future Forum 2013
Claus Kjeldsen - Future Forum 2013Claus Kjeldsen - Future Forum 2013
Claus Kjeldsen - Future Forum 2013Friuli Future Forum
 
David Winickoff, Session 1
David Winickoff, Session 1David Winickoff, Session 1
David Winickoff, Session 1OECD Governance
 
Capital and infrastructure solution brochure Single FINAL
Capital and infrastructure solution brochure Single FINALCapital and infrastructure solution brochure Single FINAL
Capital and infrastructure solution brochure Single FINALChristopher Matthews
 
20210312 Victoria University Presentation
20210312 Victoria University Presentation 20210312 Victoria University Presentation
20210312 Victoria University Presentation McGuinness Institute
 
CEO's Guide to Sound Decision Making in 21st Century
CEO's Guide to Sound Decision Making in 21st CenturyCEO's Guide to Sound Decision Making in 21st Century
CEO's Guide to Sound Decision Making in 21st CenturyVivek Sood
 
Foresight Methods and Practice: Lessons Learned from International Foresight ...
Foresight Methods and Practice: Lessons Learned from International Foresight ...Foresight Methods and Practice: Lessons Learned from International Foresight ...
Foresight Methods and Practice: Lessons Learned from International Foresight ...Totti Könnölä
 
IFI Seminar - Foresight for Science, Technology & Innovation
IFI Seminar - Foresight for Science, Technology & InnovationIFI Seminar - Foresight for Science, Technology & Innovation
IFI Seminar - Foresight for Science, Technology & InnovationChaiyatorn Limapornvanich
 
Governance & integrated thinking….through a responsible investor lens: The ch...
Governance & integrated thinking….through a responsible investor lens: The ch...Governance & integrated thinking….through a responsible investor lens: The ch...
Governance & integrated thinking….through a responsible investor lens: The ch...Dr Raj Thamotheram
 
Communicating science and quality for understanding climate change-Talk
Communicating science and quality for understanding climate change-TalkCommunicating science and quality for understanding climate change-Talk
Communicating science and quality for understanding climate change-TalkDr. Pamposh Kumar
 
"Potential Impacts of Privacy Regulation and Litigation on Vision Technology,...
"Potential Impacts of Privacy Regulation and Litigation on Vision Technology,..."Potential Impacts of Privacy Regulation and Litigation on Vision Technology,...
"Potential Impacts of Privacy Regulation and Litigation on Vision Technology,...Edge AI and Vision Alliance
 
Thin and ultra thin film market global scenario
Thin and ultra thin film market global scenarioThin and ultra thin film market global scenario
Thin and ultra thin film market global scenarioManisha Sharma
 
Introduction to Horizon Scanning 2016
Introduction to Horizon Scanning 2016Introduction to Horizon Scanning 2016
Introduction to Horizon Scanning 2016Howard Smith
 

Similar to Judgement: Managing the Cognitive Factor to Improve the accuracy of Technology Foresight (20)

Getsmarter Survey: Disruptive tech report 2018
Getsmarter Survey: Disruptive tech report 2018Getsmarter Survey: Disruptive tech report 2018
Getsmarter Survey: Disruptive tech report 2018
 
Getsmarter Survey: Disruptive tech report 2018
Getsmarter Survey: Disruptive tech report 2018Getsmarter Survey: Disruptive tech report 2018
Getsmarter Survey: Disruptive tech report 2018
 
Experiment of the Assessment of Societal and Economic Impacts by Policy Simul...
Experiment of the Assessment of Societal and Economic Impacts by Policy Simul...Experiment of the Assessment of Societal and Economic Impacts by Policy Simul...
Experiment of the Assessment of Societal and Economic Impacts by Policy Simul...
 
Orbit Foundations of Responsible Research and Innovation
Orbit Foundations of Responsible Research and InnovationOrbit Foundations of Responsible Research and Innovation
Orbit Foundations of Responsible Research and Innovation
 
Claus Kjeldsen - Future Forum 2013
Claus Kjeldsen - Future Forum 2013Claus Kjeldsen - Future Forum 2013
Claus Kjeldsen - Future Forum 2013
 
David Winickoff, Session 1
David Winickoff, Session 1David Winickoff, Session 1
David Winickoff, Session 1
 
Capital and infrastructure solution brochure Single FINAL
Capital and infrastructure solution brochure Single FINALCapital and infrastructure solution brochure Single FINAL
Capital and infrastructure solution brochure Single FINAL
 
R-10 Tech_M_Group_8.pptx
R-10 Tech_M_Group_8.pptxR-10 Tech_M_Group_8.pptx
R-10 Tech_M_Group_8.pptx
 
20210312 Victoria University Presentation
20210312 Victoria University Presentation 20210312 Victoria University Presentation
20210312 Victoria University Presentation
 
CEO's Guide to Sound Decision Making in 21st Century
CEO's Guide to Sound Decision Making in 21st CenturyCEO's Guide to Sound Decision Making in 21st Century
CEO's Guide to Sound Decision Making in 21st Century
 
Foresight Methods and Practice: Lessons Learned from International Foresight ...
Foresight Methods and Practice: Lessons Learned from International Foresight ...Foresight Methods and Practice: Lessons Learned from International Foresight ...
Foresight Methods and Practice: Lessons Learned from International Foresight ...
 
IFI - Foresight for Science, Technology and Innovation
IFI - Foresight for Science, Technology and InnovationIFI - Foresight for Science, Technology and Innovation
IFI - Foresight for Science, Technology and Innovation
 
IFI Seminar - Foresight for Science, Technology & Innovation
IFI Seminar - Foresight for Science, Technology & InnovationIFI Seminar - Foresight for Science, Technology & Innovation
IFI Seminar - Foresight for Science, Technology & Innovation
 
Governance & integrated thinking….through a responsible investor lens: The ch...
Governance & integrated thinking….through a responsible investor lens: The ch...Governance & integrated thinking….through a responsible investor lens: The ch...
Governance & integrated thinking….through a responsible investor lens: The ch...
 
Communicating science and quality for understanding climate change-Talk
Communicating science and quality for understanding climate change-TalkCommunicating science and quality for understanding climate change-Talk
Communicating science and quality for understanding climate change-Talk
 
"Potential Impacts of Privacy Regulation and Litigation on Vision Technology,...
"Potential Impacts of Privacy Regulation and Litigation on Vision Technology,..."Potential Impacts of Privacy Regulation and Litigation on Vision Technology,...
"Potential Impacts of Privacy Regulation and Litigation on Vision Technology,...
 
2013 oas healing americas-1.7
2013 oas healing americas-1.72013 oas healing americas-1.7
2013 oas healing americas-1.7
 
Thin and ultra thin film market global scenario
Thin and ultra thin film market global scenarioThin and ultra thin film market global scenario
Thin and ultra thin film market global scenario
 
S fahey
S faheyS fahey
S fahey
 
Introduction to Horizon Scanning 2016
Introduction to Horizon Scanning 2016Introduction to Horizon Scanning 2016
Introduction to Horizon Scanning 2016
 

Recently uploaded

Decision on Curriculum Change Path: Towards Standards-Based Curriculum in Ghana
Decision on Curriculum Change Path: Towards Standards-Based Curriculum in GhanaDecision on Curriculum Change Path: Towards Standards-Based Curriculum in Ghana
Decision on Curriculum Change Path: Towards Standards-Based Curriculum in GhanaPrince Armah, PhD
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...Nguyen Thanh Tu Collection
 
Digital Footprints to Career Pathways - Building a Strong Professional Online...
Digital Footprints to Career Pathways - Building a Strong Professional Online...Digital Footprints to Career Pathways - Building a Strong Professional Online...
Digital Footprints to Career Pathways - Building a Strong Professional Online...Sue Beckingham
 
SSC Hindu Religion and Moral Education MCQ Solutions 2024.pdf
SSC Hindu Religion and Moral Education MCQ Solutions 2024.pdfSSC Hindu Religion and Moral Education MCQ Solutions 2024.pdf
SSC Hindu Religion and Moral Education MCQ Solutions 2024.pdfMohonDas
 
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?Can Brain Science Actually Help Make Your Training & Teaching "Stick"?
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?Aggregage
 
Kartik Nair In Media Res Media Component
Kartik Nair In Media Res Media ComponentKartik Nair In Media Res Media Component
Kartik Nair In Media Res Media ComponentInMediaRes1
 
Discussing the new Competence Framework for project managers in the built env...
Discussing the new Competence Framework for project managers in the built env...Discussing the new Competence Framework for project managers in the built env...
Discussing the new Competence Framework for project managers in the built env...Association for Project Management
 
ADAPTABILITY, Types of Adaptability AND STABILITY ANALYSIS method.pptx
ADAPTABILITY, Types of Adaptability AND STABILITY ANALYSIS  method.pptxADAPTABILITY, Types of Adaptability AND STABILITY ANALYSIS  method.pptx
ADAPTABILITY, Types of Adaptability AND STABILITY ANALYSIS method.pptxAKSHAYMAGAR17
 
11 CI SINIF SINAQLARI - 9-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 9-2023-Aynura-Hamidova.pdf11 CI SINIF SINAQLARI - 9-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 9-2023-Aynura-Hamidova.pdfAynouraHamidova
 
Brochure-Project-tryve-elementary-dept..docx
Brochure-Project-tryve-elementary-dept..docxBrochure-Project-tryve-elementary-dept..docx
Brochure-Project-tryve-elementary-dept..docxAnnaLizaTadeo1
 
Add Products From Catalog in Odoo 17 Sales
Add Products From Catalog in Odoo 17 SalesAdd Products From Catalog in Odoo 17 Sales
Add Products From Catalog in Odoo 17 SalesCeline George
 
Google Ad Grants Services at TechSoup.pdf
Google Ad Grants Services at TechSoup.pdfGoogle Ad Grants Services at TechSoup.pdf
Google Ad Grants Services at TechSoup.pdfTechSoup
 
spring_bee_bot_creations_erd primary.pdf
spring_bee_bot_creations_erd primary.pdfspring_bee_bot_creations_erd primary.pdf
spring_bee_bot_creations_erd primary.pdfKonstantina Koutsodimou
 
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdfAynouraHamidova
 
Healthy Habits for Happy School Staff - presentation
Healthy Habits for Happy School Staff - presentationHealthy Habits for Happy School Staff - presentation
Healthy Habits for Happy School Staff - presentationPooky Knightsmith
 
Andreas Schleicher_ Strengthening Upper Secondary Education in Lithuania
Andreas Schleicher_ Strengthening Upper Secondary  Education in LithuaniaAndreas Schleicher_ Strengthening Upper Secondary  Education in Lithuania
Andreas Schleicher_ Strengthening Upper Secondary Education in LithuaniaEduSkills OECD
 
2.27.24 Malcolm X and the Black Freedom Struggle.pptx
2.27.24 Malcolm X and the Black Freedom Struggle.pptx2.27.24 Malcolm X and the Black Freedom Struggle.pptx
2.27.24 Malcolm X and the Black Freedom Struggle.pptxMaryPotorti1
 
Odontogenesis and its related anomiles.pptx
Odontogenesis and its related anomiles.pptxOdontogenesis and its related anomiles.pptx
Odontogenesis and its related anomiles.pptxMennat Allah Alkaram
 
Practical Research 1, Lesson 5: DESIGNING A RESEARCH PROJECT RELATED TO DAILY...
Practical Research 1, Lesson 5: DESIGNING A RESEARCH PROJECT RELATED TO DAILY...Practical Research 1, Lesson 5: DESIGNING A RESEARCH PROJECT RELATED TO DAILY...
Practical Research 1, Lesson 5: DESIGNING A RESEARCH PROJECT RELATED TO DAILY...Katherine Villaluna
 

Recently uploaded (20)

Decision on Curriculum Change Path: Towards Standards-Based Curriculum in Ghana
Decision on Curriculum Change Path: Towards Standards-Based Curriculum in GhanaDecision on Curriculum Change Path: Towards Standards-Based Curriculum in Ghana
Decision on Curriculum Change Path: Towards Standards-Based Curriculum in Ghana
 
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...
BÀI TẬP BỔ TRỢ TIẾNG ANH 11 THEO ĐƠN VỊ BÀI HỌC - CẢ NĂM - CÓ FILE NGHE (GLOB...
 
Digital Footprints to Career Pathways - Building a Strong Professional Online...
Digital Footprints to Career Pathways - Building a Strong Professional Online...Digital Footprints to Career Pathways - Building a Strong Professional Online...
Digital Footprints to Career Pathways - Building a Strong Professional Online...
 
SSC Hindu Religion and Moral Education MCQ Solutions 2024.pdf
SSC Hindu Religion and Moral Education MCQ Solutions 2024.pdfSSC Hindu Religion and Moral Education MCQ Solutions 2024.pdf
SSC Hindu Religion and Moral Education MCQ Solutions 2024.pdf
 
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?Can Brain Science Actually Help Make Your Training & Teaching "Stick"?
Can Brain Science Actually Help Make Your Training & Teaching "Stick"?
 
Kartik Nair In Media Res Media Component
Kartik Nair In Media Res Media ComponentKartik Nair In Media Res Media Component
Kartik Nair In Media Res Media Component
 
Discussing the new Competence Framework for project managers in the built env...
Discussing the new Competence Framework for project managers in the built env...Discussing the new Competence Framework for project managers in the built env...
Discussing the new Competence Framework for project managers in the built env...
 
ADAPTABILITY, Types of Adaptability AND STABILITY ANALYSIS method.pptx
ADAPTABILITY, Types of Adaptability AND STABILITY ANALYSIS  method.pptxADAPTABILITY, Types of Adaptability AND STABILITY ANALYSIS  method.pptx
ADAPTABILITY, Types of Adaptability AND STABILITY ANALYSIS method.pptx
 
11 CI SINIF SINAQLARI - 9-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 9-2023-Aynura-Hamidova.pdf11 CI SINIF SINAQLARI - 9-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 9-2023-Aynura-Hamidova.pdf
 
Brochure-Project-tryve-elementary-dept..docx
Brochure-Project-tryve-elementary-dept..docxBrochure-Project-tryve-elementary-dept..docx
Brochure-Project-tryve-elementary-dept..docx
 
Add Products From Catalog in Odoo 17 Sales
Add Products From Catalog in Odoo 17 SalesAdd Products From Catalog in Odoo 17 Sales
Add Products From Catalog in Odoo 17 Sales
 
Google Ad Grants Services at TechSoup.pdf
Google Ad Grants Services at TechSoup.pdfGoogle Ad Grants Services at TechSoup.pdf
Google Ad Grants Services at TechSoup.pdf
 
spring_bee_bot_creations_erd primary.pdf
spring_bee_bot_creations_erd primary.pdfspring_bee_bot_creations_erd primary.pdf
spring_bee_bot_creations_erd primary.pdf
 
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf
11 CI SINIF SINAQLARI - 10-2023-Aynura-Hamidova.pdf
 
CLUBE PERLINGUAS .
CLUBE PERLINGUAS                        .CLUBE PERLINGUAS                        .
CLUBE PERLINGUAS .
 
Healthy Habits for Happy School Staff - presentation
Healthy Habits for Happy School Staff - presentationHealthy Habits for Happy School Staff - presentation
Healthy Habits for Happy School Staff - presentation
 
Andreas Schleicher_ Strengthening Upper Secondary Education in Lithuania
Andreas Schleicher_ Strengthening Upper Secondary  Education in LithuaniaAndreas Schleicher_ Strengthening Upper Secondary  Education in Lithuania
Andreas Schleicher_ Strengthening Upper Secondary Education in Lithuania
 
2.27.24 Malcolm X and the Black Freedom Struggle.pptx
2.27.24 Malcolm X and the Black Freedom Struggle.pptx2.27.24 Malcolm X and the Black Freedom Struggle.pptx
2.27.24 Malcolm X and the Black Freedom Struggle.pptx
 
Odontogenesis and its related anomiles.pptx
Odontogenesis and its related anomiles.pptxOdontogenesis and its related anomiles.pptx
Odontogenesis and its related anomiles.pptx
 
Practical Research 1, Lesson 5: DESIGNING A RESEARCH PROJECT RELATED TO DAILY...
Practical Research 1, Lesson 5: DESIGNING A RESEARCH PROJECT RELATED TO DAILY...Practical Research 1, Lesson 5: DESIGNING A RESEARCH PROJECT RELATED TO DAILY...
Practical Research 1, Lesson 5: DESIGNING A RESEARCH PROJECT RELATED TO DAILY...
 

Judgement: Managing the Cognitive Factor to Improve the accuracy of Technology Foresight

  • 1. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED. JUDGEMENT: MANAGING THE COGNITIVE FACTOR TO IMPROVE THE ACCURACY OF TECHNOLOGY FORESIGHT 4th Workshop Military University of Technology 04 - 2019 Pawel Fleischer President Institute for Forecasting and International Studies
  • 2. Contents © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED. 1. BIBLIOGRAPHY 2. THE ABSOLUTE WORST TECHNOLOGY PREDICTIONS 3. COGNITIVE BIASES 4. GOOD JUDGEMENT PROJECT 1. 4.1 GJP – EXAMPLES 2. 4.3 GJP - RESULTS 3. 4.4 BIASES IN FORECASTING 5. LL'S FOR TECHNOLOGY FORESIGHT?
  • 3. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED. Bibliography • Exploring the cognitive value of technology foresight: The case of the Cisco Technology Radar, Siri Boe-Lillegraven, Stephan Monterde, • Expert forecast and realized outcomes in technology foresight, Riccardo Apredaa, Andrea Bonaccorsib, Felice dell'Orlettac, Gualtiero Fantonib, • Strategic foresight in corporate organizations: Handling the effect and response uncertainty of technology and social drivers of change, Riccardo Vecchiato, Claudio Roveda • Pitfalls in and success factors of corporate foresight projects, M. Atilla Öner, Senem Göl, Int. J. Foresight and Innovation Policy, Vol. 3, No. 4, 2007 • UNIDO TECHNOLOGY FORESIGHT MANUAL, Volume 1 - Organization and Methods • Evidence on good forecasting practices from the Good Judgment Project: an accompanying blog post • Making the Best Use of Judgmental Forecasting, Shayne Kavanagh, Daneil Willaims, Government Finance Review • Superforecasting: The Art and Science of Prediction, Philip E. Tetlock, Dan Gardner
  • 4. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED. The absolute worst technology predictions 1936: “A rocket will never be able to leave the Earth’s atmosphere.” —New York Times. 1977: “There is no reason for any individual to have a computer in his home.” —Ken Olsen, founder of Digital Equipment Corp. 1989: “We will never make a 32-bit operating system.”  —Bill Gates, co-founder and chairman of Microsoft. 2003: “The subscription model of buying music is bankrupt. I think you could make available the Second Coming in a subscription model, and it might not be successful.” —Steve Jobs, in Rolling Stone
  • 5. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED. Cognitive Biases
  • 6. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED.
  • 7. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED. Good Judgement Project - examples • Will the UN Security Council adopt more than one resolution concerning the Democratic People's Republic of Korea between 8 March 2018 and 31 May 2018? • Before 8 September 2018, will Poland, Estonia, Latvia, or Lithuania accuse Russia of intervening militarily in its territory without permission? • Before 1 August 2018, will the Moroccan government and the Polisario Front meet for official negotiations over Western Sahara? • Will Turkish President Recep Tayyip Erdoğan experience a significant leadership disruption by 31 August 2018? • How much crude oil will Libya produce in May 2018?
  • 8. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED. Good Judgement Project – results • „In year 1 GJP beat the official control group by 60%. In year 2, beat the control group by 78%. GJP also beat its university-affiliated competitors, including the University of Michigan and MIT, by hefty margins, from 30% to 70%.” • “The Good Judgment Project outperformed a prediction market inside the intelligence community, which was populated with professional analysts who had classified information, by 25 or 30 percent” • “Teams of ordinary forecasters beat the wisdom of the crowd by about 10%. Prediction markets beat ordinary teams by about 20%. And [teams of superforecasters] beat prediction markets by 15% to 30%.” • On average, teams were 23% more accurate than individuals.
  • 9. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED. Good Judgement Project – Biases in Forecasting Type of Bias Description of Bias Example Overconfidence Unjustified belief in the accuracy of forecast assumptions Being too certain about the potential future behavior of key revenue drivers Conservatism Failing to change one’s mind in light of new information Ignoring new, less favorable information on the financial impact of a new development project “Recency” Allowing recent events to dominate over events further in the past Allowing news headlines of recent economic events to obscure larger, long-term trends Availability Relying on specific events that are easily recalled from memory at the expense of other pertinent information Making issues that are particularly memorable and may have received a lot of political attention the focus of a forecast, even though other issues were more important Anchoring Being unduly influenced by initial informatio, which is given more weight in forecasting proces Using a historical high or low as the starting point of discussions for a long-term, multi-year forecast
  • 10. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED. Good Judgement Project – Biases in Forecasting Type of Bias Description of Bias Example Search for Supportive Evidence Gathering facts that lead to certain conclusions and disregarding contradictory information Consulting only sources of economic analysis that support the outcome that the forecaster wants Underestimating Uncretainty Underestimating future uncertainty n order to reduce anxiety Showing a point forecast or single forecast as the only future outcome and not accounting for possible variance Selective Perception Seeing problems in terms of one’s own background and expertise Considering only the forecast variables that the forecaster knows the most about
  • 11. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED. Are there LL's for technology foresight? Reduce congnitive biases  by playing video gamesby watching instructional videos
  • 12. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED. Are there LL's for technology foresight? • 10-15% boost from screening forecasters on fluid intelligence/ active open- mindedness • 10-20% boost from collaborative work in teams or competitvely in prediction markets • 10% boost from cognitive debiasing exercises • 15-30% boost from more weight to better forecasters and then „extremize” to compensate for conservatism of aggregate forecasts.
  • 13. © INSTITUTE FOR FORECASTING AND INTERNATIONAL STUDIES ALL RIGHTS RESERVED.