How to Reduce Analytical Bias by Applying Analysis of Competing Hypotheses (ACH)
•
5 likes•2,990 views
This document summarizes a webinar about reducing analytical bias in competitive intelligence by applying Analysis of Competing Hypotheses (ACH). It introduces ACH as a technique developed by a CIA veteran to overcome cognitive biases. The webinar discusses how ACH follows a structured, logical process and allows collaboration. Examples are provided of using ACH to analyze scenarios around the Chinese economy and decisions facing a school district.
Recommended
More Related Content
Viewers also liked
Viewers also liked (20)
Similar to How to Reduce Analytical Bias by Applying Analysis of Competing Hypotheses (ACH)
Similar to How to Reduce Analytical Bias by Applying Analysis of Competing Hypotheses (ACH) (20)
More from IntelCollab.com
More from IntelCollab.com (20)
Recently uploaded
Recently uploaded (20)
How to Reduce Analytical Bias by Applying Analysis of Competing Hypotheses (ACH)
- 1. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby How to Reduce Analytical Bias by Applying Analysis of Competing Hypotheses (ACH) A Complimentary Webinar from Aurora WDC 12:00 Noon Eastern /// 24 July 2013 ~ featuring ~ Dan Mulligan Michel Bernaiche
- 2. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Dan Mulligan Dan Mulligan is on the faculty of the Academy of Competitive Intelligence (ACI) and has presented at SCIP conferences every year since 2007. At the 2011 SCIP Conference in Orlando, Florida, he was recognized with the “Best Conference Presentation”. His corporate experience includes work in the medical, pharmaceutical, manufacturing, and consulting industries. Through the Dale Carnegie organization, he trained, taught, and consulted with over 200 of the Fortune 500 companies over the past 19 years, focusing on leadership and communications. Email: DanMulligan82@gmail.com The Intelligence Collaborative is the online learning and networking community powered by Aurora WDC, our clients, partners and other friends and dedicated to exploring how to apply intelligence methods to solve real- world business problems. Apply for a free 30-day trial membership at http://IntelCollab.com or learn more about Aurora at http://AuroraWDC.com – see you next time!
- 3. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby α Use the Questions pane on your GoToWebinar control panel and all questions will be answered in the second half of the hour. α You are welcome to tweet any comments on Twitter where we are monitoring the hashtag #IntelCollab or eavesdrop via http://tweetchat.com/room/IntelCollab α Slides will be available after the webinar for embedding and sharing via http://slideshare.net/IntelCollab α To view the recording and download the PPT file, please register for a trial membership at http://IntelCollab.com. Questions, Commentary & Content
- 4. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Agenda ► Competitive Intelligence Process ► Analytic problems in CI ► Analysis of Competing Hypotheses (ACH) ► Examples ► Summary, Q&A and Discussion
- 5. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Purpose of Intelligence
- 6. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby How?
- 7. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Information and Intelligence 7Introduction to Intelligence Information Intelligence Scattered Bits of Data Evaluated, Filtered, Distilled, Analyzed Information Unrefined End Product of a Complex Process Raw & Unfinished Basis for Action
- 8. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby The Intelligence Process 8
- 9. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby KITs + 1 ►Aurora and Dr. Craig Fleisher ►“KITs + 1: The Next Generation of Intelligence Analysis” ►http://aurorawdc.com/videos/
- 10. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby INESCAPABLE DILEMMA
- 11. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 12. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 13. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Blindspots are caused by faulty internal or external assumptions.
- 14. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 15. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby WHY BUSINESS SCHOOLS MAKE IT WORSE
- 16. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Specialists Accounting Banking Business Statistics Marketing Finance Business Law Economics Operations Strategy Management
- 17. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 18. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Competitive Intelligence
- 19. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Analysis Of Competing Hypotheses ►Richards J. Heuer Jr. ►45yr. CIA veteran ►Developed ACH after studying cognitive psychology Richards J. Heuer Jr. Photo courtesy the Center for the Study of Intelligence
- 20. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Analysis Of Competing Hypotheses Cognitive Biases Analysis Of Competing Hypotheses Scientific Method
- 21. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Analysis Of Competing Hypotheses • Reduces cognitive biases • Logical • Auditable Strengths • Snapshot in time • Dependent on validity of evidence • Time consuming Weaknesses
- 22. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Clear Estimate? Simple Hypotheses Test Target Collection Increase Complexity YesNo Structured Analysis Of Competing Hypotheses
- 23. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby • Easy to think, write, collaborate • Easy to present • Easy to defend • Easy to supervise Structured Analysis Of Competing Hypotheses
- 24. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Free ACH Software
- 25. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby PARC Software ►Developed by the Palo Alto Research Center (PARC) ►ACH2.0.3 ►Developed in collaboration with Richards J. Heuer, Jr. D d http://www2.parc.com/istl/projects/ach/ach.html
- 26. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby http://www2.parc.com/istl/projects/ach/ach.html
- 27. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 28. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 29. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 30. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 31. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 32. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 33. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Chinese Economy
- 34. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 35. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby SWOT Strengths Opportunities Weaknesses Threats
- 36. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 37. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Erie School District
- 38. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 39. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 40. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby
- 41. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Analysis Of Competing Hypotheses • Reduces cognitive biases • Logical • Defensible • Great for Collaboration • Auditable
- 42. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Richards Heuer’s ACH ►https://www.cia.gov/library/center-for-the- study-of-intelligence/csi- publications/books-and- monographs/psychology-of-intelligence- analysis/art11.html
- 43. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby ►Robert Steel ►Vice Chairman, Goldman Sachs ►“Information costs money.” ►“Intelligence makes money.” Herring (2006)
- 44. The Intelligence Collaborative http://IntelCollab.com #IntelCollab Poweredby Dan Mulligan Thank you! Now how about a little Q&A? Email: DanMulligan82@gmail.com The Intelligence Collaborative is the online learning and networking community powered by Aurora WDC, our clients, partners and other friends and dedicated to exploring how to apply intelligence methods to solve real- world business problems. Apply for a free 30-day trial membership at http://IntelCollab.com or learn more about Aurora at http://AuroraWDC.com – see you next time!
Editor's Notes
- Richards Heuer is the author of Psychology of Intelligence Analysis. Heuer spent 45 years gaining experience in intelligence and security working for the Central Intelligence Agency (CIA). In addition to publishing work on the psychology of intelligence analysis, he also published works on counterintelligence, deception, analytical methodologies, and personnel security issues.
- The scientific method is a proven process based on the rejection of hypotheses in which the analyst accepts only those which are not disproven. Traditional intelligence analysis does not seek to refute hypotheses, but tends to seek confirmation of initial judgments. Cognitive biases are mental errors caused by simplifying information processing strategies. Emotional or intellectual predispositions toward certain judgments do not cause cognitive biases; rather, cognitive biases are the result of subconscious mental shortcuts for processing information.Analysis of Competing Hypotheses ACH assists analysts in identifying and questioning basic assumptions providing a means to test hypotheses. ACH is an eight-step procedure that allows analysts to leave an audit trail, unlike other decision-making processes. In order to prevent distortions caused by cognitive biases, Heuer’s method requires analysts to explicitly identify all reasonable hypotheses. Hypotheses compete for the analyst’s favor, eliminating satisficing because the analyst is working with all possible rational hypotheses. Typically, analysts identify what appears to be the most promising hypothesis before collecting and organizing information based on whether it supports or refutes the hypothesis the analyst has chosen. Often incongruent information is not fully exploited, nor are other hypotheses tested. When testing one hypothesis at a time, biases and mindsets are harder to dispel.
- AuditableWith ACH, analysts note all evidence and hypotheses on paper, making the path from collection to final estimate easier to follow.Overcomes cognitive biasesIndividually analyzing each piece of evidence can give analysts a skewed view of the overall picture. Using ACH, disproving hypotheses provides a more effective method for avoiding established mental roadblocks.LogicalThe competitive aspect of ACH forces analysts to examine hypotheses more logically than separating them for reviewing one at a time.ACH is similar to Multi-Criteria Decision Making (MCDM).Although ACH is more effective than less-systematic, traditional approaches, no process is without drawbacks.Snapshot in timeThe evidence used in the analysis is static, while events are dynamic. In order to provide a timely analytic product, analysts must determine a point at which to halt collection in order to prepare the estimate. Dependent on validity of evidenceAnalysis is only as good as the evidence that goes into the process. Unreliable evidence can undermine even the most thoughtful process execution. As evidence can be ambiguous and prone to the pitfalls of deceptive tactics, it is difficult to determine reliability and validity of the inputs. No better than the skill of the analyst using the processThe degree to which ACH can reduce cognitive biases reflects the skill of the analyst using it. A successful analyst must be imaginative when generating hypotheses, keeping his mind open to all possibilities, rather than attempting to anticipate an outcome before examining the evidence. He must also carefully weigh the reliability and relevance of the evidence.Time consumingPerforming the ACH process manually can be long and arduous, especially when there are numerous hypotheses and a large body of evidence. Due to the amount of time that it takes to do this by hand and the time pressures analysts usually encounter, they do not apply the method as often as might be appropriate. For this reason, automation of ACH is critical and will be discussed in the next lesson.
- Structure is provided by delving deeper into the previous hypothesis and further exploiting the evidence, or gathering more.Automation, and structure with PARC Software facilitates the analysis of the second level of hypothesis testingIn other words, structure enables analysis of complex problems consisting of several questions by deconstructing the complex question into simple questions with either yes, no answers or mutually exclusive answers. Then ACH is applied to the root question, and the remaining questions are added to the winning hypothesis and ACH is applied again. (Added by NSA, 1Sep06)
- Using SACH eases the process for analysts from start to finish. SACH closes evidence gaps, creating a tighter finished product.Documenting every step in SACH later helps analysts easily follow the estimative process. Because SACH provides a trail of auditable information, should the analyst provide an incorrect estimate, it is easier to evaluate what went wrong.In conclusion:SACH allows analysts to coordinate which evidence to test with the hypotheses the analyst is testing. It also allows the analyst to deal more effectively with large amounts of evidence.SACH assists managers in supervising and monitoring analysts’ progress.SACH provides decisionmakers with higher-quality products, and makes it easier for them to see exactly how the analyst arrived at the final estimate. When presenting to decisionmakers, analysts will more effectively produce documents with the bottom lines up front (BLUF) because the automation of SACH helps analysts discover and prove the bottom line.SACH improves oversight, as products are more easily defendable by clearly tracing all thought processes back to the original evidence.
- The software takes an analyst through a process for making a well-reasoned, analytical judgment. Usefulfor issues that require weighing of alternative explanations of what has/is/or may happenIt helps the analyst overcomesome cognitive limitations that make prescient intelligence analysis difficult. ACH is grounded in psychology, decision analysis, and the scientific method. It helps analysts protect themselves from avoidable errorImproves their chances of making a correct judgment
- Analysis of Competing HypothesesClassic Intelligence Techniques
- AuditableWith ACH, analysts note all evidence and hypotheses on paper, making the path from collection to final estimate easier to follow.Overcomes cognitive biasesIndividually analyzing each piece of evidence can give analysts a skewed view of the overall picture. Using ACH, disproving hypotheses provides a more effective method for avoiding established mental roadblocks.LogicalThe competitive aspect of ACH forces analysts to examine hypotheses more logically than separating them for reviewing one at a time.ACH is similar to Multi-Criteria Decision Making (MCDM).Although ACH is more effective than less-systematic, traditional approaches, no process is without drawbacks.Snapshot in timeThe evidence used in the analysis is static, while events are dynamic. In order to provide a timely analytic product, analysts must determine a point at which to halt collection in order to prepare the estimate. Dependent on validity of evidenceAnalysis is only as good as the evidence that goes into the process. Unreliable evidence can undermine even the most thoughtful process execution. As evidence can be ambiguous and prone to the pitfalls of deceptive tactics, it is difficult to determine reliability and validity of the inputs. No better than the skill of the analyst using the processThe degree to which ACH can reduce cognitive biases reflects the skill of the analyst using it. A successful analyst must be imaginative when generating hypotheses, keeping his mind open to all possibilities, rather than attempting to anticipate an outcome before examining the evidence. He must also carefully weigh the reliability and relevance of the evidence.Time consumingPerforming the ACH process manually can be long and arduous, especially when there are numerous hypotheses and a large body of evidence. Due to the amount of time that it takes to do this by hand and the time pressures analysts usually encounter, they do not apply the method as often as might be appropriate. For this reason, automation of ACH is critical and will be discussed in the next lesson.