Uncertainty, variability and weight of evidence
โข
1 likeโข300 views
Presentation of the EFSA's second scientific conference, held on 14-16 October 2015 in Milan, Italy. WEIGHING EVIDENCE AND ASSESSING UNCERTAINTY
Recommended
Recommended
More Related Content
Similar to Uncertainty, variability and weight of evidence
Similar to Uncertainty, variability and weight of evidence (20)
More from EFSA EU
More from EFSA EU (20)
Recently uploaded
Recently uploaded (20)
Uncertainty, variability and weight of evidence
- 1. Uncertainty, Variability and Weight of Evidence: How Well Do We Know Environmental Risks? Glenn Suter U.S. Environmental Protection Agency EFSA Scientific Conference, Milan, Italy The views expressed in this presentation are those of the author and do not necessarily represent the views or policies of the U.S. Environmental Protection Agency
- 2. What do decision makers & stakeholders need? โข Qualitative information โข Does this chemical cause cancer? โข Is this ecosystem impaired? โข What is the cause of observed effects? โข Quantitative information โข What concentration is protective? โข How much salmon habitat would be lost? โข How long will natural remediation require? โข Confidence in both
- 4. What is Scatter? Anything expressed as a numerical distribution โข Variability โข natural variation in some quantity โข Cannot be reduced โข Uncertainty โข Incomplete or imprecise data โข Can be reduced โข Degree of belief โข Subsumes variability and uncertainty for subjectivists โข If you were betting on the teratogenicity of a chemical in use, What odds would you accept? This is statisticsโcontroversial but familiar
- 5. What is weight of evidence? โข Metaphor based on the scales of justice โข The influence evidence should have on an inference โข The weight of a piece of evidence dropped in a pan โข A summary interpretation of multiple pieces of evidence โข The weight of the body of evidence in one pan relative to the other โข An inferential approach that uses this concept โข Often simply an expert narrative โข We are trying to make it more formal, transparent and defensible
- 6. How do we weigh evidence? Assemble Evidence Evaluate and Weight Evidence Integrate and Weigh Body of Evidence
- 7. Assembling evidence Assemble Evidence Screen Design and Conduct Studies Search Literature Extract and Analyze Categorize Weight Evidence Weigh Body of Evidence
- 8. Assembling evidence Assemble Evidence Screen Design and Conduct Studies Search Literature Extract and Analyze Categorize Weight Evidence Weigh Body of Evidence Systematic Review thorough and unbiased
- 9. Assembling evidence Assemble Evidence Screen Design and Conduct Studies Search Literature Extract and Analyze Categorize Weight Evidence Weigh Body of Evidence Systematic Review Complement existing evidence to create a coherent body of evidence
- 10. Assembling evidence Assemble Evidence Screen Design and Conduct Studies Search Literature Extract and Analyze Categorize Weight Evidence Weigh Body of Evidence Systematic Review Complement existing Evidence to create a coherent body of evidence Eliminate irrelevant and unacceptable studies
- 11. Assembling evidence Assemble Evidence Screen Design and Conduct Studies Search Literature Extract and Analyze Categorize Weight Evidence Weigh Body of Evidence Systematic Review Complement existing Evidence to create a coherent body of evidence Eliminate irrelevant and unacceptable studies Put the studies in bins that may be treated as a type of evidence
- 12. Assembling evidence Assemble Evidence Screen Design and Conduct Studies Search Literature Extract and Analyze Categorize Weight Evidence Weigh Body of Evidence Systematic Review Complement existing evidence to create a coherent body of evidence Eliminate irrelevant and unacceptable studies Put the studies in bins that may be treated as a type of evidenceExtract the data, reanalyze, and combine with other information to create evidence
- 13. Weighting evidence Weight Evidence Combine Weights Weight Reliability Weight Relevance Weight Strength Assemble Evidence Weigh Body of Evidence Evaluate with respect to 3 attributes and assign scores
- 14. Weighting evidence Weight Evidence Combine Weights Weight Reliability Weight Relevance Weight Strength Assemble Evidence Weigh Body of Evidence Evaluate with respect to 3 attributes and assign scores Combine into weight of piece or type of evidence
- 15. Weighing bodies of evidence Weigh Body of Evidence Interpret Bodies of Evidence Explain Ambiguities & Discrepancies Integrate Pieces or Categories of Evidence Weight Evidence Assemble Evidence Determine the overall weight of the body of evidence based on the weights of pieces, or types of evidence and consistency
- 16. Weighing bodies of evidence Weigh Body of Evidence Interpret Bodies of Evidence Explain Ambiguities & Discrepancies Integrate Pieces or Categories of Evidence Weight Evidence Assemble Evidence Which, if any, hypothesis is supported by the weight of evidence and with how much weight? Determine the overall weight of the body of evidence based on the weights of pieces, or types of evidence and consistency
- 17. Weighing bodies of evidence Weigh Body of Evidence Interpret Bodies of Evidence Explain Ambiguities & Discrepancies Integrate Pieces or Categories of Evidence Weight Evidence Assemble Evidence Which, if any, hypothesis is supported by the weight of evidence and with how much weight? Determine the overall weight of the body of evidence based on the weights of pieces, or types of evidence and consistency Deal with cases in which the evidence does not support any hypothesis
- 18. Expressing qualitative results โข Example Conclusion The chemical is a vertebrate teratogen โข Example Weight High relevance Moderate strength Moderate reliability High consistency
- 19. Linking qualitative to quantitative results Summary of previous multicolored boxes Weigh Evidence for the Quality to be Quantified Merge Quantitative Evidence Choose Best Quantitative Evidence Define Confidence in Quantitative Evidence
- 20. Linking qualitative to quantitative results Weighted mean or other meta- analysis Summary of previous multicolored boxes Weigh Evidence for the Quality to be Quantified Merge Quantitative Evidence Choose Best Quantitative Evidence Define Confidence in Quantitative Evidence
- 21. Linking qualitative to quantitative results Weighted mean or other meta- analysis Weigh evidence to choose best source of the quantity Summary of previous multicolored boxes Weigh Evidence for the Quality to be Quantified Merge Quantitative Evidence Choose Best Quantitative Evidence Define Confidence in Quantitative Evidence
- 22. Linking qualitative to quantitative results Weighted mean or other meta- analysis Weigh evidence to choose best source of the quantity Integrate results Summary of previous multicolored boxes Weigh Evidence for the Quality to be Quantified Merge Quantitative Evidence Choose Best Quantitative Evidence Define Confidence in Quantitative Evidence
- 23. Expressing quantitative results โข Quality Protection from deformities โข Number 22 โข Units mg/L โข Scatter ยฑ 3 โข Weight Moderate Each has supporting information and detail, but this is the basic information (tip of the hat to Ravetz and Funtowiczโs NUSAP system)
- 24. Summary โข Formal weighing of evidence can โข Provide greater utility of results โข Increase defensibility โข Increase transparency โข With flexibility and adaption to purpose, โข WoE need not be onerous โข In combination with conventional statistics โข WoE can convey confidence in results The views expressed in this presentation are those of the author and do not necessarily represent the views or policies of the U.S. Environmental Protection Agency