David V. Budescu

1. Communicating Uncertainty
with Probability Phrases
David V. Budescu
Fordham University, NYC, USA
IPCC Communication Meeting
Oslo, Feb 9-10, 2016
I

2. Translation Table for IPCC Authors
Phrase Likelihood Conveyed
Virtually certain > 99%
Very likely > 90%
Likely > 66%
About as likely as not 33% to 66%
Unlikely < 33%
Very unlikely < 10%
Exceptionally unlikely < 1%
5

3. Reservations
• The ranges of the terms are not mutually
exclusive
• The ranges associated with the terms are
arbitrary and are not related to people's
“natural” understanding of the language
• People prefer to receive numerical information
if available
• Large variability in interpretation of the terms

4. The 2014 Intl Study
• Read 8 statements extracted from the IPCC executive
summary containing 4 terms: very likely, likely, unlikely, and
very unlikely (2 statements for each term)
• Provide Best estimate and Lower and Upper bounds for each
term (as intended by authors)
• Presentation format: Translation or Dual (VN)
• 27 samples in 25 countries and 17 languages (~400 /country)
• All materials were translated and back translated by native
speakers
• Analyze n=10,239 in 25 samples (51% men; mean age = 41)
7

5. Translation Screen shot
8

6. Translation screen shot with
guidelines
9

7. Verbal-Numerical screen shot
10

8. Estimates are Regressive, but the VN
Presentation Makes Them More Extreme
12
0
20
40
60
80
100
T VN T VN T VN T VN
Very Unlikely
<10%
Unlikely
<33%
Likely
>66%
Very Likely
>90%
BestEstimates

9. Consistency with IPCC Guidelines Improves
(especially for extremes)
13
16.13
36.62 37.64
18.95
31.76
43.14
47.59
36.65
0
10
20
30
40
50
Very Unlikely Unlikely Likely Very Likely
%ofEstimatesComplyingwith
Guidelines
Overall Translation Overall VN

10. 21%
24%
25%
25%
25%
26%
27%
27%
27%
27%
27%
27%
27%
28%
28%
28%
28%
28%
29%
30%
31%
31%
31%
32%
33%
35%
28%
32%
43%
36%
41%
39%
37%
46%
38%
41%
37%
40%
43%
35%
43%
39%
40%
38%
44%
45%
43%
40%
48%
46%
38%
54%
India
Netherland
UK
Brazil
Hong Kong
Italy
Spain
Canada
Quebec
Sweden
Australia
Overall
Germany
Japan
Taiwan
Russia
Poland
France
Slovakia
USA
Chile
China
RSA
Turkey
Korea
Israel
% Compliance Translation % Compliance Verbal-Numeric
14

11. The Benefits of Multiple Modalities
• The dual scale was superior in several senses:
– Better differentiation and discrimination,
– Higher agreement with the prescribed meaning,
– More uniform interpretations across various groups
– It works for people with different preferences
• It makes communicators more mindful of the
terms’ intended meaning
• It can be used in more creative and flexible ways
to accommodate, and signal, different levels of
(im)precision in various cases

12. The Evidence Based Approach
• Identify the target audience
• Identify a subset of terms that are reasonable
candidates for inclusion in the lexicon
• Establish empirically how the target population
understands, interprets and uses these terms
(using single estimates, ranges, MFs)
• Use statistical methods to Identify optimal
cutoff points that archive the highest consensus
in interpretation in the target population

13. Evidence-Based Lexicons
Group Membership FunctionsDistribution of Estimates

14. Application to the IPCC
Sample Lexicon Consistency (%)
UK IPCC 27
Peak 44
Membership 50
Australia IPCC 25
Peak 41
Membership 45

15. Evidence-Based Lexicons
• There is a clear advantage to deriving evidence -
based lexicons over “intuitive” ones
• Such lexicons are constructed to take full advantage
of the common norms and meanings shared by a
majority of participants in the target group
• As such they are less sensitive to idiosyncratic
preferences of the committees that construct them
and they are easier to implement and maintain
• There are many possible methods one can use in this
context.

16. Evidence-Based Lexicons
• There is no single universal lexicon that serves
all purposes
• It makes sense to choose the size and
composition of the lexicon to
– Fit the needs of the target population
– Be sensitive to the precision and discrimination of
the information prevalent in the decision
environment
– Be sensitive to the probabilities of interest

17. Thank you
budescu@fordham.edu