The document discusses the Global Burden of Disease (GBD) study and methodology for assigning disability weights. It provides details on: - The GBD aims to measure disease burden globally in terms of years lived with disability (YLD) and disability-adjusted life years (DALYs). - Disability weights provide a scale from 0 (perfect health) to 1 (death) to quantify the severity of non-fatal health outcomes. - The study involved household surveys in multiple countries and an internet survey to derive weights for 220 health states based on paired comparisons of health state descriptions. - Statistical modeling was used to analyze the paired comparison data and map the results to a 0-1 disability weight

2. Salomon – Disability weights - 2
2
Global Burden of Disease Study
• Global Burden of Disease (GBD) Study aims to measure impact
of disease and injury in terms of losses in population health
• GBD 1990 study launched in 1991, updated by WHO in 2000s
• GBD 2010 study, undertaken from 2007-2012, provides first
comprehensive overhaul since 1996
• GBD quantifies
• Magnitude of different health problems in units of disability-adjusted
life years (DALYs)
• Overall population health in units of healthy life expectancy (HALE)

3. Salomon – Disability weights - 3
3
Disability weights
• Disability weights provide the bridge between mortality and
non-fatal outcomes in DALYs and in healthy life expectancy
• To measure health impact of non-fatal outcomes, GBD needs
weights for all unique sequelae, which capture the major
health consequences of all of the causes in the study
• Disability weights quantify severity of outcomes as percentage
reductions from perfect health, which are multiplied by years
lived in each sequela to give years lived with disability (YLD)
 e.g., If weight for blindness were 0.20, then 5 years lived with
blindness would be equivalent to dying one year prematurely

4. Salomon – Disability weights - 4
Intervention 2:
Prevent 1y of
deafness for
2000
individuals
Intervention 1:
Extend life by 1y in
1000
healthy individuals
Disability weights in the 1996 GBD revision
Person trade-off: which would you choose?
• Expert panel used ‘person
trade-off’ to assign values
to 22 indicator conditions

5. Salomon – Disability weights - 5
1
Disability weights
2 3 4 5 6 7
Class 1:
• Vitiligo on face
Class 4:
• Below-knee amputation
• Deafness
Class 7:
• Active psychosis
• Quadriplegia
Disability weights in the 1996 GBD revision
• Expert panel used ‘person
trade-off’ to assign values
to 22 indicator conditions
• These 22 conditions used
as operational definitions
of 7 disability classes
0.0 0.2 0.4 0.6 0.8 1.0

6. Salomon – Disability weights - 6
1
Disability weights
2 3 4 5 6 7
Rheumatoid arthritis cases
Average disability weight
=0.2*0.07 + 0.4*0.18 + 0.4*0.30
=0.21
Disability weights in the 1996 GBD revision
• Expert panel used ‘person
trade-off’ to assign values
to 22 indicator conditions
• These 22 conditions used
as operational definitions
of 7 disability classes
• Remaining conditions
allocated across classes to
compute average weights
0.0 0.2 0.4 0.6 0.8 1.0

7. Salomon – Disability weights - 7
7
Disability weights measurement study goals
• Derive weights for all 220 health states capturing nonfatal
outcomes from 291 disease and injury causes in GBD 2010
• Address criticisms of previous approaches by:
 Focusing on valuations from community respondents…
 … in a diverse range of settings
 … using suitable measurement methods
• Specific research aims
 Develop valid and reliable data collection tools for population-
based surveys
 Empirical examination of variation in weights

8. Salomon – Disability weights - 8
• Background
• Study design and methods
• Key findings
• Interpretations, limitations, conclusions
Disability weights measurement in GBD 2010

9. Salomon – Disability weights - 9
9
Study components
• Population-based household surveys
• Face-to-face interviews in Tanzania,
Bangladesh, Indonesia, Peru
• Telephone interview in random sample of
US households
• Focus on paired comparisons for 108
health-states
• Key objectives include comparative analysis
across diverse settings and benchmarking
Internet survey against community samples
• Open-access Internet surveys
• Available in English, Spanish and Mandarin
• Key objectives are to fill in gaps with
remaining sequelae and to anchor scale for
paired comparison responses
Bangladesh
Tanzania

10. Salomon – Disability weights - 10
Web survey
Web survey included 16,328 respondents from 167 countries

11. Salomon – Disability weights - 11
Web survey included 16,328 respondents from 167 countries
Web Responses
0
1 - 9
10 - 49
50 - 99
100 - 499
> 499
Web survey
1 - 9
10 - 49
50 - 99
100 - 499
500+

12. Salomon – Disability weights - 12
12
Measurement methods: paired comparisons
• Primary mode of eliciting responses is paired comparison
• Respondents hear (or read) two descriptions of hypothetical people,
each with a randomly selected condition
• Respondents indicate which person is healthier
• Paired comparison questions chosen for relative ease of
comprehension, administration and analysis
• Literacy and numeracy not essential
• Health comparisons not tied to external “calibrators” such as risk
• Appealing intuitive basis and established strategies for analysis

13. Salomon – Disability weights - 13
13
Framing paired comparisons
• Basis for all comparison are lay descriptions of sequelae, which
highlight major functional consequences and symptoms
associated with each sequela
• Must be brief: restricted to <35 words based on pretest results
• Must use simple, non-clinical vocabulary
• Prologue to paired comparison questions orients respondents
to focus on functioning
A person’s health may limit how well parts of his body or his mind work. As a result,
some people are not able to do all of the things in life that others may do, and some
people are more severely limited than others.
I am going to ask you a series of questions about different health problems. In each
question I will describe two different people … Imagine they have the same number
of years left to live, and will experience the health problems that I describe for the
rest of their lives. I will ask you to tell me which person you think is healthier overall,
in terms of having fewer physical or mental limitations on what they can do in life…

14. Salomon – Disability weights - 14
Paired comparison example
The first person has vision problems that make it difficult to see and recognize faces
of family or friends across a room.
The second person has severe back and leg pain, which causes difficulty dressing,
sitting, standing, walking, and lifting things. The person sleeps poorly and feels
worried.
Imagine that both people will have these problems for the rest of their lives. Who
would you say is healthier overall, the first person or the second person?
• Respondents in household surveys each answered 15 of these
questions, with pairs of sequelae drawn at random from the
universe of 108 108 possible pairs

15. Salomon – Disability weights - 15
15
Analyzing paired comparison data
Bangladesh Tanzania
• Simple ordering of
outcomes based on
“winning” proportions, as
in various websites using
paired comparisons to rank
large pools of competitors
http://kittenwar.com/
home
winningest kittens
losingest kittens
newest kittens
add your kitten
facebook group
kittenwar myspace
faq
e-mail us
kitten search:
Go
48% of people
agree that Misty is
cuter than Ling.
Prada Von Fluffy Doom Cheshire
Click the cutest kitten picture!
Can't decide? Refresh the page for a draw.

16. Salomon – Disability weights - 16
16
Analyzing paired comparison data
Bangladesh Tanzania
Won 78% of 511 battles Won 77% of 539 battles
Winningest
Lost 80% of 846 battles Lost 80% of 1872 battles
Losingest
• Simple ordering of
outcomes based on
“winning” proportions, as
in various websites using
paired comparisons to rank
large pools of competitors
http://kittenwar.com/

17. Salomon – Disability weights - 17
17
Analyzing paired comparison data
Bangladesh Tanzania
• Simple ordering of
outcomes based on
“winning” proportions, as
in various websites using
paired comparisons to rank
large pools of competitors
• More sophisticated
algorithms incorporate
information on the relative
strength of the opponent
(e.g. Elo rating system in
Chess)
• Statistical approaches
enable maximum
likelihood estimation of
underlying scores

18. Salomon – Disability weights - 18
18
Statistical modeling of paired comparisons data
• Conceptual foundation for statistical modeling of paired comparisons data
comes from Thurstone (1927), through Luce (1959), McFadden (1974) and
others
• Intuitively, a pair of health states that are similar in severity are likely to
produce greater disagreement over which is healthier than a pair of states
that are different in severity
• Consider a population in which:
 70% of people think depression is worse than blindness
 90% of people think blindness is worse than arthritis
• Reasonable to conclude that depression and blindness are nearer on some
unobserved scale than blindness and arthritis

19. Salomon – Disability weights - 19
-4 -2 0 2 4 6 8
blindness
90%

20. Salomon – Disability weights - 20
-4 -2 0 2 4 6 8
depression
blindness
90%70%

21. Salomon – Disability weights - 21
-4 -2 0 2 4 6 8
depression
blindness
90%

22. Salomon – Disability weights - 22
-4 -2 0 2 4 6 8
depression
blindness
arthritis
90%

23. Salomon – Disability weights - 23
-4 -2 0 2 4 6 8
depression
blindness
arthritis

24. Salomon – Disability weights - 24
24
Statistical modeling of paired comparisons data
• Statistical model formalizes this intuition
 Each health state i has an unobserved actual health level Xi as perceived by an
individual rater
 The rater makes a choice in the paired comparison based on which of the
health states is regarded as ‘healthier’
State 1 is chosen over State 2 if X1 > X2
• Thurstone model assumes that Xi is normally distributed, implying that the
choice probabilities are also normal
• We can model response probabilities using probit regression
 Requires arbitrary identifying assumptions
 Assumes that each health state has the same variance around the disability
weight

25. Salomon – Disability weights - 25
From paired comparisons to disability weights
Analysis of paired
comparisons
Analysis of “population
health equivalence”
responses
Rescaling of paired
comparison results
47

26. Salomon – Disability weights - 26
From paired comparisons to disability weights
Analysis of paired
comparisons
Analysis of “population
health equivalence”
responses
Rescaling of paired
comparison results
• Pooled data from all household
surveys, CATI and web survey
• Estimated probit model with separate
variance for each survey

27. Salomon – Disability weights - 27
From paired comparisons to disability weights
Analysis of paired
comparisons
Analysis of “population
health equivalence”
responses
Rescaling of paired
comparison results
• Some Web survey respondents
randomly assigned to “population
health equivalence” (PHE)
questions, for subset of 30 sequelae
• PHE provides separate estimates for
these sequelae that are anchored on
(0,1) disability scale
49

28. Salomon – Disability weights - 28
From paired comparisons to disability weights
Analysis of paired
comparisons
Analysis of “population
health equivalence”
responses
Rescaling of paired
comparison results
• Probit scale mapped to (0,1) disability
scale by regressing probit coefficients
on PHE anchors
• Non-constant variance accommodated
by regression on logit-transformed
PHE values, then back-transforming
values into (0,1) space
50

29. Salomon – Disability weights - 29
• Background
• Study design and methods
• Key findings
• Interpretations, limitations, conclusions
Disability weights measurement in GBD 2010

30. Salomon – Disability weights - 30
Results: paired comparison responses
• Probabilities of responses
on paired comparisons
(‘Who is healthier?’)
summarized in heat
maps.
Best Worst
Best
Worst
First health state in pair
Second
health
state in
pair

31. Salomon – Disability weights - 31
• Probabilities of responses
on paired comparisons
(‘Who is healthier?’)
summarized in heat maps.
High agreement in choices between very
healthy vs. unhealthy outcomes (>90%)
… or vice versa
(<10%)
Split responses for similar
outcomes (~50%)
Results: paired comparison responses

32. Salomon – Disability weights - 32
Measurement error in paired comparisons
• In the household survey, we
assessed test-retest
reliability by randomly
assigning 20% of
respondents to have the
same pair repeated as 1st
and 15th comparison
• Consistency of responses
ranged between 60% and
70%
• Consistency of 2 coin flips
would be 50%
• Kappa values range from
20% to 40%
Percent of respondents with consistent retest responses
0
10
20
30
40
50
60
70
80
Bangladesh
Indonesia
Peru
Tanzania
AllH
H
36

33. Salomon – Disability weights - 33
• Probabilities of responses
on paired comparisons
(‘Who is healthier?’)
summarized in heat maps.
High agreement in choices between very
healthy vs. unhealthy outcomes (>90%)
… or vice versa
(<10%)
Split responses for similar
outcomes (~50%)
Results: paired comparison responses

34. Salomon – Disability weights - 34
Results: paired comparisons across surveys

35. Salomon – Disability weights - 35
35
Results: probit values across surveys
-2 0 2 4 6
-2
0
2
4
6
Pooled
Bangladesh
r = 0.75
-2 0 2 4 6
-2
0
2
4
6
Pooled
Peru
r = 0.94
-2 0 2 4 6
-2
0
2
4
6
Pooled
UnitedStates
r = 0.97
-2 0 2 4 6
-2
0
2
4
6
Pooled
Indonesia
r = 0.90
-2 0 2 4 6
-2
0
2
4
6
Pooled
Tanzania
r = 0.94
-2 0 2 4 6
-2
0
2
4
6
Pooled
Web
r = 0.98

36. Salomon – Disability weights - 36
Web
None
Primary
Secondary
Higher
0
5000
10000
15000
Numberofrespondents
Results: comparison of household and web surveys
• Web respondents comprise
non-random, highly
educated, self-selected
sample
Household
None
Primary
Secondary
Higher
0
1000
2000
3000
4000
Numberofrespondents
Educational attainment in HH & web
samples

37. Salomon – Disability weights - 37
Results: comparison of household and web surveys
Tanzania (N=2,613)
Web (N=3,417)
• Web respondents comprise
non-random, highly
educated, self-selected
sample
• But, response probabilities
are virtually
indistinguishable from those
in household surveys

38. Salomon – Disability weights - 38
Results: comparison of household and web surveys
• Web respondents comprise
non-random, highly
educated, self-selected
sample
• But, response probabilities
are virtually
indistinguishable from those
in household surveys
• And estimated weights from
probit regressions are very
highly correlated Pooled household surveys
Websurvey
r = 0.88

39. Salomon – Disability weights - 39
Results: new disability weights

40. Salomon – Disability weights - 40
Results: new disability weights

41. Salomon – Disability weights - 41
Results: new disability weights

42. Salomon – Disability weights - 42
Results: new disability weights

43. Salomon – Disability weights - 43
Results: new disability weights

44. Salomon – Disability weights - 44
Results: comparison to previous disability weights
0.0 0.2 0.4 0.6 0.8 1.0
0.0
0.2
0.4
0.6
0.8
1.0
Previous disability weight
Newdisabilityweight

45. Salomon – Disability weights - 45
Framing challenge: drug use disorders
…uses heroin daily and has difficulty
controlling the habit. When the effects
wear off, the person feels severe
nausea, agitation, vomiting and fever. The
person has a lot of difficulty in daily
activities.

46. Salomon – Disability weights - 46
Framing challenge: drug use disorders
…uses heroin daily and has difficulty
controlling the habit. When the effects
wear off, the person feels severe
nausea, agitation, vomiting and fever. The
person has a lot of difficulty in daily
activities.

47. Salomon – Disability weights - 47
Framing experiment: caffeine addiction
Alternative descriptions for caffeine addiction
…drinks several cups of coffee a day in order to increase energy and stay alert.
When the effects wear off, the person feels tired and irritable and sometimes
gets headaches
…takes medication several times a day in order to increase energy and stay
alert. When the effects wear off, the person feels tired and irritable and
sometimes gets headaches
…uses an addictive substance several times a day in order to increase energy
and stay alert. When the effects wear off, the person feels tired and irritable
and sometimes gets headaches
…uses an addictive drug several times a day in order to increase energy and
stay alert. When the effects wear off, the person feels tired and irritable and
sometimes gets headaches
…uses an illegal, addictive drug several times a day in order to increase
energy and stay alert. When the effects wear off, the person feels tired and
irritable and sometimes gets headaches

48. Salomon – Disability weights - 48
Framing experiment: caffeine addiction
Alternative descriptions for caffeine addiction
Disability
weights
…drinks several cups of coffee a day in order to increase energy and stay alert.
When the effects wear off, the person feels tired and irritable and sometimes
gets headaches
0.018
…takes medication several times a day in order to increase energy and stay
alert. When the effects wear off, the person feels tired and irritable and
sometimes gets headaches
0.064
…uses an addictive substance several times a day in order to increase energy
and stay alert. When the effects wear off, the person feels tired and irritable
and sometimes gets headaches
0.067
…uses an addictive drug several times a day in order to increase energy and
stay alert. When the effects wear off, the person feels tired and irritable and
sometimes gets headaches
0.136
…uses an illegal, addictive drug several times a day in order to increase
energy and stay alert. When the effects wear off, the person feels tired and
irritable and sometimes gets headaches
0.198

49. Salomon – Disability weights - 49
…takes daily medication
and has difficulty going
without it. When the
effects wear off, the
person feels severe
nausea, agitation,
vomiting and fever. The
person has a lot of
difficulty in daily
activities.
…uses heroin daily and has difficulty
controlling the habit. When the effects
wear off, the person feels severe
nausea, agitation, vomiting and fever. The
person has a lot of difficulty in daily
activities.
Framing challenge: drug use disorders

50. Salomon – Disability weights - 50
…takes daily medication
and has difficulty going
without it. When the
effects wear off, the
person feels severe
nausea, agitation, vomiti
ng and fever. The
person has a lot of
difficulty in daily
activities.
…uses heroin daily and has difficulty
controlling the habit. When the effects
wear off, the person feels severe nausea,
agitation, vomiting and fever. The person
has a lot of difficulty in daily activities.
Framing challenge: drug use disorders

51. Salomon – Disability weights - 51
• Background
• Study design and methods
• Key findings
• Interpretations, limitations, conclusions
Disability Weights Measurement Study

52. Salomon – Disability weights - 52
Interpretations
Largest empirical effort to date to measure weights for health
outcomes across range of populations shows:
• Feasible to collect this sort of information in virtually any population
• Simple data collection tools can be combined with straightforward analytic
techniques to yield meaningful weights
• Weights appear highly consistent across diverse cultural settings and
respondent characteristics
New disability weights provide critical resource for assessment
of burden of disease, healthy life expectancy and intervention
cost-effectiveness

53. Salomon – Disability weights - 53
Limitations
• Selection of countries to provide diversity, not as random
sample of world’s population
• Web survey heavily skewed toward North America, Australia,
Western Europe, with few respondents from Africa or Middle
East
• Responses depend on validity of lay descriptions
 Are some key consequences omitted?
 Does inclusion of labels in selected instances (e.g. drug use disorders)
bias disability weights for these conditions?
• Future work should prioritize
 Exploring use of standardized health-state classifications
 More empirical data on distribution of sequelae across health-states

54. Salomon – Disability weights - 54
Lancet 2012; 380: 2129-2143.