Disability weights measurement in the Global Burden of Disease Study 2010

1,751 views

Published on

GHME 2013 Conference
Session: Disability weights measurement workshop
Date: June 18 2013
Presenter: Josh Salomon
Institution:
Harvard School of Global Health

Published in: Health & Medicine, Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,751
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
75
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Disability weights measurement in the Global Burden of Disease Study 2010

  1. 1. Salomon – Disability weights - 2 2 Global Burden of Disease Study • Global Burden of Disease (GBD) Study aims to measure impact of disease and injury in terms of losses in population health • GBD 1990 study launched in 1991, updated by WHO in 2000s • GBD 2010 study, undertaken from 2007-2012, provides first comprehensive overhaul since 1996 • GBD quantifies • Magnitude of different health problems in units of disability-adjusted life years (DALYs) • Overall population health in units of healthy life expectancy (HALE)
  2. 2. Salomon – Disability weights - 3 3 Disability weights • Disability weights provide the bridge between mortality and non-fatal outcomes in DALYs and in healthy life expectancy • To measure health impact of non-fatal outcomes, GBD needs weights for all unique sequelae, which capture the major health consequences of all of the causes in the study • Disability weights quantify severity of outcomes as percentage reductions from perfect health, which are multiplied by years lived in each sequela to give years lived with disability (YLD)  e.g., If weight for blindness were 0.20, then 5 years lived with blindness would be equivalent to dying one year prematurely
  3. 3. Salomon – Disability weights - 4 Intervention 2: Prevent 1y of deafness for 2000 individuals Intervention 1: Extend life by 1y in 1000 healthy individuals Disability weights in the 1996 GBD revision Person trade-off: which would you choose? • Expert panel used ‘person trade-off’ to assign values to 22 indicator conditions
  4. 4. Salomon – Disability weights - 5 1 Disability weights 2 3 4 5 6 7 Class 1: • Vitiligo on face Class 4: • Below-knee amputation • Deafness Class 7: • Active psychosis • Quadriplegia Disability weights in the 1996 GBD revision • Expert panel used ‘person trade-off’ to assign values to 22 indicator conditions • These 22 conditions used as operational definitions of 7 disability classes 0.0 0.2 0.4 0.6 0.8 1.0
  5. 5. Salomon – Disability weights - 6 1 Disability weights 2 3 4 5 6 7 Rheumatoid arthritis cases Average disability weight =0.2*0.07 + 0.4*0.18 + 0.4*0.30 =0.21 Disability weights in the 1996 GBD revision • Expert panel used ‘person trade-off’ to assign values to 22 indicator conditions • These 22 conditions used as operational definitions of 7 disability classes • Remaining conditions allocated across classes to compute average weights 0.0 0.2 0.4 0.6 0.8 1.0
  6. 6. Salomon – Disability weights - 7 7 Disability weights measurement study goals • Derive weights for all 220 health states capturing nonfatal outcomes from 291 disease and injury causes in GBD 2010 • Address criticisms of previous approaches by:  Focusing on valuations from community respondents…  … in a diverse range of settings  … using suitable measurement methods • Specific research aims  Develop valid and reliable data collection tools for population- based surveys  Empirical examination of variation in weights
  7. 7. Salomon – Disability weights - 8 • Background • Study design and methods • Key findings • Interpretations, limitations, conclusions Disability weights measurement in GBD 2010
  8. 8. Salomon – Disability weights - 9 9 Study components • Population-based household surveys • Face-to-face interviews in Tanzania, Bangladesh, Indonesia, Peru • Telephone interview in random sample of US households • Focus on paired comparisons for 108 health-states • Key objectives include comparative analysis across diverse settings and benchmarking Internet survey against community samples • Open-access Internet surveys • Available in English, Spanish and Mandarin • Key objectives are to fill in gaps with remaining sequelae and to anchor scale for paired comparison responses Bangladesh Tanzania
  9. 9. Salomon – Disability weights - 10 Web survey Web survey included 16,328 respondents from 167 countries
  10. 10. Salomon – Disability weights - 11 Web survey included 16,328 respondents from 167 countries Web Responses 0 1 - 9 10 - 49 50 - 99 100 - 499 > 499 Web survey 1 - 9 10 - 49 50 - 99 100 - 499 500+
  11. 11. Salomon – Disability weights - 12 12 Measurement methods: paired comparisons • Primary mode of eliciting responses is paired comparison • Respondents hear (or read) two descriptions of hypothetical people, each with a randomly selected condition • Respondents indicate which person is healthier • Paired comparison questions chosen for relative ease of comprehension, administration and analysis • Literacy and numeracy not essential • Health comparisons not tied to external “calibrators” such as risk • Appealing intuitive basis and established strategies for analysis
  12. 12. Salomon – Disability weights - 13 13 Framing paired comparisons • Basis for all comparison are lay descriptions of sequelae, which highlight major functional consequences and symptoms associated with each sequela • Must be brief: restricted to <35 words based on pretest results • Must use simple, non-clinical vocabulary • Prologue to paired comparison questions orients respondents to focus on functioning A person’s health may limit how well parts of his body or his mind work. As a result, some people are not able to do all of the things in life that others may do, and some people are more severely limited than others. I am going to ask you a series of questions about different health problems. In each question I will describe two different people … Imagine they have the same number of years left to live, and will experience the health problems that I describe for the rest of their lives. I will ask you to tell me which person you think is healthier overall, in terms of having fewer physical or mental limitations on what they can do in life…
  13. 13. Salomon – Disability weights - 14 Paired comparison example The first person has vision problems that make it difficult to see and recognize faces of family or friends across a room. The second person has severe back and leg pain, which causes difficulty dressing, sitting, standing, walking, and lifting things. The person sleeps poorly and feels worried. Imagine that both people will have these problems for the rest of their lives. Who would you say is healthier overall, the first person or the second person? • Respondents in household surveys each answered 15 of these questions, with pairs of sequelae drawn at random from the universe of 108 108 possible pairs
  14. 14. Salomon – Disability weights - 15 15 Analyzing paired comparison data Bangladesh Tanzania • Simple ordering of outcomes based on “winning” proportions, as in various websites using paired comparisons to rank large pools of competitors http://kittenwar.com/ home winningest kittens losingest kittens newest kittens add your kitten facebook group kittenwar myspace faq e-mail us kitten search: Go 48% of people agree that Misty is cuter than Ling. Prada Von Fluffy Doom Cheshire Click the cutest kitten picture! Can't decide? Refresh the page for a draw.
  15. 15. Salomon – Disability weights - 16 16 Analyzing paired comparison data Bangladesh Tanzania Won 78% of 511 battles Won 77% of 539 battles Winningest Lost 80% of 846 battles Lost 80% of 1872 battles Losingest • Simple ordering of outcomes based on “winning” proportions, as in various websites using paired comparisons to rank large pools of competitors http://kittenwar.com/
  16. 16. Salomon – Disability weights - 17 17 Analyzing paired comparison data Bangladesh Tanzania • Simple ordering of outcomes based on “winning” proportions, as in various websites using paired comparisons to rank large pools of competitors • More sophisticated algorithms incorporate information on the relative strength of the opponent (e.g. Elo rating system in Chess) • Statistical approaches enable maximum likelihood estimation of underlying scores
  17. 17. Salomon – Disability weights - 18 18 Statistical modeling of paired comparisons data • Conceptual foundation for statistical modeling of paired comparisons data comes from Thurstone (1927), through Luce (1959), McFadden (1974) and others • Intuitively, a pair of health states that are similar in severity are likely to produce greater disagreement over which is healthier than a pair of states that are different in severity • Consider a population in which:  70% of people think depression is worse than blindness  90% of people think blindness is worse than arthritis • Reasonable to conclude that depression and blindness are nearer on some unobserved scale than blindness and arthritis
  18. 18. Salomon – Disability weights - 19 -4 -2 0 2 4 6 8 blindness 90%
  19. 19. Salomon – Disability weights - 20 -4 -2 0 2 4 6 8 depression blindness 90%70%
  20. 20. Salomon – Disability weights - 21 -4 -2 0 2 4 6 8 depression blindness 90%
  21. 21. Salomon – Disability weights - 22 -4 -2 0 2 4 6 8 depression blindness arthritis 90%
  22. 22. Salomon – Disability weights - 23 -4 -2 0 2 4 6 8 depression blindness arthritis
  23. 23. Salomon – Disability weights - 24 24 Statistical modeling of paired comparisons data • Statistical model formalizes this intuition  Each health state i has an unobserved actual health level Xi as perceived by an individual rater  The rater makes a choice in the paired comparison based on which of the health states is regarded as ‘healthier’ State 1 is chosen over State 2 if X1 > X2 • Thurstone model assumes that Xi is normally distributed, implying that the choice probabilities are also normal • We can model response probabilities using probit regression  Requires arbitrary identifying assumptions  Assumes that each health state has the same variance around the disability weight
  24. 24. Salomon – Disability weights - 25 From paired comparisons to disability weights Analysis of paired comparisons Analysis of “population health equivalence” responses Rescaling of paired comparison results 47
  25. 25. Salomon – Disability weights - 26 From paired comparisons to disability weights Analysis of paired comparisons Analysis of “population health equivalence” responses Rescaling of paired comparison results • Pooled data from all household surveys, CATI and web survey • Estimated probit model with separate variance for each survey
  26. 26. Salomon – Disability weights - 27 From paired comparisons to disability weights Analysis of paired comparisons Analysis of “population health equivalence” responses Rescaling of paired comparison results • Some Web survey respondents randomly assigned to “population health equivalence” (PHE) questions, for subset of 30 sequelae • PHE provides separate estimates for these sequelae that are anchored on (0,1) disability scale 49
  27. 27. Salomon – Disability weights - 28 From paired comparisons to disability weights Analysis of paired comparisons Analysis of “population health equivalence” responses Rescaling of paired comparison results • Probit scale mapped to (0,1) disability scale by regressing probit coefficients on PHE anchors • Non-constant variance accommodated by regression on logit-transformed PHE values, then back-transforming values into (0,1) space 50
  28. 28. Salomon – Disability weights - 29 • Background • Study design and methods • Key findings • Interpretations, limitations, conclusions Disability weights measurement in GBD 2010
  29. 29. Salomon – Disability weights - 30 Results: paired comparison responses • Probabilities of responses on paired comparisons (‘Who is healthier?’) summarized in heat maps. Best Worst Best Worst First health state in pair Second health state in pair
  30. 30. Salomon – Disability weights - 31 • Probabilities of responses on paired comparisons (‘Who is healthier?’) summarized in heat maps. High agreement in choices between very healthy vs. unhealthy outcomes (>90%) … or vice versa (<10%) Split responses for similar outcomes (~50%) Results: paired comparison responses
  31. 31. Salomon – Disability weights - 32 Measurement error in paired comparisons • In the household survey, we assessed test-retest reliability by randomly assigning 20% of respondents to have the same pair repeated as 1st and 15th comparison • Consistency of responses ranged between 60% and 70% • Consistency of 2 coin flips would be 50% • Kappa values range from 20% to 40% Percent of respondents with consistent retest responses 0 10 20 30 40 50 60 70 80 Bangladesh Indonesia Peru Tanzania AllH H 36
  32. 32. Salomon – Disability weights - 33 • Probabilities of responses on paired comparisons (‘Who is healthier?’) summarized in heat maps. High agreement in choices between very healthy vs. unhealthy outcomes (>90%) … or vice versa (<10%) Split responses for similar outcomes (~50%) Results: paired comparison responses
  33. 33. Salomon – Disability weights - 34 Results: paired comparisons across surveys
  34. 34. Salomon – Disability weights - 35 35 Results: probit values across surveys -2 0 2 4 6 -2 0 2 4 6 Pooled Bangladesh r = 0.75 -2 0 2 4 6 -2 0 2 4 6 Pooled Peru r = 0.94 -2 0 2 4 6 -2 0 2 4 6 Pooled UnitedStates r = 0.97 -2 0 2 4 6 -2 0 2 4 6 Pooled Indonesia r = 0.90 -2 0 2 4 6 -2 0 2 4 6 Pooled Tanzania r = 0.94 -2 0 2 4 6 -2 0 2 4 6 Pooled Web r = 0.98
  35. 35. Salomon – Disability weights - 36 Web None Primary Secondary Higher 0 5000 10000 15000 Numberofrespondents Results: comparison of household and web surveys • Web respondents comprise non-random, highly educated, self-selected sample Household None Primary Secondary Higher 0 1000 2000 3000 4000 Numberofrespondents Educational attainment in HH & web samples
  36. 36. Salomon – Disability weights - 37 Results: comparison of household and web surveys Tanzania (N=2,613) Web (N=3,417) • Web respondents comprise non-random, highly educated, self-selected sample • But, response probabilities are virtually indistinguishable from those in household surveys
  37. 37. Salomon – Disability weights - 38 Results: comparison of household and web surveys • Web respondents comprise non-random, highly educated, self-selected sample • But, response probabilities are virtually indistinguishable from those in household surveys • And estimated weights from probit regressions are very highly correlated Pooled household surveys Websurvey r = 0.88
  38. 38. Salomon – Disability weights - 39 Results: new disability weights
  39. 39. Salomon – Disability weights - 40 Results: new disability weights
  40. 40. Salomon – Disability weights - 41 Results: new disability weights
  41. 41. Salomon – Disability weights - 42 Results: new disability weights
  42. 42. Salomon – Disability weights - 43 Results: new disability weights
  43. 43. Salomon – Disability weights - 44 Results: comparison to previous disability weights 0.0 0.2 0.4 0.6 0.8 1.0 0.0 0.2 0.4 0.6 0.8 1.0 Previous disability weight Newdisabilityweight
  44. 44. Salomon – Disability weights - 45 Framing challenge: drug use disorders …uses heroin daily and has difficulty controlling the habit. When the effects wear off, the person feels severe nausea, agitation, vomiting and fever. The person has a lot of difficulty in daily activities.
  45. 45. Salomon – Disability weights - 46 Framing challenge: drug use disorders …uses heroin daily and has difficulty controlling the habit. When the effects wear off, the person feels severe nausea, agitation, vomiting and fever. The person has a lot of difficulty in daily activities.
  46. 46. Salomon – Disability weights - 47 Framing experiment: caffeine addiction Alternative descriptions for caffeine addiction …drinks several cups of coffee a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches …takes medication several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches …uses an addictive substance several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches …uses an addictive drug several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches …uses an illegal, addictive drug several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches
  47. 47. Salomon – Disability weights - 48 Framing experiment: caffeine addiction Alternative descriptions for caffeine addiction Disability weights …drinks several cups of coffee a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches 0.018 …takes medication several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches 0.064 …uses an addictive substance several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches 0.067 …uses an addictive drug several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches 0.136 …uses an illegal, addictive drug several times a day in order to increase energy and stay alert. When the effects wear off, the person feels tired and irritable and sometimes gets headaches 0.198
  48. 48. Salomon – Disability weights - 49 …takes daily medication and has difficulty going without it. When the effects wear off, the person feels severe nausea, agitation, vomiting and fever. The person has a lot of difficulty in daily activities. …uses heroin daily and has difficulty controlling the habit. When the effects wear off, the person feels severe nausea, agitation, vomiting and fever. The person has a lot of difficulty in daily activities. Framing challenge: drug use disorders
  49. 49. Salomon – Disability weights - 50 …takes daily medication and has difficulty going without it. When the effects wear off, the person feels severe nausea, agitation, vomiti ng and fever. The person has a lot of difficulty in daily activities. …uses heroin daily and has difficulty controlling the habit. When the effects wear off, the person feels severe nausea, agitation, vomiting and fever. The person has a lot of difficulty in daily activities. Framing challenge: drug use disorders
  50. 50. Salomon – Disability weights - 51 • Background • Study design and methods • Key findings • Interpretations, limitations, conclusions Disability Weights Measurement Study
  51. 51. Salomon – Disability weights - 52 Interpretations Largest empirical effort to date to measure weights for health outcomes across range of populations shows: • Feasible to collect this sort of information in virtually any population • Simple data collection tools can be combined with straightforward analytic techniques to yield meaningful weights • Weights appear highly consistent across diverse cultural settings and respondent characteristics New disability weights provide critical resource for assessment of burden of disease, healthy life expectancy and intervention cost-effectiveness
  52. 52. Salomon – Disability weights - 53 Limitations • Selection of countries to provide diversity, not as random sample of world’s population • Web survey heavily skewed toward North America, Australia, Western Europe, with few respondents from Africa or Middle East • Responses depend on validity of lay descriptions  Are some key consequences omitted?  Does inclusion of labels in selected instances (e.g. drug use disorders) bias disability weights for these conditions? • Future work should prioritize  Exploring use of standardized health-state classifications  More empirical data on distribution of sequelae across health-states
  53. 53. Salomon – Disability weights - 54 Lancet 2012; 380: 2129-2143.

×