This figure shows the standard cost-effectiveness plane with the dots representing the expected incremental benefit and cost of the intervention for individuals that may differ, for example, in preferences or other personal attributes. When considering cost-effectiveness at the population level, we typically calculate the expected benefit and expected cost of this distribution, shown here by point m, and compare it to the cost effectiveness threshold. So in this case, the standard analysis says the intervention is not cost-effective. However, looking at this figure, it is clear that for patients to the left of the y axis, the expected benefits are negative. If these patients are harmed, it is logical to wonder whether they might reject the treatment.
Perfect self-selection is the extreme form of this in which all patients harmed by an intervention reject it (now shown in orange dots), while all patients who benefit from the intervention choose it (the blue dots). If so, the point m no longer represents the expected benefits of offering the intervention.
Instead the orange points drop out since these patients rejecting the intervention now have neither benefits nor costs, and the expectation is over the remaining blue dots showing the patients who have chosen the treatment. This shifts the expected benefits to the right to m’, and in this case makes the intervention cost-effective. Thus, in this example, the standard CE analysis suggested the intervention was not cost-effective, but accounting for self-selection showed it to be CE. This illustrates the potential bias of traditional CEA and the potential importance of accounting for self-selection.
Self selection can also matter even when it is not perfect. Again here the blue dots are the patients choosing the treatment and the orange dots are those rejecting it. Looking at the figure, we see that most of the patients who benefit from the intervention (to the right of the y axis) still choose it, as shown by the blue dots, while most of those who are harmed (to the left of the y axis) reject it. While a few patients end up with the wrong choice for them, the patients choosing the treatment tend to be those most likely to benefit from it, which shifts the mean of the blue dots of patients getting the intervention to the right, improving expected benefits, and thus likely cost-effectiveness. This is the empirical self selection effect, and, as before, neglecting it creates a potential bias in traditional methods of CEA. To determine whether this matters in practice, it is useful to consider examples.
To do this, we focus on diabetes care among the elderly. We do so because - although diabetes care guidelines clearly call for intensive lowering of glucose among younger patients - It is unclear if this should apply to older patients. One reason for this uncertainty is that gains in life expectancy for older patients are smaller So, for example, variations in preferences about side effects of treatment may dominate Indeed some of the best CE models of intensive therapy in older patients: Show minimal or even negative effects on QALYs And that intensive therapy is not cost-effective Moreover we know many patients refuse intensive therapy This suggests that self-selection due to variation in preferences may have important effects on cost-effectiveness analysis in diabetes
To collect data for our study, we interviewed 515 diabetes patients over age 65 attending University of Chicago clinics over 2 years WE obtained utilities by time trade-off questions Examining TREATMENT HEALTH STATES of Conventional and intensive glucose lowering (using insulin or oral medications) AND COMPLCIATION HEALTH STATES of Blindness, end-stage renal disease, and lower extremity amputation Additional data was collected by medical records review
Examining these effects on empirical self-selection we find intensive therapy isn’t quite beneficial but nearly so, with a decline in QALYs of only .03, and certainly not nearly so harmful as suggested by the standard cost-effectiveness analysis.
This shows the same result graphically, with the expected costs and benefits moving from m to m’ as the orange dot patients reject intensive therapy, so that M’ is now to the right of the CE threshold and intensive therapy becomes cost-effective. This of course assumes perfect self selection
Examining these effects on empirical self-selection we find intensive therapy isn’t quite beneficial but nearly so, with a decline in QALYs of only .03, and certainly not nearly so harmful as suggested by the standard cost-effectiveness analysis.
Turning now to actual data on empirical selection, we in fact do see a tendency for the patients who choose intensive treatment ( the blue dots) to be those patients with greater expected benefits.
Examining these effects on empirical self-selection we find intensive therapy isn’t quite beneficial but nearly so, with a decline in QALYs of only .03, and certainly not nearly so harmful as suggested by the standard cost-effectiveness analysis.
Self selection can also matter even when it is not perfect. Again here the blue dots are the patients choosing the treatment and the orange dots are those rejecting it. Looking at the figure, we see that most of the patients who benefit from the intervention (to the right of the y axis) still choose it, as shown by the blue dots, while most of those who are harmed (to the left of the y axis) reject it. While a few patients end up with the wrong choice for them, the patients choosing the treatment tend to be those most likely to benefit from it, which shifts the mean of the blue dots of patients getting the intervention to the right, improving expected benefits, and thus likely cost-effectiveness. This is the empirical self selection effect, and, as before, neglecting it creates a potential bias in traditional methods of CEA. To determine whether this matters in practice, it is useful to consider examples.
Self selection can also matter even when it is not perfect. Again here the blue dots are the patients choosing the treatment and the orange dots are those rejecting it. Looking at the figure, we see that most of the patients who benefit from the intervention (to the right of the y axis) still choose it, as shown by the blue dots, while most of those who are harmed (to the left of the y axis) reject it. While a few patients end up with the wrong choice for them, the patients choosing the treatment tend to be those most likely to benefit from it, which shifts the mean of the blue dots of patients getting the intervention to the right, improving expected benefits, and thus likely cost-effectiveness. This is the empirical self selection effect, and, as before, neglecting it creates a potential bias in traditional methods of CEA. To determine whether this matters in practice, it is useful to consider examples.
Self selection can also matter even when it is not perfect. Again here the blue dots are the patients choosing the treatment and the orange dots are those rejecting it. Looking at the figure, we see that most of the patients who benefit from the intervention (to the right of the y axis) still choose it, as shown by the blue dots, while most of those who are harmed (to the left of the y axis) reject it. While a few patients end up with the wrong choice for them, the patients choosing the treatment tend to be those most likely to benefit from it, which shifts the mean of the blue dots of patients getting the intervention to the right, improving expected benefits, and thus likely cost-effectiveness. This is the empirical self selection effect, and, as before, neglecting it creates a potential bias in traditional methods of CEA. To determine whether this matters in practice, it is useful to consider examples.
Self selection can also matter even when it is not perfect. Again here the blue dots are the patients choosing the treatment and the orange dots are those rejecting it. Looking at the figure, we see that most of the patients who benefit from the intervention (to the right of the y axis) still choose it, as shown by the blue dots, while most of those who are harmed (to the left of the y axis) reject it. While a few patients end up with the wrong choice for them, the patients choosing the treatment tend to be those most likely to benefit from it, which shifts the mean of the blue dots of patients getting the intervention to the right, improving expected benefits, and thus likely cost-effectiveness. This is the empirical self selection effect, and, as before, neglecting it creates a potential bias in traditional methods of CEA. To determine whether this matters in practice, it is useful to consider examples.
Self selection can also matter even when it is not perfect. Again here the blue dots are the patients choosing the treatment and the orange dots are those rejecting it. Looking at the figure, we see that most of the patients who benefit from the intervention (to the right of the y axis) still choose it, as shown by the blue dots, while most of those who are harmed (to the left of the y axis) reject it. While a few patients end up with the wrong choice for them, the patients choosing the treatment tend to be those most likely to benefit from it, which shifts the mean of the blue dots of patients getting the intervention to the right, improving expected benefits, and thus likely cost-effectiveness. This is the empirical self selection effect, and, as before, neglecting it creates a potential bias in traditional methods of CEA. To determine whether this matters in practice, it is useful to consider examples.
"Cost-Effectiveness Analysis and the Value of Research" (PPT).
1.
Cost-Effectiveness Analysis and the Value of Research David Meltzer MD, PhD The University of Chicago
2.
Overview <ul><li>Cost-effectiveness analysis has long been used to assess the value of medical treatments and the information that comes from diagnostic tests </li></ul><ul><li>Newer value of information techniques have extended these tools to assess the value of medical research </li></ul><ul><li>Understanding behaviors determining use of medical interventions in the context of heterogeneity is key to assessing their value and priorities for research </li></ul><ul><li>Research may be especially valuable when it can be used to individualize care </li></ul>
3.
Value of Medical Treatments <ul><li>Health effects </li></ul><ul><ul><li>Length/quality of life: QALYs </li></ul></ul><ul><li>Cost effects </li></ul><ul><li>Choose all interventions for which cost/ QALY < threshold </li></ul><ul><ul><li>Often $50-100K/QALY </li></ul></ul><ul><li>Widely accepted, >> 1000 applications </li></ul>
4.
Value of Diagnostic Testing Test Don’t Test S H S H Max{pU(T|S)+(1-p)U(T|H), pU(N|S)+(1-p)U(N|H)} U(T|S) U(N|H) pU(T|S)+(1-p)U(N|H)
6.
Cost-Effectiveness of Pap Smears $830,000/LY $750 8 hours $7,300/LY $1,500 71 days 8 hours 1 year $91,000/LY $250 1 day $3,900/LY $750 71 days 2 years $2,600/LY $500 70 days $2,600/LY $500 70 days 3 years Marginal Cost per Life-Yr Saved Marginal Increase in Cost Marginal Increase in LE Average Cost per Life-Yr Saved Increase in Cost vs. no screening Increase in LE vs. no screening Frequency
7.
Testing as Value of Information Test Don’t Test S H S H Max{pU(T|S)+(1-p)U(T|H), pU(N|S)+(1-p)U(N|H)} U(T|S) U(N|H) pU(T|S)+(1-p)U(N|H)
8.
Research as Value of Information Test Don’t Test S H S H Max{pU(T|S)+(1-p)U(T|H), pU(N|S)+(1-p)U(N|H)} U(T|S) U(N|H) pU(T|S)+(1-p)U(N|H)
9.
Value of Information Approach to Value of Research <ul><li>Without information </li></ul><ul><ul><li>Make best compromise choice not knowing true state of the world (e.g. don’t know if intervention is good, bad) </li></ul></ul><ul><ul><ul><li>With probability p: get V(Compromise|G) </li></ul></ul></ul><ul><ul><ul><li>With probability 1-p: get V(Compromise|B) </li></ul></ul></ul><ul><li>With information </li></ul><ul><ul><li>Make best decision knowing true state </li></ul></ul><ul><ul><ul><li>With probability p: get V(Best choice|G) </li></ul></ul></ul><ul><ul><ul><li>With probability 1-p: get V(Best choice|B) </li></ul></ul></ul><ul><li>Value of information </li></ul><ul><ul><li>= E(outcome) with information - E(outcome) w/o information </li></ul></ul><ul><li> = {p*V(Best choice|G) + (1-p)*V(Best choice|B)} - </li></ul><ul><li>{p*V(Compromise|G) + (1-p)*V(Compromise|B)} </li></ul><ul><li>= Value of Research </li></ul>
10.
Practical Applications of Value of Information <ul><li>Several full applications </li></ul><ul><ul><li>UK (NICE): Alzheimer’s Disease Tx, wisdom teeth removal </li></ul></ul><ul><ul><li>US (AHRQ): Hospitalist research </li></ul></ul><ul><ul><li>But needed data can be hard to obtain </li></ul></ul><ul><li>Bound with more limited data </li></ul><ul><ul><li>Murphy/Topel: LE 3mo/yr*$50K/LY = $10K/person/yr = $3 Trillion/yr </li></ul></ul><ul><ul><li>Real value of research may be far less than expected, e.g., for prostate cancer: </li></ul></ul><ul><ul><ul><li>Maximal value of research = $ 5 Trillion </li></ul></ul></ul><ul><ul><ul><li>Expected value of perfect information = $21 Billion </li></ul></ul></ul><ul><ul><ul><li>Expected value of information = $ 1 Billion </li></ul></ul></ul><ul><li>Area of active investigation </li></ul><ul><ul><li>Most promising clearly for applied research </li></ul></ul>
11.
“ Bayesian Value of information analysis: An application to a policy model of Alzheimer's disease.”
17.
Practical Applications of Value of Information <ul><li>Several full applications </li></ul><ul><ul><li>UK (NICE): Alzheimer’s Disease Tx, wisdom teeth removal </li></ul></ul><ul><ul><li>US (AHRQ): Hospitalist research </li></ul></ul><ul><ul><li>But needed data can be hard to obtain </li></ul></ul><ul><li>Bound with more limited data </li></ul><ul><ul><li>Murphy/Topel: LE 3mo/yr*$50K/LY = $10K/person/yr = $3 Trillion/yr </li></ul></ul><ul><ul><li>Real value of research may be far less than expected, e.g., for prostate cancer: </li></ul></ul><ul><ul><ul><li>Maximal value of research = $ 5 Trillion </li></ul></ul></ul><ul><ul><ul><li>Expected value of perfect information = $21 Billion </li></ul></ul></ul><ul><ul><ul><li>Expected value of information = $ 1 Billion </li></ul></ul></ul><ul><li>Area of active investigation </li></ul><ul><ul><li>Most promising clearly for applied research </li></ul></ul><ul><ul><li>Increasing interest among pharma </li></ul></ul>
18.
Behavioral Cost-Effectiveness Analysis <ul><li>Value of health interventions depend on how they are used </li></ul><ul><ul><li>Especially in the presence of heterogeneity </li></ul></ul><ul><ul><li>True for treatments and for diagnostics </li></ul></ul><ul><li>Understanding behaviors determining use of health interventions key to their evaluation </li></ul><ul><ul><li>Optimizing behavior: self-selection/diagnostic testing </li></ul></ul><ul><ul><li>Non-optimal behavior: non-selective use </li></ul></ul>
19.
Standard CEA with Heterogeneous Individuals costs effectiveness m CE Blue Dots = Treated Patients
20.
Optimal Selection with Heterogeneity: via Self-selection or Diagnostic Testing costs effectiveness m CE Blue Dots=Pts gain from Tx; Orange Dots=Pts lose from Tx
21.
Effect of Perfect Selection on CEA costs effectiveness m CE m’ Blue Dots=Pts gain from Tx; Orange Dots=Pts lose from Tx (reject)
22.
Empirical Selection costs effectiveness m CE Blue Dots=Pts choose Tx; Orange Dots=Pts reject Tx
23.
Background: Diabetes in the Elderly <ul><li>Diabetes care guidelines call for intensive lowering of glucose among younger patients </li></ul><ul><li>However, unclear if this should apply to older patients </li></ul><ul><ul><li>Gains in life expectancy smaller </li></ul></ul><ul><ul><li>Side effects of treatment may dominate </li></ul></ul><ul><ul><li>CE models of intensive therapy in older patients: </li></ul></ul><ul><ul><ul><li>Minimal or even negative effects on QALYs </li></ul></ul></ul><ul><ul><ul><li>Not cost-effective </li></ul></ul></ul><ul><ul><li>Know many patients refuse intensive therapy </li></ul></ul><ul><li>Suggests self-selection may have important effects on CEA in diabetes </li></ul>
24.
Methods <ul><li>Interviewed 500 older diabetes patients to obtain data on preferences </li></ul><ul><ul><li>Conventional and intensive glucose lowering (using insulin or oral medications) </li></ul></ul><ul><ul><li>Blindness, end-stage renal disease, lower extremity amputation </li></ul></ul><ul><li>Collected data on treatment choices and patient characteristics by medical records review </li></ul><ul><li>Used CDC simulation model of intensive therapy for type 2 diabetes and patient-specific demographic, health, and preference data to get person-specific estimates of lifetime costs and benefits </li></ul><ul><li>Analyses of cost-effectiveness of intensive vs. conventional therapy contrasting all patients vs. perfect self-selection vs. empirical self-selection </li></ul>
25.
Results: Intensive vs. Conventional Therapy Standard CE Approach Full Population Group -- -0.49 8076 543 CE Ratio ($/QALY) Change in QALYs Change in Costs ($) N
26.
Perfect Self-Selection Effect for Intensive Therapy m m’ CE Blue dots--the cost-effectiveness values of individuals with an expected benefit from intensive therapy. Orange dots-- the cost-effectiveness values of individuals with a decrement in expected benefits with intensive therapy. M-- CE ratio for whole population. M’—CE ratio after self-selection.
27.
Results: Intensive vs. Conventional Therapy -- -3.25 7906 131 QALY<0 Perfect Self-Selection Standard CE Approach QALY>0 Full Population Group 20K 0.40 8165 403 -- -0.49 8076 543 CE Ratio ($/QALY) Change in QALYs Change in Costs ($) N
28.
Empirical Self-Selection Effect for Intensive Therapy Blue dots-- cost-effectiveness values for individuals who identify their care as intensive therapy. Orange dots-- cost-effectiveness values for all other individuals. M-- CE ratio for orange dot individuals. M’-- CE ratio for blue dot individuals.
29.
Results: Intensive vs. Conventional Therapy -- -0.80 8164 364 All others 47K 0.17 7948 154 Self-identified intensive therapy Empirical Self-Selection -- -3.25 7906 131 QALY<0 Perfect Self-Selection Standard CE Approach QALY>0 Full Population Group 20K 0.40 8165 403 -- -0.49 8076 543 CE Ratio ($/QALY) Change in QALYs Change in Costs ($) N
30.
Implications - I <ul><li>Results of standard CEA may be misleading </li></ul><ul><ul><li>In contrast to the suggestion of standard CEA, offering intensive glucose lowering to all older people likely cost-effective </li></ul></ul><ul><ul><li>CEAs should consider the importance of self-selection </li></ul></ul><ul><li>Distinction between perfect and empirical self-selection is potentially important </li></ul><ul><ul><li>Data on who will use a treatment if it is offered is important </li></ul></ul>
31.
Implications - II <ul><li>A framework to account for heterogeneity in patient benefits is key to valuing diagnostic tests, guidelines, decision-aids, or improved patient-doctor communication that can make care more consistent with variation in patient benefits </li></ul>
32.
Motivation for Diagnostic Test/Decision Aids costs effectiveness m CE Blue Dots=Pts choose Tx; Orange Dots=Pts reject Tx
33.
Aim of Diagnostic Test/Decision Aids costs effectiveness m CE Blue Dots=Pts choose Tx; Orange Dots=Pts reject Tx
34.
Value of Diagnostic Test/Decision Aids costs effectiveness m CE Blue Dots=Pts choose Tx; Orange Dots=Pts reject Tx c e
35.
Value of Diagnostic Test/Decision Aid <ul><li>Effectiveness = Pts e </li></ul><ul><li>Costs = Pts c </li></ul><ul><li>Total Benefit </li></ul><ul><ul><li>Cost-Benefit = (1/ Pts e + Pts c </li></ul></ul><ul><ul><li>Net Health Benefit = Pts e + Pts c </li></ul></ul>
36.
Per Capita Value of Identifying Best Population-level and Individual-level Treatment in Prostate Cancer $29 Best Population-level Treatment $2958 Best Individual-level Treatment Value
37.
Implications - III <ul><li>Modeling heterogeneity and selection suggests a framework to design co-payment systems to enhance the cost-effectiveness of therapies </li></ul>
38.
Motivation for Copayment ( c) costs effectiveness m CE Blue Dots=Pts choose Tx; Orange Dots=Pts reject Tx c
39.
Motivation for Copayment ( c) costs effectiveness m CE Blue Dots=Pts choose Tx; Orange Dots=Pts reject Tx c
40.
Per Capita Value of Identifying Best Population-level and Individual-level Care in Prostate Cancer with Full Insurance $29 Best Population-level Therapy $41 Best Individual-level Therapy with Full Insurance $2958 Best Individual-level Therapy Value
41.
Conclusions <ul><li>Cost-effectiveness analysis can be used to value diagnostic testing and research on diagnostic testing </li></ul><ul><ul><li>Approaches exist to bound calculations with limited data </li></ul></ul><ul><li>Understanding behaviors determining use of medical interventions in the context of heterogeneity is key to assessing their value and priorities for research </li></ul><ul><ul><li>Research may be especially valuable when it can be used to individualize care </li></ul></ul><ul><ul><li>Insurance and other determinants of use can significantly alter value of research </li></ul></ul>
43.
Implications of Empirical CEA <ul><li>Need to consider how a treatment will be used in deciding if it will be welfare improving </li></ul><ul><li>Highlights importance of efforts to promote selective use of treatments </li></ul><ul><ul><li>Biomarkers valuable if encourage selective use of treatments </li></ul></ul><ul><li>Need to consider how a biomarker will be used in deciding if it will be welfare improving </li></ul><ul><li>Highlights importance of efforts to promote selective use of biomarkers </li></ul><ul><ul><li>Biomarkers valuable if encourage selective use of treatments </li></ul></ul>
44.
Non-selective Use and Empirical Cost-effectiveness <ul><li>Cost-effectiveness analyses of interventions often stratify cost-effectiveness by indication </li></ul><ul><li>Yet technologies are often used non-selectively </li></ul><ul><li>The actual (empirical) costs and effectiveness of an intervention may be strongly influenced by patterns of use </li></ul>