This is the file for a podium presentation that I gave at the 2011 Society for Medical Decision Making Conference in Chicago, IL. The content deals with using probabilistic methods to determine the affected population in value of information analyses. This method can be used to reflect "real-world" uncertainties in incidence, diagnosis, and time-horizon, and yields a credible interval around value of information metrics like EVPI, EVPPI, EVSI, and EVSPI.

1. ACCOUNTING FOR UNCERTAINTY IN
AFFECTED POPULATION IN VALUE
OF INFORMATION ANALYSES
AN APPLICATION IN ADVANCED BILIARY TRACT CANCER
Joshua A. Roth & Josh J. Carlson
University of Washington, Pharmaceutical Outcomes Research & Policy Program
Society for Medical Decision Making International Meeting
Chicago, IL
October 22, 2011

2. Study Rationale & Objective
• Value of information (VOI) methods can be used to calculate the
societal value of additional research to reduce uncertainty
• Estimating the affected population is a key element of VOI
calculations, but there is often uncertainty
• Typical VOI approaches use deterministic affected population
values that do not account for uncertainty
• Objective: Demonstrate impact and utility of accounting for
uncertainty in affected population calculations through a case
study in advanced biliary tract cancer
2

3. Background: Value of Information (VOI) Analysis
• VOI can be calculated using probabilistic simulation models
• Driven by net-benefit: (WTP*QALYs)-Cost
• Expected Value of Perfect Information (EVPI)
• A measure of the value to society associated with reducing all
uncertainty and expected opportunity loss
• EVPI can be compared with the cost of additional studies to
inform and prioritize research investments
Ades et al., Medical Decision Making, 2004; Koerkamp et al., Value in Health, 2010; Meltzer et al., MDM, 2011
3

4. Background: Typical Simulation EVPI Calculation
1. Specify willingness to pay (WTP) threshold
2. Conduct probabilistic analysis to obtain the incremental net-
benefit distribution at the given WTP threshold
3. Calculate per-patient EVPI by examining the frequency and
magnitude of non-optimal decisions
4. Specify the affected population using disease incidence,
time horizon, and rate of technology diffusion
5. Multiply the per-patient EVPI by the affected population to
obtain population EVPI
Meltzer et al., MDM, 2011
4

5. Background: Affected Population Issues
1. Affected population parameters are often highly
uncertain (incidence, diagnosis, time horizon)
2. A technology diffusion rate needs to be specified:
• Instant Diffusion: All of the affected population receives the
optimal strategy instantly
• Constant Gradual Diffusion: The affected population that
receives the optimal strategy increases at a constant rate
over the lifetime of the technology
• Time-Dependent Gradual Diffusion: The affected population
that receives the optimal strategy increases at a time-
dependent rate over the lifetime of the technology
– Rogers Diffusion of Innovation Curve
Rogers, Diffusion of Innovations, 1962
5

6. Background: Advanced Biliary Tract Cancer
• Biliary tract cancer is cancer of the bile duct:
• Difficult to detect, and incidence is somewhat uncertain
• Often diagnosed at an advanced stage, but different
screening methods could change this over time
• New evidence shows that Gemcitabine+Cisplatin may
result in favorable health outcomes relative to standard
treatment with Gemcitabine monotherapy
• Uncertain rate of change in medical practice
• Affected population is highly uncertain if calculating
the value of additional research
Valle et al., NEJM, 2010
6

7. Methods: Simplified Model Schematic
,-.$2'$(;.$%&(
6$7/*&"8*%$95*':1"#%(
<=(,-.$2'$(;.$%&(
!"#$%&'()*&+(,-."%/$-(
0*1*"23(42"/&(5"%/$2(
,-.$2'$(;.$%&(
6$7/*&"8*%$(
>=%=&+$2":3(
<=(,-.$2'$(;.$%&(
?,11(:"#$%&'(@$2$(A=11=@$-(B%#1(-$"&+(
Roth et al., J Gastrointestinal Cancer, 2010
7

8. Methods: Analysis
• Using a previously-published cost-effectiveness
model, we calculated per-patient EVPI in the U.S. at
a willingness to pay threshold of $150,000/QALY
• Ran 10,000 probabilistic simulations with incidence,
diagnosis, and time horizon parameters
• Used instant, constant gradual, and time-dependent
gradual technology diffusion approaches
• Compared the results of each approach
8

9. Methods: Selected Model Parameters
Deterministic Low High Distribution Reference
Annual Incidence of
9,500 7,600 11,400 Normal Roth, 2010
Biliary Tract Cancer
Proportion of Biliary Tract
Cancer Diagnosed at 0.650 0.455 0.845 Beta Jarnagin, 2001
Advanced Stage
Time Horizon for
10 5 15 Normal Assumption
Technology (Years)
9

10. Results: Per-Patient EVPI
• Proportion of simulations with non-optimal decision: 10.6%
• Average opportunity loss from non-optimal decision: $7,947
• Per-patient EVPI: $842
10

11. Results: Affected Population By Method
10,000
9,000
8,000 Instant Diffusion
(Deterministic)
7,000
Affected Population
6,000
Constant Gradual
5,000 Diffusion
(Probabilistic)
4,000
3,000 Time-Dependent
Gradual Diffusion
2,000 (Probabilistic)
1,000
0
Year 0 Year 2 Year 4 Year 6 Year 8 Year 10
11

12. Results: Population EVPI
Affected Population (95% CI) Population EVPI (95% CI)
Instant Diffusion
61,750 (N/A) $51,994,000 (N/A)
(Deterministic)
Constant Gradual Diffusion
33,345 (16,673-50,018) $28,076,000 ($14,038,000-$42,115,000)
(Probabilistic)
Time-Dependent Gradual
28,251 (14,125-42,376) $23,787,000 ($11,894,000-$35,681,000)
Diffusion (Probabilistic)
12

13. Issues & Limitations
• Gradual diffusion approaches are conservative relative to
instant diffusion approaches, and may better represent
real world experience
• Probabilistic affected population calculation adds
additional complexity to VOI calculation, but provides
credible intervals to inform decisions
• It is unclear if medical decision-makers would prefer these
types of approaches to more simple and standard VOI
methods
13

14. Conclusions
• Many VOI analyses utilize deterministic affected population
estimates that do not reflect uncertainty
• Probabilistic approaches can be used to reflect uncertainty in
affected population parameters
• Similar modeling approaches can be utilized to calculate
expected value of perfect parameter information (EVPPI) and
sample information (EVSI)
• Stakeholders using VOI to inform research decisions may find
these approaches more informative than standard
deterministic affected population calculation methods
14

15. Acknowledgements
• David Veenstra
• Scott Ramsey
• Rahber Thariani
• David Blough
Supported in part by:
Center for Comparative Effectiveness Research in Cancer Genomics (5RC2CA148570-01, Ramsey S, PI)
15

16. Contact Information
Joshua A. Roth
University of Washington
Pharmaceutical Outcomes Research & Policy Program
1959 NE Pacific St, Health Sciences Building
Seattle, WA 98195-7630
rothja@uw.edu