This talk was presented during the PMED Program Transition Workshop.

1. Estimation and Optimization of Composite Outcomes
Daniel J. Luckett
Department of Biostatistics
University of North Carolina at Chapel Hill
May 20, 2019
Luckett (UNC BIOS) Composite Outcomes May 2019 1 / 19

2. Precision Medicine
• Improving outcomes by leveraging patient heterogeneity
• Reproducible and generalizable, will work with future patients
• Data-driven personalized medicine
• Large complex data sets provide a wealth of opportunity
• Statistics and machine learning play an important part
Luckett (UNC BIOS) Composite Outcomes May 2019 2 / 19

3. Motivating Example: Bipolar Disorder
• Characterized by episodes of depression and mania
• Antidepressants can be used to treat depressive episodes
• Antidepressants may induce manic episodes
• Determine which patients should receive antidepressants
• Balance the trade-off between depression and mania
• The Systematic Treatment Enhancement Program for Bipolar
Disorder Standard Care Pathway (STEP-BD SCP)
Luckett (UNC BIOS) Composite Outcomes May 2019 3 / 19

4. Depression by Treatment and History of Substance Abuse
0
1
2
3
No Yes
Substance abuse
Depressionsymptomseverity
Antidepressant
No
Yes
Figure 1: Severity of depression symptoms by substance abuse and treatment.
Luckett (UNC BIOS) Composite Outcomes May 2019 4 / 19

5. Mania by Treatment and History of Substance Abuse
0
1
2
3
No Yes
Substance abuse
Maniasymptomseverity
Antidepressant
No
Yes
Figure 2: Severity of mania symptoms by substance abuse and treatment.
Luckett (UNC BIOS) Composite Outcomes May 2019 5 / 19

6. Composite Outcomes: Introduction
• Clinical decision making in the presence of more than one outcome:
• Disconnect with the traditional definition of optimal regimes
• Preference elicitation:
• Construct a composite outcome from preference questionnaires
• Butler et al. (2017) “Incorporating Patient Preferences into Estimation
of Optimal Individualized Treatment Rules”
• Estimating a composite outcome from observational data:
• Assume that clinicians act approximately optimally with respect to an
unknown composite outcome
• Luckett et al. (2018) “Estimation and Optimization of Composite
Outcomes.” arXiv preprint arXiv:1711.10581
Luckett (UNC BIOS) Composite Outcomes May 2019 6 / 19

7. Notation
• X ∈ X ⊆ Rp are covariates
• A ∈ {−1, 1} is treatment
• Y and Z are two real-valued outcomes with higher values preferable
• QY (x, a) = E {Y |X = x, A = a} is the mean of Y given X and A
• dopt
Y (x) = arg maxa∈{−1,1}QY (x, a) is the decision to maximize Y
• QZ and dopt
Z are defined similarly
Luckett (UNC BIOS) Composite Outcomes May 2019 7 / 19

8. Utility Functions
• Let ω(X; θ) = expit(X⊺θ) for θ ∈ Rp and define the utility function
u(Y, Z; X, θ) = ω(X; θ)Y + {1 − ω(X; θ)} Z
• Define
Qθ(x, a) = E {u(Y, Z; X, θ)|X = x, A = a}
and
dopt
θ (x) = arg max
a∈{−1,1}
Qθ(x, a) = sign {Dθ(x)}
where
Dθ(x) = ω(x; θ) {QY (x, 1) − QY (x, −1)}
+ {1 − ω(x; θ)} {QZ(x, 1) − QZ(x, −1)}
Luckett (UNC BIOS) Composite Outcomes May 2019 8 / 19

9. Pseudo-likelihood Estimation of Utility Functions
• Assume a true utility function defined by θ0 such that observed
decisions were made with the intent of maximizing u(Y, Z; X, θ0)
• Assume that
Pr A = dopt
θ0
(X) = expit(X⊺
β0)
for some β0 ∈ Rp
• The likelihood for (θ, β) is
Ln(θ, β) ∝
n
i=1
exp X⊺
i β1 Ai = dopt
θ (Xi)
1 + exp (X⊺
i β)
,
which can be used to estimate the utility function and the probability
that any patient would be treated optimally in standard care
Luckett (UNC BIOS) Composite Outcomes May 2019 9 / 19

10. Pseudo-likelihood Estimation (continued)
• The likelihood for (θ, β) depends on the unknown function dopt
θ
• Let QY,n and QZ,n be estimators for QY and QZ
• For any θ, let
Qθ,n(x, a) = ω(x; θ)QY,n(x, a) + {1 − ω(x; θ)} QZ,n(x, a)
and
dθ,n(x) = arg max
a∈{−1,1}
Qθ,n(x, a)
• We can replace dopt
θ with dθ,n to obtain the pseudo-likelihood
Ln(θ, β) ∝
n
i=1
exp X⊺
i β1 Ai = dθ,n(Xi)
1 + exp (X⊺
i β)
Luckett (UNC BIOS) Composite Outcomes May 2019 10 / 19

11. Simulation Results
• Let ω (X; θ) = expit(θ1) = ω for all X ∈ X
• Let Pr A = dopt
ω (X) = expit(β1) = ρ for all X ∈ X
n ω ρ ωn ρn Error rate
100 0.25 0.6 0.29 0.62 0.10
0.8 0.23 0.80 0.03
0.75 0.6 0.63 0.62 0.13
0.8 0.73 0.80 0.03
500 0.25 0.6 0.23 0.60 0.04
0.8 0.24 0.80 0.01
0.75 0.6 0.73 0.60 0.04
0.8 0.75 0.80 0.01
Table 1: Simulation results for fixed utility and probability of optimal treatment.
Luckett (UNC BIOS) Composite Outcomes May 2019 11 / 19

12. Simulation Results (continued)
• Evaluate treatment policies with estimated mean outcomes
• Mean outcome under optimal treatment ≈ 1.9 across scenarios
n ω ρ Estimated ω Y only Standard of care
100 0.25 0.6 1.75 0.39 0.39
0.8 1.88 0.39 1.14
0.75 0.6 1.69 1.76 0.40
0.8 1.89 1.76 1.15
500 0.25 0.6 1.88 0.38 0.37
0.8 1.90 0.38 1.13
0.75 0.6 1.88 1.76 0.37
0.8 1.90 1.76 1.13
Table 2: Estimated value for fixed utility and probability of optimal treatment.
Luckett (UNC BIOS) Composite Outcomes May 2019 12 / 19

13. Asymptotic Distribution
• Define RY (X) = QY (X, 1) − QY (X, −1) and RY,n(X) analogously
• Assume the existence of influence and basis functions such that
√
n RY,n(x) − RY (x) − φY (x)⊺
n−1/2
n
i=1
ψi,Y
X
= oP (1)
and likewise for Z
• Define R0(X) = RY (X) − RZ(X)
• Let Pβ(X) = expit (X⊺β) and I0 = E [Pβ0 (X) {1 − Pβ0 (X)} XX⊺]
• Define ψi,A = 1 Ai = dopt
θ0
(Xi) − Pβ0 (Xi) Xi
• Finally, let Σ0 = E ψ⊺
1,Y , ψ⊺
1,Z, ψ⊺
1,A
⊺ ⊗2
Luckett (UNC BIOS) Composite Outcomes May 2019 13 / 19

14. Asymptotic Distribution (continued)
Theorem 2.1 (Asymptotic distribution)
Define
k0(ZY , ZZ, u) = E X {2Pβ0 (X) − 1} · ω(X; θ0)RY (X) {φY (X)⊺
ZY } +
{1 − ω(X; θ0)} RZ(X) {φZ(X)⊺
ZZ} + R0(X) ˙ωθ0 (X)⊺
u Dθ0 (X) = 0 .
Then, under mild regularity conditions, we have that
√
n
θn − θ0
βn − β0
U
I−1
0 {ZA − k0(ZY , ZZ, U)}
,
where Z⊺
Y , Z⊺
Z, Z⊺
A
⊺
∼ N(0, Σ0) and U = arg minu∈Rp β⊺
0 k0(ZY , ZZ, u).
Luckett (UNC BIOS) Composite Outcomes May 2019 14 / 19

15. The STEP-BD SCP
• Covariates: age and history of substance abuse
• Outcomes: SUM-D scale for depression and SUM-M scale for mania
• Treatment: antidepressant or not
• Goals:
• Determine the convex combination of SUM-D and SUM-M that
clinicians seek to optimize
• Determine which factors contribute to a patient receiving optimal
treatment in standard care
• Estimate and evaluate policies to assign treatment to optimize the
underlying composite outcome
Luckett (UNC BIOS) Composite Outcomes May 2019 15 / 19

16. Results of Analysis of STEP-BD SCP Data
Policy SUM-D SUM-M Value (% improvement) ωn ρn
fixed-fixed 2.351 0.867 0.1% 0.115 0.403
fixed-variable 2.315 0.840 3.1% 0.115 0.405
variable-variable 2.297 0.838 7.1% 0.173 0.405
standard of care 2.480 0.868 0.0% · ·
Table 3: Results of analysis of STEP-BD data for SUM-D and SUM-M.
• Small values of SUM-D and SUM-M are preferable
• % improvement in value calculated from estimated utility function
• When probability of optimal treatment isn’t fixed (fixed-variable and
variable-variable), ρn = En expit X⊺βn
• When utility isn’t fixed (variable-variable), ωn = En expit X⊺θn
Luckett (UNC BIOS) Composite Outcomes May 2019 16 / 19

17. Discussion of STEP-BD Results
• The fixed-fixed policy assigns antidepressants when
sign {0.207 − 0.003(age) − 0.620(substance abuse)}
is equal to 1
• The estimated policy indicates that patients with substance abuse
history should not receive antidepressants
Luckett (UNC BIOS) Composite Outcomes May 2019 17 / 19

18. Composite Outcomes: Final Thoughts
• Clinical decision making often involves balancing trade-offs between
multiple outcomes:
• A discordance with the usual definition of optimal treatment regimes
• Expert-elicited composite outcomes may not account for heterogeneity
• An instrument to construct composite outcomes may not be available
• Composite outcomes can be estimated from observed decisions that are
assumed to be “approximately optimal”
• A new way to think about observational data in precision medicine:
taking advantage of treatment assignment that is not randomized
Luckett (UNC BIOS) Composite Outcomes May 2019 18 / 19

19. Acknowledgments
• Collaborators: Michael Kosorok, Eric Laber
• STEP-BD data courtesy of the National Institute of Mental Health
(NIMH): Sachs et al. (2007). “Effectiveness of Adjunctive
Antidepressant Treatment for Bipolar Depression”
• Luckett et al. (2017) “Estimation and Optimization of Composite
Outcomes.” arXiv preprint arXiv:1711.10581
• http://www.laber-labs.com/2018/07/02/the-two-outcome-problem/
Luckett (UNC BIOS) Composite Outcomes May 2019 19 / 19