This presentation was given as the foundation for the Savvysherpa's Student Working Group problem lab. Working group members used this information to begin researching their problem and later went on to present their findings to their peers.

1. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
Global Research & Development
Precision Medicine for Rheumatoid Arthritis
SAMSI 2018 IMSM Workshop – July 17, 2018

2. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
WHO WE ARE
• Develop better models of healthcare delivery using data science and technology
• Research partnerships across the healthcare system
– Health systems
– Device and technology
– Pharma
– Academia
• Long-term focus
• Real world implementation
– Largest, most diversified company in the industry

3. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
PRECISION AND PERSONALIZED MEDICINE
• Precision medicine:
– In concept: “tailoring medical treatments to the individual characteristics
of each patient” (NRC)
– In practice: relatively limited application (usually focused on specific
genetic tests and markers)
• Why?
– Hard for busy clinicians to keep up with latest advances (especially
outside of academic medical centers)
– Need for more data sharing and better decision support systems
• Opportunity to consider the person in personalized medicine
– “Can this patient use this medication without side effects?”

4. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
RHEUMATOID ARTHRITIS (RA)
• Highly heterogeneous disease
• Many expensive treatment options with significant side-effect profiles
• No single gold-standard marker for disease activity
– Objective lab measurements do not correlate well with clinical
observation
an example of the need for personalized medicine

5. Introduction to the Problem:
Rheumatoid Arthritis

6. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
WHAT IS RHEUMATOID ARTHRITIS (RA)?
• Rheumatoid arthritis is an autoimmune disease where the body mistakenly
attacks the joints, resulting in chronic inflammation and pain
• Affects 1% of the world’s population, more prevalent in women
• Typically diagnosed in patients 30–60 years old
• Permanent joint damage is very common
• Estimated healthcare cost burden of RA is over $8 billion annually
• Total estimated societal costs of nearly $20 billion

7. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
TESTING FOR RA
• Doctors test for many biomarkers when making an RA diagnosis
– Rheumatoid factor
– Anticitrullinated protein antibody (ACPA) positivity
– Antinuclear antibodies
– Erythrocyte sedimentation rate (ESR)
– C-reactive protein (CRP)
• Variability as to which biomarkers rheumatologists will test for
• Not all RA patients have these biomarkers, and some people without RA will
test positive for them

8. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
TREATING RA
• RA is treated using disease-modifying anti-rheumatic drugs (DMARDs)
• There are many categories and types of DMARDs, ranging in price and strength
– ACR recommends that patients begin therapy with a low dose of methotrexate
(MTX)
– Most patients will then add or switch to another biologic or non-biologic
DMARDs
• RA is also treated using pain-killers (NSAIDs, opioids) and steroids
(glucocorticoids)
• There are high amounts of uncertainty in treating RA, and many of the ACR
recommendations have moderate to low evidence of actually working
the American College of Rheumatology (ACR) treatment guidelines

9. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
TRADITIONAL DMARDS
• Includes MTX and other non-biologic DMARDs, such as:
– Hydroxychloroquine
– Sulfasalazine
– Leflunomide
• Work by suppressing the immune system overall
• Typically taken in pill form and can take several months to work
• Some moderate side effects:
– Liver damage (MTX)
– Nausea, vomiting, and increased risk of infection

10. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
BIOLOGIC DMARDS
• Target specific proteins thought to be associated with RA and the immune
response
• Typically taken as an injection
• Response time is much shorter than traditional DMARDs
• Highly effective when they do work, but not helpful for all RA patients
• Biologics can have several life-threatening side effects:
– Increased risk of pneumonia
– Higher risk of infection and reactivation of tuberculosis
– Increased risk of skin cancer and lymphoma
– Skin reaction/rash at injection site

11. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
OUR GOAL
• Use data to build a decision-support tool for treating RA
• Inform future research and studies aimed at accurately targeting and
treating this disease (and possibly many others)

12. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
OUR DATA
• 2011–2016 aggregated weekly claims data for over 6,000 RA patients
• Member-level information
– Age, sex, general comorbidities
• Weekly information
– Drugs taken and dosage
– Time from diagnosis and initial treatment
• Drug Information
– DMARDs, NSAIDs, opioids, glucocorticoids
– Treatment groupings informed by the ACR guidelines
• No “response” variable
– One possible response: tolerable disease activity proxied by absence of pain
relievers and corticosteroids

13. Methodology:
Q-Learning for Dynamic
Treatment Regimes
Summary and Application in Prostate Cancer

14. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
BACKGROUND
• Dataset:
– Population: patients with prostate cancer
– Time scope: after first diagnosis of prostate cancer
– Information: demographic, health condition, treatments, and events
• Example: one individual with remission
• Research question: Can we find a personalized treatment path that
maximizes expected life span length?

15. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
BACKGROUND

16. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
Given data Set ! = #, and
Minimize loss function
with
If ! = 1
Set ! = ! − 1, and
Output
Yes
No
ALGORITHM

17. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
ALGORITHM
• Sequential randomization assumption: no unmeasured confounders
• Consistency assumption: covariates and rewards outcome observed in the
study are those that potentially would be seen under actual treatment
• Stable unit treatment value assumption: covariates and rewards are not
affected by how treatments are allocated to patients
• Positivity assumption: treatment options should be presented in the
observed data
assumptions

18. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
RESULTS
18
treatment regime in first treatment point

19. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
• For Radiation + Chemo
• Recommended + Used
• Recommended + Not Used
• Not Recommended + Used
RESULTS
compare survival function of three groups compare survival function of three groups

20. Thank you! Questions?

21. Appendix A:
Additional RA Information

22. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
OTHER RA MANIFESTATIONS
• Can impact other parts of the body
– Lungs, eyes, skin, other organs
• RA patients are 60% more likely to have heart attack within first year
• RA patients are twice as likely to suffer from depression
• Infection is common, and 25% of RA-related deaths are a result of infection

23. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
COSTS OF RA TREATMENT
• Biologics are much more expensive than traditional DMARDs
• Cost differences per month are in the thousands
• Previous analysis work demonstrated that biologics patients may have
$20,000 - $30,000 higher costs per year than patients who stay on
traditional DMARDs

24. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
COSTS OF RA TREATMENT
• Biologic drugs targeted towards rheumatoid arthritis are some of the best selling
pharmaceuticals on the market
• Humira (adalimumab) is the number one best-selling drug with revenue of over $18
billion in 2017 (15% increase)
– Humira is produced by AbbVie, who has blocked similar medications from being
introduced into the market until 2023
• Enbrel (etanercept) is the number five best-selling drug with revenue of nearly $8
billion in 2017 (10% increase)
• An article in 2016 states that the costs of Enbrel and Humira have risen 80% and
68% respectively since 2013

25. Appendix B:
Data

26. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
DATA TABLES OVERVIEW
• 2011–2016 weekly data for RA patients (n=6,720)
• MemberSummaryWk_v2: MemberID, time (from 1st Dx/trt), DMARD code,
Gluco/NSAIDs/Opioid
• Member: MemberID, covariates (age, sex, comorbidity)
• DrugTable_v2: DMARD code, DMARD name
– Define treatment combination using 2015 ACR guideline

27. Appendix C:
Q-Learning Algorithm

28. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
PROBLEM SETTING
• Time steps: ! = 1, 2, … '
• Variables for individual (, at time !
– )*,+: Health condition: a random variable depending on )*,+,- and .*,+,-
– .*,+: Treatment taken for individual ( at time !
– /*,+: Reward value: a random variable depending on )*,+ and .*,+
• For example: in prostate cancer
– ' = 2
• ! = 1: initial treatment decision time
• ! = 2: recurrence treatment decision time
– )*,+: a vector including age, Charlson Index, RAF (risk adjust factor), …
– .*,+: A combination of 5 treatments: radiation, chemo, surgery, hormone and immune
– /*,+: life span length to recurrence (! = 1) or death (! = 2).

29. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
PROBLEM SETTING
• A policy !:
• Question: to find an optimal policy.
– What is optimal?

30. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
PROBLEM SETTING
• Defind ! function
• To find a policy such that
• And define !∗ function

31. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
PROBLEM SETTING
• Defind ! function
• Similarly, define !∗ = !$∗ , then we get
• So target become to estimate

32. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
SOLUTION: BACKWARD PROPAGATION
• From data, we can’t observe realization of !",$
∗
, except for the last step:
• And the !∗ function can be obtained as

33. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
Given data Set ! = #, and
Minimize loss function
with
If ! = 1
Set ! = ! − 1, and
Output
Yes
No
ALGORITHM

34. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
ALGORITHM
• Remarks
– Regularization: penalize parameters when estimating betas
– Another way to estimate !"
∗
– In this problem, rewards are time length until certain event happen, so:
– Observational data: IPCW, propensity score

35. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
RESULTS
• Compare result of Q-learning and result of baseline model, which
estimating observed rewards:

36. © 2018 UnitedHealth Group. Any use, copying or distribution without written permission from UnitedHealth Group is prohibited.
RESULTS
36