Example of high-ROI data science project I did from start to finish. Please contact me if you are a hiring manager at GOOG, AMZN, McKinsey/BCG/Bain, or Booz Allen Hamilton and need someone who can do something like this!

1. RESEARCH POSTER PRESENTATION DESIGN © 2012
www.PosterPresentations.com
RESEARCH POSTER PRESENTATION DESIGN © 2015
www.PosterPresentations.com
0
2
4
6
8
10
12
14
16
18
20
Aug-14 Sep-14 Oct-14 Nov-14 Dec-14 Jan-15 Feb-15 Mar-15 Apr-15 May-15 Jun-15 Jul-15
PROBLEM
To identify patients with the highest risk of falling, a logistic regression was developed using SAS EG
• The model is trying to predict the probability that a patient will fall with injury during their hospital stay
• Universe: Adult Inpatient Stays at all PHS facilities (both Central and Regional Delivery Systems) Apr 2017- Mar 2018
METHODOLOGY
Adult Inpatient Falls with Injury
had been slightly trending upward
for 12 months until July 15, 2015
when the No One Walks Alone
(NOWA) protocol was launched
Marmi Le, MBA, Business Analytics Consultant, Quality Informatics
Predictive Model for Falls
Stephanie Mora, MBA, Analytics Associate, Tools & Solutions
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
Observed Rate per 1000 Eligible Discharges National Risk-Adjusted Rate per 1000 Eligible Discharges
0
2
4
6
8
10
12
14
16
18
20
Aug-15 Sep-15 Oct-15 Nov-15 Dec-15 Jan-16 Feb-16 Mar-16 Apr-16 May-16 Jun-16 Jul-16
0
2
4
6
8
10
12
14
16
18
20
…and for the one year period following the NOWA launch,
the number of falls did have a downward trend
However, for the last 2 years, the incidence
of falls has been back on the rise
For FY 2018, Presbyterian Hospital is also above the
national average for In-Hospital Falls with Hip Fracture.
1. Fisher, Steve R. et al. “Early Ambulation and Length of Stay in Older Adults Hospitalized for Acute Illness.” Archives of internal
medicine 170.21 (2010): 1942–1943. PMC. Web. 25 Apr. 2018.
2. Total risk score = 1.734*(History of falling)+1.03*(Many diagnoses)+1.044*(Depression)+1.082*(Frailty)+1.108*(Total Rx
Gaps)+1.026*(Unplanned Inpt Hosp Stays Count)+1.025*(Total Cost Predicted Risk)+117.52*(Probability of Hospital
Injury)+1.374*(Probability of Unplanned 30-day Readmission)+1.274*(Dementia)+1.550*(Dehydration)+1.674*(Admitted
between 3pm and 7am)
This measure is a component of the
Patient Safety Indicator (PSI) Composite
– CMS can reduce payments to the worst-
performing hospitals against this index.
Data source: Midas indicator “22875 C Harm Adult
Inpatient Falls with Injury bes”
Data source: Tableau dashboard “Hospital Acquired
Condition Reduction Program”
(1) Model using only
variables from Midas
output had unsatisfactory
performance.
(2) Added innovative calculated measures known
as ACG scores (diagnosis-based, statistically valid
proprietary case-mix methodology developed by
Johns Hopkins University) from Clarity table as
explanatory variables – model got better!
(3) ACG measures are not available for every
patient in the universe. Therefore a conservative
imputation method was first used in order to
retain the full universe for model estimation –
replace all nulls with the value of ZERO.
(4) Model quality was much improved when the
missing ACG measures were instead imputed
using the mean of values that were present for
each measure.
Variables in Final Model
History of falling Total Cost Predicted Risk
Many Diagnoses
(16 or more)
Probability of Hospital Injury
Depression Dementia
Frailty Dehydration
Count of Unplanned
Inpatient Hospital Stays
Probability of Unplanned
30-day Readmission
Total Rx Gaps Admitted 3pm to 7am
Total observations: 24,928
Number of observations where patient fell: 150
C-statistic: 0.62
Odds Ratio Estimates and Wald Confidence Intervals
Effect Unit Estimate 95% Confidence Limits
HxOrRepeatedFalls 1.0000 1.734 0.829 3.625
ManyDx 1.0000 1.030 0.719 1.475
Depression 1.0000 1.044 0.687 1.587
Frailty 1.0000 1.082 0.710 1.650
TotalRxGaps 1.0000 1.108 0.986 1.246
Unplanned_Inpt_Hosp_ 1.0000 1.026 0.862 1.222
Total_Cost_Pred_Risk 1.0000 1.025 0.988 1.063
Probability_Hosp_Inj 1.0000 117.520 1.135 >999.999
Prob_Unplnd_30d_Read 1.0000 1.374 0.088 21.327
Dementia 1.0000 1.274 0.716 2.269
Dehydration 1.0000 1.550 1.024 2.346
Admit3p_7a 1.0000 1.674 1.170 2.394
How to interpret the Odds
Ratio (OR) Estimate: “Patients
who are admitted between
3pm and 7am have a 67%
higher chance of falling than if
they were not admitted
between 3pm and 7am.”
For every $1 in marginal spending to operationalize
the predictive model, how much money would we save
by preventing a patient from falling with injury? The table
below of possible levels of ROI makes the following
reasonable assumptions in the calculation of savings and
spending:
• The average daily charges for an inpatient encounter is
$7305.42.
• The marginal daily charges to prevent a patient from falling is
$51.28 (based on the cost of eight 10-minute assisted walks per
day by a nurse making $80k/year).
• The average length of stay (LOS) of a patient who falls with
injury is 5.48 days. However, with frequent ambulation it can
be 2 days shorter1.
Potential ROI
$4.05 $3.47 $2.46 $2.28 $2.16
# of patients selected by decision rule in a year 1,551 2,130 3,288 4,528 6,020
# of falls patients captured 28 33 36 46 58
# of false positives 1,523 2,097 3,252 4,482 5,962
Potential savings $1.11M $1.32M $1.44M $1.84M $2.32M
Money spent to get the savings $276k $379k $585k $806k $1.07M
Net gain $842k $938k $852k $1.03M $1.24M
Criteria in decision rule for patient to be selected as a falls candidate:
Minimum risk score based on model OR estimates2 12.43 12.43 8 8 7.3
Frail X X X X X
Has many diagnoses (at least 16) X X X X
Has probability of hospital injury) >0 X X X X
Depressed X X X
No history of falling X
Admitted between 3pm and 7am X
Has at least one prescription drug gap X
Has prob(unplanned 30d readmission)>=0.08 X
LOWER
ROI
HIGHER
ROI
Easier for IT to
implement
Higher “Profit” in
absolute dollars
More manageable
caseload
More precise with
fewer “false positives”
If the goal is this: Then do this:
Prevent the greatest number of
patients from falling
Implement decision rule with the least restrictions possible
while still achieving positive ROI
Maximize ROI Implement strictest decision rule that is feasible within budget
Both are good, can't decide!
Compromise by implementing decision rule somewhere in the
middle (for example, one of the two requiring
a minimum total risk score of 8).
RESULTS
RETURN ON INVESTMENT