This presentation was delivered by PBT Australia's Chief Data Scientist Martin Rennhackkamp and Director Department Intensive Care at Austin Health Dr Graeme Hart.
It summarises a project carried out in collaboration between PBT Australia and Austin Health to analyse data around unplanned hospital readmissions and predict likelihood based on a range of factors. If you would like more information about this presentation please get in touch with PBT Australia on LinkedIn or via our website https://www.pbtaustralia.com/contact-us
1. HDA16 - Predicting 30-day
unplanned hospital readmissions
Martin Rennhackkamp
Chief Data Scientist, PBT Australia
www.pbtaustralia.com
blog: www.martinsights.com
Dr Graeme Hart
Director Dept Intensive Care
Austin Health
Victoria
2. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Agenda
1. The project
2. Unplanned readmissions predictive model
3. Operationalising predictive analytics
4. Next steps
2
3. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Health Insights Challenge
4. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Mark Petty
Austin Health
Project sponsor
Dr Graeme Hart
Austin Health
Team lead
Sanjiv Govind
Deakin University
Research student
Martin
Rennhackkamp
PBT Australia
Team lead
Susan Walker
Entity Three
Team lead / SME
Jenni Doore
Entity Three
SME
Damminda
Alahakoon
Deakin University
Assoc Prof
Ali Tamaddoni
Deakin University
Assoc Prof
Amy McKimm
CHI
Dir of
Transformation
Ray Robbins
Austin Health
Snr Data Analyst
Ronald Ma
Austin Health
Snr Data Analyst
Mark Brown
PBT Australia
DBA
Project Team
5. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Timeline
5
PHASE 1 - POC Model & Preliminary Insights PHASE 2 - Final Model & Actionable Insights
Dec '14 Jan '15 Feb '15 Mar '15 Apr '15 May '15 June '15 July '15
ACTIVITY w1 w2 w3 w4 w1 w2 w3 w4 w1 w2 w3 w4 w1 w2 w3 w4 w1 w2 w3 w4 w1 w2 w3 w4 w1 w2 w3 w4 w1 w2 w3 w4
HIC Finalist Announced
Kick-off Meeting – Austin Health
Literature Review
Review Board Meeting 1
Received Datasets from AH
Data Cleansing & Preparation
Data Preparation Review (AH)
Review Board Meeting 2
HIC Progress Meeting 1 with AH
Data Exploration & Visualisation
1
Review Board Meeting 3
HIC Progress Meeting 2 with AH
POC Predictive Model
Review Board Meeting 4
HIC Progress Meeting 3 with AH
HIC Breakfast Event 1 @ CHI
Received Validation Dataset
(AH)
Data Exploration & Visualisation
2
Final Predictive Model &
Validation
Review Board Meeting 5
HIC Progress Meeting 4 with AH
HIC Executive Workshop with
AH
HIC Breakfast Event 2 @ CHI
6. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Our Approach
6
Problem
Definition
Literature
Review
Data
Preparation
Data
Exploration
Predictive
Modelling
Model
Assessment
Implement
Model
7. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Problem Definition
7
Practical Outcomes Analytical Tasks
To be the exemplar amongst its peers in
deploying effective predictive models that
reduce unplanned readmissions.
Identify drivers of unplanned hospital
readmission.
Predict the likely impact of avoidable bed days
on financial outcomes, throughput and waiting
lists.
Identify the most “at risk” patient groups for 30-
day unplanned readmissions.
Improve operational and human resources
planning.
Improve patient discharge planning processes.
Predict likelihood of 30-day unplanned
readmissions occurring.
Evaluate the impact of the predicted 30-day
unplanned readmissions.
“Austin Health plans to develop a validated business and quality
improvement case to justify expenditure to redesign its discharge planning
process and reduce unplanned readmissions”
8. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Variables Identified in Research
8
9. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Data Preparation
9
Training dataset: 5 years data from the period 1 July 2009 to 31 October 2014
No. Activity Data Tables Extract Filename No. of Records PoC
1 Emergency department ed_episode 382,936
2 Inpatient "spells" ip_spell 291,068 x
3 Inpatient episodes ip_episode 308,069 x
4 Inpatient co-morbidities ip_episode_comorbidity 308,069 x
5 Inpatient ICD coding ip_episode_icd_code 2,194,635
6 Inpatient ward transfers ip_episode_ward_transfer 527,096
7 Inpatient transfer detail ip_episode_transfer_detail 1,044,914
No. Reference Tables Extract Filename No. of Records PoC
8 Ward ward 67 x
9 Clinical unit unit 136 x
10 Hospital campus hospital_campus 396 x
No. External Reference Tables Extract Filename No. of Records PoC
11 ICD-10 code descriptions external 5373 (used) x
12 DRG code descriptions external (AR-DRG) 679 x
13 Language code descriptions external 85 (used) x
14 Ethnic code descriptions external 9 (used) x
Initial Data
Extract
308,069
Minus Regular Attenders +
Dr. Fosters Business Rules
280,317 280,078
Filtered data
to match FY
11,952
Unplanned 30-day
Readmissions
56,831
Unplanned
Readmissions
Episode Data
10. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Unplanned Readmissions – 5 year trend
10
Austin Health’s 30-Day ‘all-cause’ unplanned
readmissions 5-year average = 4.51%
Austin’s 30-Day unplanned readmission rates appear to
be decreasing year-on-year
11. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
ICD10 (Roll-Up) by Age Group
11
12. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Journeys of Frequent Readmitting Patients
12
13. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Journeys of Frequent Readmitting Patients by First
Admit ICD-10 Roll-Up
13
14. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Model Validation Technique Incorporating Validation
Dataset
14
• Dataset partitioned into
“training” and “validation”
datasets.
• Several variable selection
techniques are performed;
• Followed by multiple
variable transformations;
• Missing values are
imputed.
• Modeling techniques:
• Forward Regression;
• Backward Regression;
• Decision Trees.
• Models are
compared;
• Best model
selected.
• Out-of-sample dataset
is used to validate the
model’s discriminative
ability on “unseen” data
and to test for overfit
1 2 3 5 6
• Dataset of 280,000 IP
episodes, 141 variables;
• Used variables that were
available at admissions;
SAS Enterprise Miner Report
Process Flow Diagram
4
15. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Significant Predictor Variables
15
Number of ED Episodes *
Total number of Spells (Lifetime) *
Total Length of Stay Days (Lifetime) *
Admissions Ward *
Charlson Score at Admission
Admissions Unit *
Length of Stay Days **
Length of Stay Type **
Age *
Mild Liver Disease Flag at Admission
Renal Disease Flag at Admission
Malignancy Flag at Admission
COPD Flag at Admission
Moderate – Severe Liver Disease Flag at Admission
Number of Spells
Admission Source
Care Type
Cerebrovascular Disease Flag at Admission
Non Chronic Diabetes Flag at Admission
Dementia Flag at Admission
20 Significant Variables
SAS Enterprise Miner Report
Selected Variable Importance
16. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Predicted Unplanned Readmissions within 30 Days
16
Key Finding: Many predicted unplanned readmissions occurred after 30 days
“Most flexible model allowed users to define their own risk threshold”
Predicted Unplanned Readmissions after 30 Days
17. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Model Comparison to Literature Review
18. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Model Performance – Out of Sample dataset
18
Key Metrics POC Model Description
Sensitivity 70% Probability of identifying patients who are likely to readmit (event)
Specificity 74% Probability of identifying who are not likely to readmit (non-event)
Accuracy 74% Accuracy of events & non-events as a % of total episodes
Misclassification Rate 26% Overall prediction error
19. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Predicted Likelihood of Unplanned Readmissions as
an Alert
20. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Potential Impact of Acting on High-Likelihood Readmissions
20
Managing 6 patients per day will affect
1,132 unique patients per year and their
2,227 episodes per year, which if all were effective would have had
a potential opportunity of 9,287 bed-days
that had direct costs of $9,007,799 in FY2014
“The approach is continuing to get attraction although quantification of intervention is lacking”
21. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Analytical Outcomes Integrated in Operational
Systems
21
PAS
EMR
Pharma
cy
Patholog
y
Analytical model process
Reports and dashboards
BI Ecosystem
22. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
Next Steps
Tactical Strategic
Model Deployment
• Implement model to redesign
discharge planning
• Visualisation / dashboard
design
• AaaS options
• Early Adopter Hospital
Data & Analytics Strategy
• Analytics Roadmap
• Data governance
• Other quality improvement
models / simulations
Model Refinement
• Localise for hospital datasets
• Input additional data
(pharmacy, pathology, etc.) to
improve model’s
discriminative ability
National / State Exemplar
• Publication based on applied
model
• Impact on public policy
• State-level model validation
23. PBT Group Australia – Predicting 30-day unplanned hospital readmissions
23
www.martinsights.com
martin.rennhackkamp@pbtaustralia.com
au.linkedin.com/in/rennhackkamp
@Rennhackkamp
info@PBTAustralia.com
www.PBTAustralia.com
Thank
You
