PARR-30: a predictive model for
 readmission within 30 days

 Presenter: Ian Blunt




19 June 2012                       © Nuffield Trust
Development of a predictive model for readmission within 30
days of discharge (PARR-30)

Model developed by Billings, Blunt, Steventon, Georghiou, Lewis and
Bardsley
•   Motivation
•   Development
•   Model performance
•   Testing in hospitals
•   Conclusions




                                                                  © Nuffield Trust
Why predict readmissions within 30 days?


•   Readmissions are costly, suboptimal health care - costs to the NHS
    estimated at £1.6 billion each year
•   DH guidance for the NHS proposes commissioners do not pay
    provider hospitals for emergency readmission within 30 days of a
    selected index elective admission
•   Rate of readmissions will also play an important part in monitoring
    health system performance, as one of the new English Public Health
    “outcome indicators”




                                                                       © Nuffield Trust
Not first to try this, but…


Number of international 30 day models           Model             From      C
                                                                         statistic
Predictive tools built in one setting    Halfon et al 2008                0.67
may not necessarily be accurate when
used in other health care settings       Silverstein et al                0.65
                                         2008
Used hospital episode statistics (HES)
data to develop model for NHS in         Van Walraven et al               0.68
England                                  2010
                                         Howell et al 2009                0.65
Make PARR-30 freely available for
use across the NHS in England
                                         And others…
(possibly tablet/smartphone app)
                                         See Kansagara et al JAMA 2011

                                                                           © Nuffield Trust
How is PARR30 different from PARR++?


•   Readmission in next 30 days vs next 365 days
•   Tools operate in different ways, trigger different responses
•   Next year – longer for clinicians and care managers/coordinators to contact
    and engage with high-risk patients, effect behavioural change
•   30 days – highest likelihood of an unplanned admission, focussing their
    discharge planning efforts and post-discharge support for high-risk patient
Model         Timescale       Run by         Input         Data        Data lag
                                           variables      source
PARR++        12 months        PCT           ~250           SUS      ~ 3 months
PARR30          30 days        Acute           17        PAS/notes      None
•   Aim for speed of low-variable models with accuracy of PARR
                                                                             © Nuffield Trust
How is PARR30 different from PARR++?

                                                      PARR30
                  Hospital provides SUS

                                                                Patient nears discharge




PCT runs PARR++
                                                   Risk score calculated on
                                                   ward
                        Patients selected for
                        intervention (via GP)

                                                                       Any extra intervention put
                                                                       in discharge plan




   Predicts readmission in next year –          Predicts readmission in 30 days© Nuffield Trust
   PPV 65%                                      – PPV ???%
Model development


Developed using 10% sample HES
from April 2006 to May 2009
Index discharges in FY 2008/09
Readmissions within 30 days
reflected 2011-12 operating
framework
Logistic regressions identify variables
that contributed most to predictions
Validated with split sample



                                          © Nuffield Trust
Model development

         Hospital of current admission
                            Patient age
            Deprivation (via post code)
    History of emergency admissions:
   Current? Last 30 days? Past year?


History in the prior two years of eleven
   major health conditions drawn from
       the Charlson co-morbidity index




                                           © Nuffield Trust
Results


The performance of the model
was respectable, with a positive
predictive value (PPV) of
59.2% and area under the ROC
curve (“c-statistic”) of 0.70.


For the higher-risk patients (risk
score > 50%), readmission rates
ranged from 47.7% up to 88.7%.
However, these patients only
represented a small share
(1.1%) of all patients analysed.     Receiver Operating Characteristic Curve (ROC) for the
                                     bootstrapped central estimate (red line) and 95%
                                     confidence Intervals (shaded area)                      © Nuffield Trust
Results

                                                                    £3,000
Predictive modelling only as effective
as the intervention it is used to
                                                                    £2,500
trigger. Providers need to know




                                         Mean cost of readmission
potential costs of readmission to                                   £2,000
build business case for intervention
                                                                    £1,500


For patients risk score > 50%, mean                                 £1,000
readmission cost per patient was
                                                                     £500
£1,088. Assuming that an
intervention can reduce the number
                                                                       £0
of readmissions by 10% for this                                                                Risk score
group, £109 per patient could be                                      Mean of cost readmission (readmitted patients only)
spent at breakeven                                                    Mean of cost readmission (all patients)    © Nuffield Trust
Testing PARR-30 in hospitals


Testing:                              Royal Berkshire Hospital using
                                      spreadsheet version of tool on wards:
•   Is the tool easy to use?
                                      •   Completed by junior doctors
•   Bedside info vs admin systems?
                                      •   Test tool stored its output
•   Does ward PPV reflect national?
                                      •   Later reconciled with admin
                                          systems for analysis
Chelsea & Westminster Hospital
running tool direct from their data
warehouse:                            Applied on four care of the elderly
                                      wards in Feb/March 2012
•   Proved it can be done easily
•   Looking into PPV and clinical
    engagement                                                          © Nuffield Trust
Testing PARR-30 in hospitals

Results from using spreadsheet on the wards:

•   Tool was used 88 times              •   Low risk scores – max 39%
•   Median time to complete 1m 41s      •   Em admit in last 30 days diff 10%
•   Median patient age was 86,          •   Em admits last year diff 20%, ±1,2
    mostly emergency admissions
                                        •   Even split whether tool or system
•   Average 1.3 co-morbidities, max 4       has more
•   10 patients had emergency           •   14% where system has diagnosis
    readmission within 30 days              not ticked as co-morbidity



                                                                          © Nuffield Trust
Conclusions


Built a predictive model using a limited set of variables that were
generated from hospital episode statistics
Variables easily available from patients’ notes or from the patients
themselves – can calculate from spreadsheet or in PAS
The performance of the model was respectable - highest risk patients
had a 88.7% chance of hospital readmission within 30 days – but high
risk patients relatively rare
Cost data suggests interventions need to be lower-cost to break even
Easily used on wards in trials - less than 2 minutes per application
Some differences in data on ward, but not huge
                                                                       © Nuffield Trust
www.nuffieldtrust.org.uk


         Sign-up for our newsletter
         www.nuffieldtrust.org.uk/newsletter/login.aspx


         Follow us on Twitter
         (http://twitter.com/NuffieldTrust)




19 June 2012                                              © Nuffield Trust

Ian Blunt: PARR-30: a predictive model for readmission within 30 days

  • 1.
    PARR-30: a predictivemodel for readmission within 30 days Presenter: Ian Blunt 19 June 2012 © Nuffield Trust
  • 2.
    Development of apredictive model for readmission within 30 days of discharge (PARR-30) Model developed by Billings, Blunt, Steventon, Georghiou, Lewis and Bardsley • Motivation • Development • Model performance • Testing in hospitals • Conclusions © Nuffield Trust
  • 3.
    Why predict readmissionswithin 30 days? • Readmissions are costly, suboptimal health care - costs to the NHS estimated at £1.6 billion each year • DH guidance for the NHS proposes commissioners do not pay provider hospitals for emergency readmission within 30 days of a selected index elective admission • Rate of readmissions will also play an important part in monitoring health system performance, as one of the new English Public Health “outcome indicators” © Nuffield Trust
  • 4.
    Not first totry this, but… Number of international 30 day models Model From C statistic Predictive tools built in one setting Halfon et al 2008 0.67 may not necessarily be accurate when used in other health care settings Silverstein et al 0.65 2008 Used hospital episode statistics (HES) data to develop model for NHS in Van Walraven et al 0.68 England 2010 Howell et al 2009 0.65 Make PARR-30 freely available for use across the NHS in England And others… (possibly tablet/smartphone app) See Kansagara et al JAMA 2011 © Nuffield Trust
  • 5.
    How is PARR30different from PARR++? • Readmission in next 30 days vs next 365 days • Tools operate in different ways, trigger different responses • Next year – longer for clinicians and care managers/coordinators to contact and engage with high-risk patients, effect behavioural change • 30 days – highest likelihood of an unplanned admission, focussing their discharge planning efforts and post-discharge support for high-risk patient Model Timescale Run by Input Data Data lag variables source PARR++ 12 months PCT ~250 SUS ~ 3 months PARR30 30 days Acute 17 PAS/notes None • Aim for speed of low-variable models with accuracy of PARR © Nuffield Trust
  • 6.
    How is PARR30different from PARR++? PARR30 Hospital provides SUS Patient nears discharge PCT runs PARR++ Risk score calculated on ward Patients selected for intervention (via GP) Any extra intervention put in discharge plan Predicts readmission in next year – Predicts readmission in 30 days© Nuffield Trust PPV 65% – PPV ???%
  • 7.
    Model development Developed using10% sample HES from April 2006 to May 2009 Index discharges in FY 2008/09 Readmissions within 30 days reflected 2011-12 operating framework Logistic regressions identify variables that contributed most to predictions Validated with split sample © Nuffield Trust
  • 8.
    Model development Hospital of current admission Patient age Deprivation (via post code) History of emergency admissions: Current? Last 30 days? Past year? History in the prior two years of eleven major health conditions drawn from the Charlson co-morbidity index © Nuffield Trust
  • 9.
    Results The performance ofthe model was respectable, with a positive predictive value (PPV) of 59.2% and area under the ROC curve (“c-statistic”) of 0.70. For the higher-risk patients (risk score > 50%), readmission rates ranged from 47.7% up to 88.7%. However, these patients only represented a small share (1.1%) of all patients analysed. Receiver Operating Characteristic Curve (ROC) for the bootstrapped central estimate (red line) and 95% confidence Intervals (shaded area) © Nuffield Trust
  • 10.
    Results £3,000 Predictive modelling only as effective as the intervention it is used to £2,500 trigger. Providers need to know Mean cost of readmission potential costs of readmission to £2,000 build business case for intervention £1,500 For patients risk score > 50%, mean £1,000 readmission cost per patient was £500 £1,088. Assuming that an intervention can reduce the number £0 of readmissions by 10% for this Risk score group, £109 per patient could be Mean of cost readmission (readmitted patients only) spent at breakeven Mean of cost readmission (all patients) © Nuffield Trust
  • 11.
    Testing PARR-30 inhospitals Testing: Royal Berkshire Hospital using spreadsheet version of tool on wards: • Is the tool easy to use? • Completed by junior doctors • Bedside info vs admin systems? • Test tool stored its output • Does ward PPV reflect national? • Later reconciled with admin systems for analysis Chelsea & Westminster Hospital running tool direct from their data warehouse: Applied on four care of the elderly wards in Feb/March 2012 • Proved it can be done easily • Looking into PPV and clinical engagement © Nuffield Trust
  • 12.
    Testing PARR-30 inhospitals Results from using spreadsheet on the wards: • Tool was used 88 times • Low risk scores – max 39% • Median time to complete 1m 41s • Em admit in last 30 days diff 10% • Median patient age was 86, • Em admits last year diff 20%, ±1,2 mostly emergency admissions • Even split whether tool or system • Average 1.3 co-morbidities, max 4 has more • 10 patients had emergency • 14% where system has diagnosis readmission within 30 days not ticked as co-morbidity © Nuffield Trust
  • 13.
    Conclusions Built a predictivemodel using a limited set of variables that were generated from hospital episode statistics Variables easily available from patients’ notes or from the patients themselves – can calculate from spreadsheet or in PAS The performance of the model was respectable - highest risk patients had a 88.7% chance of hospital readmission within 30 days – but high risk patients relatively rare Cost data suggests interventions need to be lower-cost to break even Easily used on wards in trials - less than 2 minutes per application Some differences in data on ward, but not huge © Nuffield Trust
  • 14.
    www.nuffieldtrust.org.uk Sign-up for our newsletter www.nuffieldtrust.org.uk/newsletter/login.aspx Follow us on Twitter (http://twitter.com/NuffieldTrust) 19 June 2012 © Nuffield Trust