Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Patient Journey Record (PaJR)           Online Prediction SystemJing Su, Lucy Hederman, Atieh Zarabzadeh, Dee Grady, Carme...
Avoidable hospitalisations• PaJR is a telephone service that targets avoidable hospitalisations• Most hospital admissions ...
$/day$5,000$1,000$100                                           Early        Alertsof                                     ...
The PaJR Service  Michael Anon                 HISI 2011   4
The PaJR Survey                  Michael Anon      HISI 2011                  5
The PaJR approach       • Use lay callers• Ask the questions that predict         hospitalisation    • Not disease specifi...
PaJR Prediction Service            Michael Anon                           • Predicts patient                             d...
Machine LearningML: automatically induce from the examples a model that accurately        predicts new cases.             ...
Simple predictive model:       a decision tree                                                             A   - decision ...
Machine learning• ML Tools such as Weka or Timbl provide  algorithms to produce prediction models  from examples (“trainin...
PaJR‟s Current ML• Predicts unplanned events, urgent  unplanned events, self rated health.• Uses decision trees.• Weights ...
Accuracy• Predicting urgent  unplanned events                             True   False  (UUE)                             ...
Error Analysis• False positive rate is worth further investigation:   – ML predicts an urgent unplanned event.   – No urge...
Challenges• Data  – ML requires lots of examples of each outcome.    Thanks to PaJR the number of unplanned events    amon...
Benefits of Machine Learning• Compared with static rule-based alerts   – ML allows identification of features that emerge ...
Thank youAny questions?   Lucy Hederman  hederman@tcd.ie       HISI 2011    16
Upcoming SlideShare
Loading in …5
×

PaJR Presentation at Health Informatics Society of Ireland - Oct 2011

1,710 views

Published on

PaJR Presentation at Health Informatics Society of Ireland - Oct 2011

Published in: Health & Medicine
  • Be the first to comment

  • Be the first to like this

PaJR Presentation at Health Informatics Society of Ireland - Oct 2011

  1. 1. Patient Journey Record (PaJR) Online Prediction SystemJing Su, Lucy Hederman, Atieh Zarabzadeh, Dee Grady, Carmel Martin, Kevin Smith, Carl Vogel, Enda Madden, Brendan Madden Trinity College Dublin, UCC, PHC Research, GroupNos HISI 2011 1
  2. 2. Avoidable hospitalisations• PaJR is a telephone service that targets avoidable hospitalisations• Most hospital admissions • are in older, sicker people with multiple diseases and conditions • are unpredictable in the short term with current systems HISI 2011 2
  3. 3. $/day$5,000$1,000$100 Early Alertsof deterioration helps prevent the patient$10 entering expensive treatment$1 Self care Complicated Complex Hospital HISI 2011 3
  4. 4. The PaJR Service Michael Anon HISI 2011 4
  5. 5. The PaJR Survey Michael Anon HISI 2011 5
  6. 6. The PaJR approach • Use lay callers• Ask the questions that predict hospitalisation • Not disease specific • Intervene early • Alert accurately … • ~ 50% reduction in hospital admissions in pilot sites HISI 2011 6
  7. 7. PaJR Prediction Service Michael Anon • Predicts patient deterioration based on record of call. – Self-rated health, taking meds,… – Brief text entries• Uses a predictive model learned from examples of calls leading to unplanned events. HISI 2011 7
  8. 8. Machine LearningML: automatically induce from the examples a model that accurately predicts new cases. HISI 2011 8
  9. 9. Simple predictive model: a decision tree A - decision node A yes no - leaf node K=y (K means B B UnplannedEvent) <n ≥n <p ≥p C A sample decision tree K=y K=n K=y on UnplannedEvent yes noEg: A = “her sister”;B = AvgWordLength; K=n K=yC = takingMeds HISI 2011 9
  10. 10. Machine learning• ML Tools such as Weka or Timbl provide algorithms to produce prediction models from examples (“training data”).• The examples must be presented to ML tool as a collection of features.• Expertise and skill is needed to identify / derive / represent features of examples that might predict the outcome. HISI 2011 10
  11. 11. PaJR‟s Current ML• Predicts unplanned events, urgent unplanned events, self rated health.• Uses decision trees.• Weights false negatives 500 more costly than false positives – A missed deterioration is bad. – An inappropraite alert to a carer to call a patient is OK. HISI 2011 11
  12. 12. Accuracy• Predicting urgent unplanned events True False (UUE) Negative 1091 4• Training data (OK) – 1621 phone calls Positive 23 453 – 27 urgent unplanned (UUE) events• False Negatives cost 500 times FPs • Fewer than 1/3 of the calls are incorrectly prioritised • Under 1/6 of the calls that should be prioritised are not. HISI 2011 12
  13. 13. Error Analysis• False positive rate is worth further investigation: – ML predicts an urgent unplanned event. – No urgent unplanned event occurred. – But is that because the PaJR caller intervened (with advice, referral, comfort, …) and averted the event ML predicted?• Analysing FP cases, we found evidence of some intervention in a small number of cases. – Further work needed.• More significantly, UUEs were rarely (6/27) „anticipated‟ by lay callers (they didn‟t intervene), whereas ML predicted 23 of them. HISI 2011 13
  14. 14. Challenges• Data – ML requires lots of examples of each outcome. Thanks to PaJR the number of unplanned events among the users is declining.• Features – We have lots of data for each case but it takes time and skill to identify features predictive of deterioration.• Prediction Engine Pipeline – The management of multiple cases, multiple models, etc. HISI 2011 14
  15. 15. Benefits of Machine Learning• Compared with static rule-based alerts – ML allows identification of features that emerge as predictive of deterioration. – ML uses evidence from data on real patients. – ML can be easily transferred to new settings and new services – ML adapts over time• Compared with experienced callers without ML – ML allows high accuracy, high volume at low cost. – ML will identify features across callers, across time, etc. – ML has perfect memory – callers go on leave, move on. HISI 2011 15
  16. 16. Thank youAny questions? Lucy Hederman hederman@tcd.ie HISI 2011 16

×