Taking ML to production - a journey
•Download as PPTX, PDF•
0 likes•193 views
Go over some of the challenges in taking a medical machine learning solution from inception and into production
Recommended
More Related Content
Similar to Taking ML to production - a journey
Similar to Taking ML to production - a journey (20)
More from Arnon Rotem-Gal-Oz
More from Arnon Rotem-Gal-Oz (20)
Recently uploaded
Recently uploaded (20)
Taking ML to production - a journey
- 1. Taking ML to production - A journey Arnon Rotem-Gal-Oz
- 2. Mental model of the probelm Admission Intubation Alert >6 hours
- 3. Challlenge 1 defining the problem
- 4. A Perfect Representation of the Machine Learning Cycle from start to end | Image Source: MLOps (Published under Creative Commons Attribution 4.0 International Public License and used
- 6. Challenge 2 – how we measure
- 9. Challenge 5 different types of data Model(s) text time series categorical
- 10. Challenge 6 labeling Admission Intubation
- 12. Challenge 8 Model experimentation cycle
- 13. Modeling
- 14. Challenge 9 – Overfit
- 17. Challenge 10 – model degredation in production Theory Reality
- 18. Challenge11 – Is it really generalized?
- 19. Challenge 12 model validation and verification
Editor's Notes
- 1,000,000 hours 1 min 500 datapoints
- First model wasn’t good General model sun or rain? Intubation model? Identify the deterioration? Predict the intubation? Peditc an respiratory deterioration? Deterioration – when?
- ”always no” alg ROCSpecificity = FPR = precision Sensitivity = TPR = recall
- What’s a good alg Specificity = FPR = fall out Sensitivity = TPR = recall = hit rate PPV (positive Predictive Power ) precision = TP/ TP+FP Accuracy (tp +tn) / (tp+tn +fp +fn) 27,000 seconds with cases Only ~7,000 are actually relevant (out of 60,000,000 minutes)
- HR, BP (nurse effect) Different systems Changes in origin systems Timing from different systems
- Imputation Challlenge 4 – what about missing data 1. do nothing 2. mean/median (bad for catrgorical, not accurate, doesn’t take uncertainity 3. Most freq/constant/zero – doesn’t factor correlation, can introduce bias 4. Find similar (k-nn) lots of compute 5 deep leadring – even more compute
- Demographics (categorical Doctot’s notes <- can we even use it? is it self fulfilling? (also how to handle PII) Time series
- Intubation is a judgment call – are we What’s a good “quiet” period Semi-supervised learning can help
- 27,000 seconds with cases Only ~7,000 are actually relevant (out of 60,000,000 minutes) Undersampling, oversampling
- Initial efforts Deep learning over fit Trees Ensamble models
- Training set – construct classifier Validation set – fine tune model (change parmeters) Test set – estimate future error – the generalization of the model Data limit
- New systems at the client Timing The difference between theory and reality is that in theory they are the same
- Need lots of historical data for each new data point Need to generalize for different systems