4. A Perfect Representation of the Machine Learning Cycle from start to end | Image Source:
MLOps (Published under Creative Commons Attribution 4.0 International Public License and used
First model wasn’t good
General model
sun or rain?
Intubation model?
Identify the deterioration?
Predict the intubation?
Peditc an respiratory deterioration?
Deterioration – when?
What’s a good alg
Specificity = FPR = fall out
Sensitivity = TPR = recall = hit rate
PPV (positive Predictive Power ) precision = TP/ TP+FP
Accuracy (tp +tn) / (tp+tn +fp +fn)
27,000 seconds with cases
Only ~7,000 are actually relevant
(out of 60,000,000 minutes)
HR, BP (nurse effect)
Different systems
Changes in origin systems
Timing from different systems
Imputation
Challlenge 4 – what about missing data
1. do nothing
2. mean/median (bad for catrgorical, not accurate, doesn’t take uncertainity
3. Most freq/constant/zero – doesn’t factor correlation, can introduce bias
4. Find similar (k-nn) lots of compute
5 deep leadring – even more compute
Demographics (categorical
Doctot’s notes <- can we even use it? is it self fulfilling? (also how to handle PII)
Time series
Intubation is a judgment call – are we
What’s a good “quiet” period
Semi-supervised learning can help
27,000 seconds with cases
Only ~7,000 are actually relevant
(out of 60,000,000 minutes)
Undersampling, oversampling
Initial efforts
Deep learning over fit
Trees
Ensamble models
Training set – construct classifier
Validation set – fine tune model (change parmeters)
Test set – estimate future error – the generalization of the model
Data limit
New systems at the client
Timing
The difference between theory and reality is that in theory they are the same
Need lots of historical data for each new data point
Need to generalize for different systems