1) The document discusses building lifesaving AI solutions in the medical field using JavaScript techniques. It describes 5 phases of building a medical AI product: data refining and model building, deploying backend/data pipelines, frontend design, real-world analysis and model fitting. 2) It provides details on using TensorFlow.js to run machine learning models in the browser for applications like interactive predictions and 3D visualizations. 3) The talk describes a case study of a company called VitalCare that is piloting an early warning system for acute diseases using these techniques to predict mortality risks and improve survival rates.

1. Lifesaving AI and JavaScript
Jaeman An <jaeman@aitrics.com>
JSConf Korea 2019

2. Introduction to Medical AI
How to build lifesaving AI solutions
Related JavaScript techniques
Various issues to dealing with AI
What you can get from this talk

3. 01 Introduction to Medical AI
02 Building Medical AI Product
- 5 phases
- Various issues & solutions like
- Use AI Model Safely
- Build a reliable data pipeline
- Represent ML model output intuitively
03 Running ML model in the web browser
- TensorFlow.js
04 Where are we and what's next?

4. Introduction to Medical AI

5. What if your code save one's life?

6. +1% survival rate
Save 1 of 100

7. 25 (U.S.)

8. +1% survival rate
Save 2500 of 250000 /1 year

9. Can software engineering and
artificial intelligence
save people's lives?

10. Problems we try to solve (VitalCare)
Early warning system for acute disease
Predict / alert for dangerous severe acute illness
Mortality, Sepsis, cardiac arrest, embolism,
acute kidney injury, ...

11. Acute disease that causes the most deaths and costs
8% / hour for delayed treatment
50% of hospital deaths are related to sepsis
250,000 deaths per year in the United States
$ 24 billion annual medical expenses in the U.S.
Sepsis ("Silent killer" in hospitals)

12. Building Medical AI Solution

13. Provides prediction scores 4~24 hours in advance
• Delayed treatment for sepsis increases mortality 8% every hour
Patient EMR Data collected AI Risk Prediction Solution
Vital sign
Lab Test
Lactate
Creatinine
…
Medical
Imaging
CT-Scan,
X-ray
...
Prescription
Clinical Note
To be
used
Currently
in Use
Risk prediction
for Sepsis
AITRICS Engine
Bayesian Optimization
Bayesian Neural Network
Interpretation Module

15. 5 phases of building Medical AI solution
How To Build Medical AI Solution
Deploying
Backend &
Data pipeline
Frontend
(Web & Mobile)
Real-world data analysis &
fitting the model
Data Refining &
Model Building

16. Data cleansing & preprocessing
Defining outcome
ex) What is death? Does it include "Do Not Rescue" patient?
ex) Predicting patients who died after 24 hours: What if patient died after 25
hours? is it wrong?
Finding the model that can predict the best from given data
Data Refining & Model Building
Deploying
Backend &
Data pipeline
Frontend
(Web & Mobile)
Real-world data analysis &
fitting the model
Data Refining &
Model Building

17. Make ML model as a microservice
Deploy on the web browsers with TensorFlow.js
Require model optimization & compression
Using AI Model Safely
How to evaluate model's output? is it trustful?
What if model returns incorrect results?
Deploying
Deploying
Backend &
Data pipeline
Frontend
(Web & Mobile)
Real-world data analysis &
fitting the model
Data Refining &
Model Building

18. Unintended predictions with unfamiliar data
Testing ML model for random input values
Property based testing
jsverify, fast-check, ...
Debugging with interpretable module
Withdraw prediction when wrong features have
significant contribution
Using AI Models Safely

19. Property based testing (fast-check)
import * as fc from 'fast-check';
import { predict } from '../src/predict';
const recordProperty = fc.record({
age: fc.number(), lab_BLOOD_CULTURE: fc.float(0.3, 1), ...
});
test('should sepsis risk high who already have sepsis', () => {
fc.assert(fc.property(fc.jsonObject(), recordProperty), record => {
const result = predict(record);
assert result.score >= 0.2;
assert result.contribution.lab_BLOOD_CULTUER >= 0.5;
}));
});

20. Combine deployed model with real-world data
Calculate predictions from incoming data in real time
Build a reliable data pipeline with Node.js
Monitoring data flow
Checking accuracy / consistency periodically
Backend & Data pipeline
Deploying
Backend &
Data pipeline
Frontend
(Web & Mobile)
Real-world data analysis &
fitting the model
Data Refining &
Model Building

21. Building a reliable data pipeline
Pycon KR 2019: Django Query Optimization for realtime medical artificial intelligence data preprocessing
환자 정보
(시계열 데이터)
n시간 뒤 환자가
급성 질환에 걸릴 확률 예측
Dashboard
*
6
급성 질환
예측 솔루션
병원
ML model
실시간 정보 동기화가 중요하다!
Update
Update
회사에서 개발 중인 Django를 이용한 서비스
! 환자A 위급!!

22. Building a reliable data pipeline
Synchronizer
(Python / Golang)
Prediction
(Python)
Medical Score
(Python)
Alert
(Python)
Data Listeners & Generators
Databases
MySQL Redis
Databases
AI Model
Fitting / re-training
Backup Scheduler
(Node.js)
Trainer Scheduler
(Node.js)
Schedulers
Monitors
Grafana dashboard healthchecks.io
Monitor
(Node.js)
Controllers
Etomer
(Node.js)
Etomer Web
Etomer Slack
Hospital
Pub/Sub (Redis Streams)

23. Building a reliable data pipeline
Synchronizer
(Python / Golang)
Prediction
(Python)
Medical Score
(Python)
Alert
(Python)
Data Listeners & Generators
Databases
MySQL Redis
Databases
AI Model
Fitting / re-training
Backup Scheduler
(Node.js)
Trainer Scheduler
(Node.js)
Schedulers
Monitors
Grafana dashboard healthchecks.io
Monitor
(Node.js)
Controllers
Etomer
(Node.js)
Etomer Web
Etomer Slack
Hospital
Pub/Sub (Redis Streams)

24. Monitor & controller
Monitor
Consume from Redis Streams
Send metrics to Grafana with StatsD
Easy to build with Node.js (event listener)
setInterval
Healthcheck
Check service metrics
Send summary to Slack
Controller
Check & Ops with Slack API
Server update

25. Frontend (Web & Mobile)
Deploying
Backend &
Data pipeline
Frontend
(Web & Mobile)
Real-world data analysis &
fitting the model
Data Refining &
Model BuildingOne of the most important process when making AI model into a product
Use of various visualization libraries & insightful pictures
How to interpret & visualize model output?

26. How ML model works
What medical staff wants to see
Intuitive representation of ML model output
Patient Data 0.015

27. Statistical analysis & Contribution analysis
- Prediction score vs
Mortality rate
Disclaimer: these are from the ECharts examples, not the real graph
- Correlation between
outcomes & features
Feature contributions -
Symptoms & diseases -

28. Visualizing model output & interpretation

29. AI models must be fitted in the real world
Data distribution changes
People behave changes
Ex) If VitalCare predict sepsis and send alert 4 hours before, medical
staff respond to it, then the pattern of sepsis would change
How the AI model keeps learning from real world data?
Real-world data analysis & Fitting the model
Deploying
Backend &
Data pipeline
Frontend
(Web & Mobile)
Real-world data analysis &
fitting the model
Data Refining &
Model Building

30. Building a reliable data pipeline
Synchronizer
(Python / Golang)
Prediction
(Python)
Medical Score
(Python)
Alert
(Python)
Data Listeners & Generators
Databases
MySQL Redis
Databases
AI Model
Fitting / re-training
Backup Scheduler
(Node.js)
Trainer Scheduler
(Node.js)
Schedulers
Monitors
Grafana dashboard healthchecks.io
Monitor
(Node.js)
Controllers
Etomer
(Node.js)
Etomer Web
Etomer Slack
Hospital
Pub/Sub (Redis Streams)

31. Building a reliable data pipeline
Synchronizer
(Python / Golang)
Prediction
(Python)
Medical Score
(Python)
Alert
(Python)
Data Listeners & Generators
Databases
MySQL Redis
Databases
AI Model
Fitting / re-training
Backup Scheduler
(Node.js)
Trainer Scheduler
(Node.js)
Schedulers
Monitors
Grafana dashboard healthchecks.io
Monitor
(Node.js)
Controllers
Etomer
(Node.js)
Etomer Web
Etomer Slack
Hospital
Pub/Sub (Redis Streams)

32. Re-training model periodically
Validation with
Current/Past data
External data
(Next) Continuous learning
Trainer for Auto ML
Data
Original Data 9/1 9/2 . . .
External Data
Hospital A Hospital B
Model
Version 1
(8/30)
Version 2
(9/1)
Version 3
(9/2)
AUC: 0.87
Ext AUC: 0.79
AUC: 0.78
Ext AUC: 0.77
AUC: 0.97
Ext AUC: 0.80
. . .

34. Library for machine learning in JavaScript
You can ...
use pre-trained models
convert existing Python models
train in the browser and Node.js
Running ML models in the web browser
import * as tf from '@tensorflow/tfjs';
function createAndCompileModel(type,
inputLength, hiddenSize, learningRate) {
// Encoder
const model = tf.sequential({
layers: [
tf.layers.dense({ units: hiddenSize,
activation: 'relu', name: 'EncoderFC1',
inputShape: [inputLength, 1] }),
tf.layers.dense({ units: hiddenSize,
activation: 'relu', name: 'EncoderFC2' }),
]
});
...
}

36. Interactive real-time prediction
Reduce server load
Visualize ML model
(-) Not all support TensorFlow API
(-) Poor community
Why Running ML Model in the web browser?

37. Model 3D Visualization (TensorSpace.js)

38. Where are we and what's next?

39. VitalCare is piloting at n hospitals in Korea
Significant performance improvement compared to existing
Expect to prove effects on improving survival rate
Many good feedbacks: can save lives by detecting
dangerous patients early, reduce my work, ...
Current status of VitalCare

40. Next things to do?
Prove that the AI solution can save one's life
through the long-term experiment
Save lives as many as possible

41. Expansion for predictive disease
Prescription recommendations
Running / training ML model on Web/Mobile
ML pipelines for accelerating research
ML Training Platform - kono
(GTC Silicon Valley 2019: How To Build ML Pipelines From The Startup Perspective)
ML on the other areas ...
Technical things to do

42. Thank you!
Jaeman An <jaeman@aitrics.com>

43. ML Ops/Engineering plays a key role in solving problems with AI
Artificial Intelligence as a Software Engineer
Research Engineering
Building AI algorithm that solves problem A Building AI solution that solves problem A
How do you get best accuracy in the certain situation?
Which model should you use to interpret the results of
the AI model?
...
How do you make an AI model a service?
How do you interpret / visualize the results from the
model?
What if the model gives wrong results?
How do you make your AI models continually trained?
What architecture should I construct to train AI models
efficiently?
...