Mentoring Roundtable Discussion “Application of Medical AI”

Mentoring Roundtable Discussion
“Application of Medical AI”
AMI Inc.
Operating Officer
Development Div. Team Leader
Takashi Nakano

2021 AMI Inc. ALL rights reserved.
2
About this documents
• This document has been revised based on the material presented at NeurIPS Meetup Japan
2021 .

3
AMI Inc.
• AMI Inc.
• https://ami.inc/
• We’re developing Super stethoscope
• Stethoscope with remote medical diagnosis support function
stethoscope
Super stethoscope
Heart disease
AI
ECG
heartsound

4
Agenda
• Usage of Medical AI
• Application of Medical AI
• Metric of Medical AI
• Features / Difficulties of Medical AI
• Examples of Medical AI Research and Application

5
Usage of Medical AI

6
Usage of Medical AI
• Classification
• Object Detection
• Segmentation
• etc …

7
Usage of Medical AI
• Classification
This Image from
https://www.kaggle.com/ratthachat/aptos-eye-preprocessing-in-diabetic-retinopathy
Eyes
(Detecting Diabetic Retinopathy)
This Image from
https://arxiv.org/abs/1612.00542
Breast
(Breast Mass Classification
from Mammograms)
This Image from
https://www.nature.com/articles/s41598-019-44839-3#Sec14
(Supplementary information)
Teeth
(Detection of Periodontal Bone Loss)
This Image from
A Deep Learning Model to Predict a Diagnosis of Alzheimer Disease by Using 18F-FDG PET of the Brain
Brain
(Alzheimer Disease )
This Image from
Artificial Intelligence-Assisted Polyp Detection for Colonoscopy: Initial Experience
Polyp

8
Usage of Medical AI
• Object Detection
These Images from
A deep learning approach to automatic teeth detection and numbering based on object detection in dental periapical films
Automatic teeth detection and numbering
An Efficient Approach for Polyps Detection in Endoscopic Videos Based on Faster R-CNN
These Images from
Polyp detection

9
Usage of Medical AI
• Segmentation
These Images from
Deep instance segmentation of teeth in panoramic X-ray images
Teeth segmentation
These Images from
Comparative evaluation of instrument segmentation and tracking methods in minimally invasive surgery
Instrument segmentation
These Images from
Surface Muscle Segmentation Using 3D U-Net Based on
Selective Voxel Patch Generation in Whole-Body CT Images
Muscle / Bone Segmentation

10
Application of Medical AI

11
Application of Medical AI
• Image
• fundus image (eye)
• endoscopic image (cancer)
• CT/ MRI (aneurysm/ cancer)
• X-ray (bone)
• Biological signal
• ECG

12
Image : fundus image (eye)
• Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through
Integration of Deep Learning
• Input : fundus images
• Output : diabetic retinopathy
• Model : CNN inspired by Alexnet23 (for more limited training sets) and the Oxford Visual
Geometry Group26 (for more extensive training sets) network architectures
• Result
• Sensitivity : 96.8％, Specificity :87.0％
• Related Company : IDx-DR

13
Image : endoscopic image (cancer)
• Application of artificial intelligence using a convolutional neural network for detecting gastric
cancer in endoscopic images
• Input : endoscopic images
• Output : gastric cancer
• Model : Single Shot MultiBox Detector
• Result
• Sensitivity : 92.2%
• Related Company : AI Medical Service Inc.

14
Image : CT/ MRI (aneurysm/ cancer)
• Deep Learning for MR Angiography: Automated Detection of Cerebral Aneurysms
• Input : MR images
• Output : aneurysm
• Model : ResNet18
• Result (top five candidate points with the highest probability of representing an aneurysm)
• Sensitivity : 93%
• Related Company : LPIXEL

15
• COVID-19 Pneumonia Image Analysis Program
• Input : CT images
• Output : the confidence level of COVID-19 pneumonia
• Result
• AUC : 0.780, Sensitivity : 89.6%, Specificity : 37.1%
• Related Company : MIC Medical, Alibaba Damo Technology, M3
https://corporate.m3.com/press_release/2020/20200629_001616.html
https://www.pmda.go.jp/files/000235943.pdf

16
• Lung nodule detection program
• Output : Pulmonary nodules
• Pulmonary nodules are whitish shadows in the lung area on X-ray or CT images, and if they are
seen, they may indicate lung cancer or other diseases.
• Result
• Sensitivity : 61.4% (doctor + program)
• Related Company : FUJIFILM
https://www.pmda.go.jp/PmdaSearch/kikiDetail/ResultDataSetPDF/671001_30200BZX00150000_A_01_03
https://www.fujifilm.com/jp/ja/news/list/4963

17
• Cancer detection in screening mammography
• Input : mammography images
• Output : cancer (positive / normal)
• Result
• AUC : 0.95, Sensitivity : 86.9%, Specificity :88.5%
• Related Company : CureMetrix
Image from
https://pubs.rsna.org/doi/pdf/10.1148/radiol.2019182908
https://www.accessdata.fda.gov/cdrh_docs/pdf18/K183285.pdf

18
• Denoised PET Images
• Input : PET Images
• Output : Denoised Images
• Related Company : SUBTLE MEDICAL
https://subtlemedical.com/usa/subtlepet/

19
Image : X-ray (bone)
• Deep neural network improves fracture detection by clinicians
• Input : X-ray images
• Output : Fracture
• Model : 2D CNN
• Result
• Sensitivity : 91.5%, Specificity :93.9%
• Related Company : Imagen Technologies

20
Biological signal : ECG
• Deep Neural Networks Can Predict New-Onset Atrial Fibrillation From the 12-Lead ECG and
Help Identify Those at Risk of Atrial Fibrillation–Related Stroke
• Input : 12-lead digital ECG wave data + Age, Sex
• 3 branch from 12-lead ECG
• I, II, V1, and V5, acquired from time (t) = 0 (start of data acquisition) to t=5 [s]
• V1, V2, V3, II, and V5 from t=5 to t=7.5 [s]
• V4, V5, V6, II, and V1 from t=7.5 to t=10 [s]
• Output : Risk of Atrial Fibrillation or not (1 year later)
• Model : 1D CNN
• Result
• AUC : 0.85
• Sensitivity : 69%, Specificity :81%
• Related Company : Tempus
3 branches from 12-lead ECG

21
Metric of Medical AI

22
• Main metric of Medical AI
• Sensitivity (Recall) : Ratio of people who have findings in people of disease
• High Sensitivity : useful for screening.
• If this test is negative, the probability of having the disease is very small.
• Specificity : Ratio of people who have findings in people of no disease
• High Specificity : useful for definitive diagnosis
• If this test is positive, the probability of having the disease are very high.
• AUROC
𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦 𝑅𝑒𝑐𝑎𝑙𝑙 =
𝑇𝑃
𝑇𝑃 + 𝐹𝑁
𝑆𝑝𝑒𝑐𝑖𝑓𝑖𝑐𝑖𝑡𝑦 =
𝑇𝑁
𝐹𝑃 + 𝑇𝑁
positive negative
positive
negative
Test (Prediction)
Disease
(Ground Truth)
TP FN
FP TN
positive negative
positive
negative
Test (Prediction)
Disease
(Ground Truth)
TP FN
FP TN

23
• High Sensitivity
• FN is few
• If test (prediction) is negative, no disease
• Whether a person with a disease be diagnosed as having a disease or not
• Effective when test results are negative, useful for screening.
• Test which High Sensitivity
• If this test is negative, the probability of having the disease is very small.
• e.g. High Sensitivity CRP
• as a marker for detecting micro-inflammation in atherosclerosis,
• It is used to predict the risk of myocardial infarction.
positive negative
positive
negative
Test (Prediction)
Disease
(Ground Truth)
TP FN
FP TN

24
• High Specificity
• FP is few
• If test (prediction) is positive, disease
• Whether properly diagnose a person as healthy if they have no diseases
• Effective when test results are positive, useful for definitive diagnosis
• High Specificity
• If that test is positive, the probability having that disease are very high.
• e.g.
• PCR test for COVID-19,
• Pregnancy test
positive negative
positive
negative
Test (Prediction)
Disease
(Ground Truth)
TP FN
FP TN

25
• Why Sensitivity and Specificity are used.
• Disease rate (prevalence) can be small. (Imbalanced data)
• PPV and NPV depend on disease rate
• If disease rate is low, PPV become small.
• FP is increased -> This is problem when screening for rare disease
• (If specificity is high, PPV become high)
positive negative
positive
negative
Test (Prediction)
Disease
(Ground Truth)
TP FN
FP TN
positive negative
positive
negative
Test (Prediction)
Disease
(Ground Truth)
TP FN
FP TN
𝑃𝑃𝑉 =
𝑇𝑃
𝑇𝑃 + 𝐹𝑃
Small Disease rate (prevalence)

26
• AUROC
1-Specificity
Sensitivity
positive negative
positive
negative
AUROC

27
Features / Difficulties of
Medical AI

28
Features / Difficulties of Medical AI
• Data handling
• Data Collection / Normalization
• Data Feature
• Annotation
• How to split data
• For application
• Domain Knowledge
• Workflow to release
• Black box / Interpretability
• Domain shift

29
Feature / Difficulty in Medical AI
• Data Collection / Normalization
• Unification of Terminology
• Notation variety, Data format depend on the facility
• item name, unit [m or cm], value type ([0, 1, ...] or [A, B, ...])
• Unification of Terminology and preprocessing for ML is important.
Item A
[Unit_A]
Item B_0
[Unit_B_0]
[0, 1, ..]
・
・
・
Item A
[Unit_A]
Item B_1
[Unit_B_1]
[A, B, …]
・
・
・
Facility A Facility B

30
• Data Feature
• Imbalanced data
• Ordered data (like severity)
Disease Normal Severity : 0 Severity : 1 Severity : 2

31
• Annotation
• The annotation will vary depending on the doctor’s ability and the facility’s policy.
Severity : 0 Severity : 1 Severity : 2
Severity : 0 Severity : 1 Severity : 2

32
• How to split data
• We need to collect data from a variety of facilities.
Train / Validation Test
Patient A Patient B Patient C Patient D Patient E
Facility A Facility B Facility C Facility D Facility E

33
• Domain Knowledge
• ML engineers are not familiar with Medical terminology and Medical knowledge
• Medical knowledge for feature engineering
Blood test
TP, ALB, GOT, GPT, HbA ...
Medical knowledge
for feature engineering
Input data Output
ML
How to input
Terminology
in Medical domain

34
• Workflow to release product medical devices
• Product with medical AI is also applied this workflow. (case in Japan)
• In the case of relearning, application is needed again.
Development
Non-clinical
test
Clinical test
Application
for approval
Commercially
available
Biological safety
Electrical Safety
Safety of Machinery
Durability Tests
Stability Tests
etc.

35
• Black box / Interpretability
• Need explanation of features for approval
• It is difficult to explain the detail because of black box
• Important points for approval
• Used Data for development
• Use case for application
Data Use case
Black box in AI
Input Output
Medical device
(w/o AI)
Known features
Important points for approval

36
• Domain shift
• device, protocol of measurement

37
Examples of Medical AI
Research and Application

38
Examples of Medical AI Research and Application
• Images
• fundus image (eye)
• endoscopic image (cancer)
• CT/ MRI (aneurysm/ cancer)
• X-ray (bone)
• Biological signal
• ECG
• Breath sound

39

40
• Improved Automated Detection of Diabetic Retinopathy on a Publicly Available Dataset Through
Integration of Deep Learning
• Input : fundus images
• Output : diabetic retinopathy
• Model : CNN inspired by Alexnet23 (for more limited training sets) and the Oxford Visual
Geometry Group26 (for more extensive training sets) network architectures
• Result
• Sensitivity : 96.8％, Specificity :87.0％
• Related Company : IDx-DR

41
Image : endoscopic image
(cancer)

42
Image : endoscopic image (cancer)
• Application of artificial intelligence using a convolutional neural network for detecting gastric
cancer in endoscopic images
• Input : endoscopic images
• Output : gastric cancer
• Model : Single Shot MultiBox Detector
• Result
• Sensitivity : 92.2%
• Related Company : AI Medical Service Inc.

43
Image : CT/ MRI (aneurysm/
cancer)

44
• Deep Learning for MR Angiography: Automated Detection of Cerebral Aneurysms
• Input : MR images
• Output : aneurysm
• Model : ResNet18 (untrained)
• Result (top five candidate points with the highest probability of representing an aneurysm)
• Sensitivity : 93%
• Related Company : LPIXEL

45
• COVID-19 Pneumonia Image Analysis Program
• Output : the confidence level of COVID-19 pneumonia
• Result
• Related Company : MIC Medical, Alibaba Damo Technology, M3
https://corporate.m3.com/press_release/2020/20200629_001616.html
https://www.pmda.go.jp/files/000235943.pdf

46
• lung nodule detection program
• Output : Pulmonary nodules
• Pulmonary nodules are whitish shadows in the lung area on X-ray or CT images, and if they are
seen, they may indicate lung cancer or other diseases.
• Result
• Sensitivity : 61.4% (doctor + program)
• Related Company : FUJIFILM
https://www.pmda.go.jp/PmdaSearch/kikiDetail/ResultDataSetPDF/671001_30200BZX00150000_A_01_03
https://www.fujifilm.com/jp/ja/news/list/4963

47
• Cancer detection in screening mammography
• Input : mammography images
• Output : cancer (positive / normal)
• Result
• Related Company : CureMetrix
Image from
https://pubs.rsna.org/doi/pdf/10.1148/radiol.2019182908 https://www.accessdata.fda.gov/cdrh_docs/pdf18/K183285.pdf

48
• Denoised PET Images
• Input : PET Images
• Output : Denoised Images
• Related Company : SUBTLE MEDICAL
https://subtlemedical.com/usa/subtlepet/

49

50
• Deep neural network improves fracture detection by clinicians
• Input : X-ray images
• Output : Fracture
• Model : 2D CNN
• Result
• Sensitivity : 91.5%, Specificity :93.9%
• Related Company : Imagen Technologies

51

52
ECG
• 12-lead ECG
• I, II, III, aVR, aVL, aVF
• V1, V2, V3, V4, V5, V6
http://www.chugaiigaku.jp/upfile/browse/browse2549.pdf
https://ja.wikipedia.org/wiki/%E5%BF%83%E9%9B%BB%E5%9B%B3
12-lead ECG (12 wave data)
ECG measurement

53
ECG
• 12-lead ECG
• I, II, III, aVR, aVL, aVF
• V1, V2, V3, V4, V5, V6
http://www.chugaiigaku.jp/upfile/browse/browse2549.pdf
https://ja.wikipedia.org/wiki/%E5%BF%83%E9%9B%BB%E5%9B%B3
12-lead ECG (12 wave data)

54
• Deep Neural Networks Can Predict New-Onset Atrial Fibrillation From the 12-Lead ECG and
Help Identify Those at Risk of Atrial Fibrillation–Related Stroke
• Input : 12-lead digital ECG wave data + Age, Sex
• Output : Risk of Atrial Fibrillation or not (1 year later)
• Feature Extraction : 3 branch from 12-lead ECG
• I, II, V1, and V5, acquired from time (t) = 0 (start of data acquisition) to t=5 [s]
• V1, V2, V3, II, and V5 from t=5 to t=7.5 [s]
• V4, V5, V6, II, and V1 from t=7.5 to t=10 [s]
• Model : 1D CNN
• Result
• AUC : 0.85
• Sensitivity : 69%, Specificity :81%
• Related Company : Tempus
3 branches from 12-lead ECG

55
• Using Deep-Learning Algorithms to Simultaneously Identify Right and Left Ventricular
Dysfunction From the Electrocardiogram
• Input :
• 12-lead ECG wave data
• Table data
• Patient age
• Corrected QT interval, PR interval, atrial rate, and ventricular rate
• Output :
• LVEF (classification, regression)
• Left ventricular ejection fraction (LVEF) is the central measure of
left ventricular systolic function and using the diagnosis of heart failure.
• RVSD or RVD (classification)
• Feature Extraction : 12-lead ECG as image data
• Model : 2D EfficientNet
• Result : Classification
• AUROC, Sensitivity, Specificity
• LVEF<=40% : 0.94, 0.87, 0.85
• 40%<LVEF<=50% : 0.73, 0.78, 0.57
• LVEF>=50% : 0.87, 0.84, 0.81
• RVSD+RVD : 0.84, 0.77, 0.75
LVEF
RVSD, RVD

56
• Deep Learning–Based Algorithm for Detecting Aortic Stenosis Using Electrocardiography
• Input : 12-lead ECG wave data, (or single-lead ECG) + Demographic Information
• Output : AS or not
• Feature Extraction :
• ECG raw data
• ECG feature
• Heart rate, AFIB/AFL, QT interval,
• QRS duration, QTc, R axis, T axis)
• Demographic Information
• Age, Sex, Weight, Height, BMI
• Model : 2D CNN + MLP
• Order of ECG
• V1, V2, V3, V4, V6, aVL, I, aVR, II, aVF, III
• Result

57
Biological signal : Breath
sound

58
Biological signal : Breath sound
• COVID-19 Artificial Intelligence Diagnosis Using Only Cough Recordings
• Input : Cough (Audio Data)
• Output : COVID-19 or not
• Feature Extraction :
• MFCC (feature which is similar to the human hearing)
• Biomarker model
• Model : 2D CNN (ResNet50)
• Result

59
Summary
• I have summarized following parts.
• Features / Difficulties of Medical AI
• Many doctors and medical staff need products to help medical task by using AI.

60
References
• 検査データの読み方と考え方
• https://www.jslm.org/books/guideline/2018/04.pdf
• 感度・特異度・ROC曲線
• https://www.jsph.jp/covid/files/5BAA6E3.pdf
• https://jeaweb.jp/files/about_epi_research/contest2016_1.pdf
• Others
• 計測・制御セレクションシリーズ 1 次世代医療AI - 生体信号を介した人とAIの融合 –
• 医療AIの知識と技術がわかる本事例・法律から画像処理・データセットまで
• 医療AIとディープラーニングシリーズ 2020-2021年版はじめての医用画像ディープラー
ニング－基礎・応用・事例－
• 医療AIとディープラーニングシリーズ 2021-2022年版標準医用画像のためのディープ
ラーニング－実践編－
• Useful websites for obtaining information on medical AI
• The Medical AI Times
• MedTech Online

Mentoring Roundtable Discussion “Application of Medical AI”

Recommended

Recommended

More Related Content

Similar to Mentoring Roundtable Discussion “Application of Medical AI”

Similar to Mentoring Roundtable Discussion “Application of Medical AI” (20)

Recently uploaded

Recently uploaded (20)

Mentoring Roundtable Discussion “Application of Medical AI”