SlideShare a Scribd company logo
Deep learning application to
medical imaging:
Perspectives as a physician
Hongyoon Choi
Cheonan Public Health Center
Department of Nuclear Medicine, Seoul National University Hospital
CONTENTS
INTRODUCTION
Era of Medical Data Scientists
DEEP LEARNING
BASED BIOMARKER
What clinicians want
PRACTICAL ISSUES &
PERSPECTIVES
What to solve and future directions
INTRODUCTION
15% of GDP
Who gets
this
money?
* For USA
 Single target
 Pharmaceutical
companies
 Systemic, multimodal,
multidimensional
 Tech companies
Current
Future
Integrating multi-omics
Genome
Phenome
Proteome
Imaging
Metabolome
Biosensor
CNN for imaging
RNN for health care records
Multidimensional data clustering
INTRODUCTION
+
Deep learning
Tsunami in medical fields
INTRODUCTION
2016
1 billion USD
2024
AI Industry in Healthcare
(from Global Market Insights)
First Target
INTRODUCTION
“Deep learning killed the radio(logy) star”
INTRODUCTION
ROC curve
- better than dermatologists
Esteva, Andre, et al. Nature 2017
INTRODUCTION
Diabetic Retinopathy
 Better or equivalent
to ophthalmologists
Normal DM
Gulshan, Varun, et al. JAMA 2016
ChestXnet
 Equivalent/Superior to radiologists (?)
Pranav Rajpurkar, … Andrew Ng, 2017. Arxiv
INTRODUCTION
• Really replace?
– Clinical unmet needs: What we want to DL
– Unique characteristics in medical imaging data
?
Deep learning-based biomarker
Disease lesion detection
Lesion segmentation
Lesion classification
Above or equivalent to human experts level
What we eventually want
Whether subjects die?
If then, when?
Appropriate treatment options
What we want to DL is a ‘biomarker’
Deep learning-based biomarker
Issues in DL application to biomedical fields
– Various purposes
• More than simple diagnosis: Prognosis, Disease
status monitor, Response prediction.
– Ambiguous ground-truth
• Diagnosis is not a simple classification
Deep learning-based biomarker
Clinical diagnosis:
Spectrum instead of
a clear-cut classification
Blood pressure = a type of biomarker
Cardiovascular
eventrisk
140/90
HypertensionNormal
Blood Pressure
(one-dimensional)
Quantitative biomarker
reflecting prognostic outcome
Direct
Most diseases…
Multiple domain and high-dimensional
Not rely on single measurement
Deep learning-based biomarker
Brain imaging
Symptoms
Drug responses
Lab Test
• Prognosis:
Whether patient will
have cognitive dysfunction
Deep neural network Quantitative biomarker
reflecting prognostic outcome
Low-dimensional features/ scores
Qualitative/Empirical
(high-dimensional
Multimodal)
Deep learning-based biomarker
 Direct biomarker: Output of supervised learning
 Indirectly enhance conventional biomarker
 Biomarker beyond diagnosis
Deep learning-based biomarker
• Direct biomarker
– Deep learning: y=f(x) where y: label. x: data
f cat
Direct biomarker
Deep learning-based biomarker
• Direct biomarker
– Deep learning: y=f(x) where y: label. x: data
f
P(y|x)
Cat : 0.98
Dog : 0.01
Cotton: 0.00
Real implementation
Get probability label for given data
Direct biomarker
Deep learning-based biomarker
• Direct biomarker
– Deep learning: y=f(x) where y: label. x: data
Parkinson 60%
Dementia 10%
In 10 Years
A subject’s future report
Depression 20%
P(y|x)
X: Image
Y: Disease
Direct biomarker
A single parameter
Direct biomarker
Deep learning-based biomarker
FDG and amyloid PET to predict future cognitive decline
Normal MCI Alzheimer
Slow progression
Rapid progression
FDG PET
Amyloid PET
AD Normal
But, real case..
Direct biomarker
Deep learning-based biomarker
AD & NC
MCI-converter & non-converter
FDG and amyloid PET to predict future cognitive decline
Choi H and Jin KH Arxiv 2017
3-D CNN architecture
Direct biomarker
Deep learning-based biomarker
Output score measured by
baseline PET
& 3-year cognitive score
f p(Alzheimer|X)
Direct biomarker
A single parameter
Choi H and Jin KH Arxiv 2017
Direct biomarker
Deep learning-based biomarker
https://adfdgpet.appspot.com
Online Demo
Input file
Web application
Output: likelihood for AD
& predicted cognitive score
Output:
Cognitive dysfunction-related map
p(Alzheimer|X)
Direct biomarker
Deep learning-based biomarker
SWEDD:
Clinically PD,
Image normal
Expert1 Expert2
Conventional
quantification
SWEDD
Deep learning-
abnormal
2-year follow-up
80%
Abnormal
Deep learning : Redefine SWEDD diagnosis
Choi, H., … ,Lee. D.S. NeuroImage: Clinical 2017
Dopamine transporter imaging for Parkinson’s disease diagnosis
3D CNN model
Direct biomarker
Deep learning-based biomarker
 Direct biomarker: Output of supervised learning
 Indirectly enhance conventional biomarker
 Biomarker beyond diagnosis
Enhancing biomarker
Deep learning-based biomarker
Enhancing biomarker
Number of reports w.r.t.
DL application to medical imaging
Litjens G et al. Arxiv 2017
Medical Imaging Segmentation
Conventional Biomarkers
- Tumor volume
- Area of necrosis
- Combined with functional images
Deep learning-based biomarker
Enhancing biomarker
Conventional image quantification
- Measurement of radiotracer binding
- Requiring accurate segmentation
normal PD
Noninvasive imaging of
dopaminergic degeneration
Combined with Functional
Imaging
Deep learning-based biomarker
Brain tissue segmentation
Enhancing biomarker
Choi, H., & Jin, K. H. J Neurosci Methods 2016
de Brebisson, et al. CVPR 2015.
Chen H, et al. Arxiv 2017
Deep learning-based biomarker
Enhancing biomarker
Image Super-resolution
Dahl et al. Arxiv 2017
Basic architecture
- X: low image input, Y: high-resolution output
- Loss: MSE(y-f(X))
- f: Deep convolutional layers
Deep learning-based biomarker
Image enhancement for better biomarker acquisition
Enhancing biomarker
Normal dose abdomen CT Low dose abdomen CT Low dose abdomen CT+CNN
Chen H et al. Biomed Opt Exp 2017
Standard dose
PET
Low dose
PET
Low dose
PET + CNN Xiang L, et al. Neurocomputing, 2017
Deep learning-based biomarker
• Image Generation
Enhancing biomarker
Discriminative
features
Features
Common deep learning model Generative model
z = f(x)
where x: data, z: discriminative features
f: classifier model
x = g(z)
where x: data, z: latent
g: generation function
Deep learning-based biomarker
• Image Generation - Autoencoder
Enhancing biomarker
X
f:
encoder
z
g:
decoder
X
X=g(f(X))
X
f:
encoder
μ, σ
Regularized latent features
z~N(0,1)
g:
decoderμ+σZ
X
Variational autoencoder
Deep learning-based biomarker
• Image Generation – Generative adversarial network
Enhancing biomarker
z~N(0,1)
G:
generator
Fake image
Real image
D:
Discriminator
1: real
0: fake
Deep learning-based biomarker
• Image Generation – Generative adversarial network
Enhancing biomarker
z~N(0,1)
G
Fake image
Real image
D 1: real
0: fake
Training D
z~N(0,1)
G
Fake image
D Fake,
But 1
Training G
Deep learning-based biomarker
• Image Generation – Generative adversarial network
Enhancing biomarker
Structural MR generation from PET
Florbetapir PET
Generator:
U-net
Skip connection
Generated MR
PETandgeneratedMRPETandrealMR
Discriminator Real or Fake
Generative Adversarial Networks
for MR generation
z G(z)
z & G(z)
z & x
Choi H and Lee DS, J Nucl Med 2017.
Deep learning-based biomarker
 Direct biomarker: Output of supervised learning
 Indirectly enhance conventional biomarker
 Biomarker beyond diagnosis
Enhancing biomarker
Deep learning-based biomarker
Beyond diagnosis
Most studies focuses on
‘classification’
Y = f(X) where f: classifier
Real world
Whether subjects die?
If then, when?
Appropriate treatment options
Clinically, really require
‘special regression’
Y = f(X) where f: regressor
Diagnosis Prognosis
Deep learning-based biomarker
Beyond diagnosis
Deep learning and survival data
Y~ f(X)
Y  Target
Model f trained by minimize
L2 distance(MSE) between Y and f(X)
Prognosis
Common regression Regression of Survival Data
Time and events
(Labeled data:
For a subject i, death or survived at
time Ti )
A model estimates risk function, h(X)
Where hazard function at time t,
λ(t,x) = λ0(t) exp (h(x))
Deep learning-based biomarker
Beyond diagnosis
Deep learning and survival data
Prognosis
λ(t,x) = λ0(t) exp (h(x))
h(x) = Σ βX for cox linear regression
h(X) = f(X) where, f : neural network model.
Training target : maximize hi(X) at death time Ti
and minimize hj(X) other subjects still at risk (i.e. survival at Time Ti)
Katzmann JL, et al. Arxiv 2016.
Deep learning-based biomarker
Beyond diagnosis
Low risk group
High risk group
Application to transcriptome data of lung cancer
Deep learning-based risk score
Choi H and Na KJ, Biomed Res Int. In Press
Deep learning-based biomarker
Estimating normal population distribution
- Disease : defined by distance from normal
- Abnormal data are not always available
- Particularly for rare disorders
Beyond diagnosis
Define normal is important
Deep learning-based biomarker
X
f:
encoder
μ, σ
g:
decoderμ+σZ
X
Variational autoencoder
Beyond diagnosis
+y : Conditional input (covariates)
Conditional variational autoencoder
X
Y: Cat
f:
encoder
μ, σ
μ+σZ
Y: Cat
g:
decoder
X
Deep learning-based biomarker
Beyond diagnosis
VAE cVAE (y = 5)
Z
X
Z
XY
Deep learning-based biomarker
• Individual vs population
– Aging of individual brain : Comparing with virtual population
Population distribution of brain
metabolism at each age by iterative
generating from the VAE model
Generator
Latent
features
N(0,1) Random sampling
from normal distribution
+ Age
Age
50
55
60
65
70
75
• Evaluating individual brain’s aging
compared with general population
• Define ‘pathologic aging’
Choi H,… Lee DS. Biorxiv 2017
Beyond diagnosis
Deep learning-based biomarker
 Direct biomarker
 Indirectly enhance conventional biomarker
 Biomarker beyond diagnosis
Use p(Y|X) as a single parameter
Segmentation/Image enhancement/Image generation
Targets different from simple classification/
Define normal population
Practical issues & Perspectives
Where to get
Hospital
• Best source
• Practical model validation: Real world
problems
• Ethical issues and relatively small data size
Public database
• Easy to get
• Red ocean
• Limitation in real-world validation
Practical issues & Perspectives
• Public databases (medical images)
– The Cancer Imaging Archives
• Cancer imaging
– Chest X-ray8 dataset
– DDSM (mammography)
– Kaggle diabetic retinopathy
– MICCAI brain tumor segmentation challenges
– ADNI (Alzheimer PET, MRI)
– PPMI (Parkinson), ABIDE (Autism), ADHD-200 (ADHD), OASIS
(Brain MR), HCP (Brain MR)
Practical issues & Perspectives
Data Size
– Does deep learning require big data?
– Always more than 100,000 images?
Answer : No
Answer : Yes
Practical issues & Perspectives
Data Size
– Voxel-based training:
Segmentation/Superresolution etc.
– Image generation (Pix2Pix)
:~100 3d volumes
– Image augmentation
: rotation/flipping
(but, cautious. Need to consult to clinician)
– Task and complexity dependent:
AD vs normal was trained by ~300 cases.
Practical issues & Perspectives
Image labels
Combined with natural language processing
Practical issues & Perspectives
Image labels
– Unlabeled data >> Labeled data @ Hospital
– Unbalanced label
 Importance of unsupervised & semi-
supervised learning
Practical issues & Perspectives
Certainty of DL-based decision
– Recently combined with Bayesian
approximation
– Estimate p(θ|X,Y) (θ – model, X: image, Y: label)
C Leibig, et al. biorxiv 2017
• Bayesian neural network
• Dropout as a Bayesian approximation
Practical issues & Perspectives
Current decision
Rule-base, Decision Tree
Practical issues & Perspectives
Future medical decision
ConcatenatingFeatures
• Diagnosis
• Management Plan
ex) Operation or chemotherapy
140/90
AbnormalNormal
Integrated biomarker based on DL
RiskatDeath
Take Home Messages
• Era of medical data scientists
– Enormous roles of math/stats/c.s.
• What clinician want?
– DL aims at the most use of data and extract the best
discriminative biomarker (feature)
– DL can directly extract biomarker or help extract conventional
biomarker
– Beyond classification: Prognosis and defining normal
• Future direction
– Many things to solve
(e.g. unsupervised, overcome small/unlabeled data, uncertainty)

More Related Content

What's hot

deep learning applications in medical image analysis brain tumor
deep learning applications in medical image analysis brain tumordeep learning applications in medical image analysis brain tumor
deep learning applications in medical image analysis brain tumor
Venkat Projects
 
Raai 2019 clinical unmet needs and its solutions of deep learning in medicine3
Raai 2019 clinical unmet needs and its solutions of deep learning in medicine3Raai 2019 clinical unmet needs and its solutions of deep learning in medicine3
Raai 2019 clinical unmet needs and its solutions of deep learning in medicine3
Namkug Kim
 
Data Standards in Radiomics Research
Data Standards in Radiomics ResearchData Standards in Radiomics Research
Data Standards in Radiomics Research
Andrey Fedorov
 
IFMIA 2019 Plenary Talk : Deep Learning in Medicine; Engineers' Perspectives
IFMIA 2019 Plenary Talk : Deep Learning in Medicine; Engineers' PerspectivesIFMIA 2019 Plenary Talk : Deep Learning in Medicine; Engineers' Perspectives
IFMIA 2019 Plenary Talk : Deep Learning in Medicine; Engineers' Perspectives
Namkug Kim
 
Definiens In Digital Pathology Hr
Definiens In Digital Pathology HrDefiniens In Digital Pathology Hr
Definiens In Digital Pathology Hr
Daniel Nicolson
 
Neuroendoscopy Adapter Module Development for Better Brain Tumor Image Visual...
Neuroendoscopy Adapter Module Development for Better Brain Tumor Image Visual...Neuroendoscopy Adapter Module Development for Better Brain Tumor Image Visual...
Neuroendoscopy Adapter Module Development for Better Brain Tumor Image Visual...
IJECEIAES
 
Aditya Bhattacharya Chest XRay Image Analysis Using Deep Learning
Aditya Bhattacharya Chest XRay Image Analysis Using Deep LearningAditya Bhattacharya Chest XRay Image Analysis Using Deep Learning
Aditya Bhattacharya Chest XRay Image Analysis Using Deep Learning
Aditya Bhattacharya
 
Professor Harrison Bai, Artificial Intelligence Applications in Radiology_mHe...
Professor Harrison Bai, Artificial Intelligence Applications in Radiology_mHe...Professor Harrison Bai, Artificial Intelligence Applications in Radiology_mHe...
Professor Harrison Bai, Artificial Intelligence Applications in Radiology_mHe...
Levi Shapiro
 
Glioblastomas brain tumour segmentation based on convolutional neural network...
Glioblastomas brain tumour segmentation based on convolutional neural network...Glioblastomas brain tumour segmentation based on convolutional neural network...
Glioblastomas brain tumour segmentation based on convolutional neural network...
IJECEIAES
 
Chest X-ray Pneumonia Classification with Deep Learning
Chest X-ray Pneumonia Classification with Deep LearningChest X-ray Pneumonia Classification with Deep Learning
Chest X-ray Pneumonia Classification with Deep Learning
BaoTramDuong2
 
Brain Tumor Detection using Neural Network
Brain Tumor Detection using Neural NetworkBrain Tumor Detection using Neural Network
Brain Tumor Detection using Neural Network
ijtsrd
 
TCIA
TCIATCIA
A novel framework for efficient identification of brain cancer region from br...
A novel framework for efficient identification of brain cancer region from br...A novel framework for efficient identification of brain cancer region from br...
A novel framework for efficient identification of brain cancer region from br...
IJECEIAES
 
Radiomics: Novel Paradigm of Deep Learning for Clinical Decision Support towa...
Radiomics: Novel Paradigm of Deep Learning for Clinical Decision Support towa...Radiomics: Novel Paradigm of Deep Learning for Clinical Decision Support towa...
Radiomics: Novel Paradigm of Deep Learning for Clinical Decision Support towa...
Wookjin Choi
 
Project report 3D visualization of medical imaging data
Project report 3D visualization of medical imaging dataProject report 3D visualization of medical imaging data
Project report 3D visualization of medical imaging data
Shashank
 
Machine Learning for Medical Image Analysis: What, where and how?
Machine Learning for Medical Image Analysis:What, where and how?Machine Learning for Medical Image Analysis:What, where and how?
Machine Learning for Medical Image Analysis: What, where and how?
Debdoot Sheet
 
IRJET- Breast Cancer Detection from Histopathology Images: A Review
IRJET-  	  Breast Cancer Detection from Histopathology Images: A ReviewIRJET-  	  Breast Cancer Detection from Histopathology Images: A Review
IRJET- Breast Cancer Detection from Histopathology Images: A Review
IRJET Journal
 
Bt36430432
Bt36430432Bt36430432
Bt36430432
IJERA Editor
 
L045047880
L045047880L045047880
L045047880
IJERA Editor
 

What's hot (20)

deep learning applications in medical image analysis brain tumor
deep learning applications in medical image analysis brain tumordeep learning applications in medical image analysis brain tumor
deep learning applications in medical image analysis brain tumor
 
323462348
323462348323462348
323462348
 
Raai 2019 clinical unmet needs and its solutions of deep learning in medicine3
Raai 2019 clinical unmet needs and its solutions of deep learning in medicine3Raai 2019 clinical unmet needs and its solutions of deep learning in medicine3
Raai 2019 clinical unmet needs and its solutions of deep learning in medicine3
 
Data Standards in Radiomics Research
Data Standards in Radiomics ResearchData Standards in Radiomics Research
Data Standards in Radiomics Research
 
IFMIA 2019 Plenary Talk : Deep Learning in Medicine; Engineers' Perspectives
IFMIA 2019 Plenary Talk : Deep Learning in Medicine; Engineers' PerspectivesIFMIA 2019 Plenary Talk : Deep Learning in Medicine; Engineers' Perspectives
IFMIA 2019 Plenary Talk : Deep Learning in Medicine; Engineers' Perspectives
 
Definiens In Digital Pathology Hr
Definiens In Digital Pathology HrDefiniens In Digital Pathology Hr
Definiens In Digital Pathology Hr
 
Neuroendoscopy Adapter Module Development for Better Brain Tumor Image Visual...
Neuroendoscopy Adapter Module Development for Better Brain Tumor Image Visual...Neuroendoscopy Adapter Module Development for Better Brain Tumor Image Visual...
Neuroendoscopy Adapter Module Development for Better Brain Tumor Image Visual...
 
Aditya Bhattacharya Chest XRay Image Analysis Using Deep Learning
Aditya Bhattacharya Chest XRay Image Analysis Using Deep LearningAditya Bhattacharya Chest XRay Image Analysis Using Deep Learning
Aditya Bhattacharya Chest XRay Image Analysis Using Deep Learning
 
Professor Harrison Bai, Artificial Intelligence Applications in Radiology_mHe...
Professor Harrison Bai, Artificial Intelligence Applications in Radiology_mHe...Professor Harrison Bai, Artificial Intelligence Applications in Radiology_mHe...
Professor Harrison Bai, Artificial Intelligence Applications in Radiology_mHe...
 
Glioblastomas brain tumour segmentation based on convolutional neural network...
Glioblastomas brain tumour segmentation based on convolutional neural network...Glioblastomas brain tumour segmentation based on convolutional neural network...
Glioblastomas brain tumour segmentation based on convolutional neural network...
 
Chest X-ray Pneumonia Classification with Deep Learning
Chest X-ray Pneumonia Classification with Deep LearningChest X-ray Pneumonia Classification with Deep Learning
Chest X-ray Pneumonia Classification with Deep Learning
 
Brain Tumor Detection using Neural Network
Brain Tumor Detection using Neural NetworkBrain Tumor Detection using Neural Network
Brain Tumor Detection using Neural Network
 
TCIA
TCIATCIA
TCIA
 
A novel framework for efficient identification of brain cancer region from br...
A novel framework for efficient identification of brain cancer region from br...A novel framework for efficient identification of brain cancer region from br...
A novel framework for efficient identification of brain cancer region from br...
 
Radiomics: Novel Paradigm of Deep Learning for Clinical Decision Support towa...
Radiomics: Novel Paradigm of Deep Learning for Clinical Decision Support towa...Radiomics: Novel Paradigm of Deep Learning for Clinical Decision Support towa...
Radiomics: Novel Paradigm of Deep Learning for Clinical Decision Support towa...
 
Project report 3D visualization of medical imaging data
Project report 3D visualization of medical imaging dataProject report 3D visualization of medical imaging data
Project report 3D visualization of medical imaging data
 
Machine Learning for Medical Image Analysis: What, where and how?
Machine Learning for Medical Image Analysis:What, where and how?Machine Learning for Medical Image Analysis:What, where and how?
Machine Learning for Medical Image Analysis: What, where and how?
 
IRJET- Breast Cancer Detection from Histopathology Images: A Review
IRJET-  	  Breast Cancer Detection from Histopathology Images: A ReviewIRJET-  	  Breast Cancer Detection from Histopathology Images: A Review
IRJET- Breast Cancer Detection from Histopathology Images: A Review
 
Bt36430432
Bt36430432Bt36430432
Bt36430432
 
L045047880
L045047880L045047880
L045047880
 

Similar to Deep learning application to medical imaging: Perspectives as a physician

Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Anjani Dhrangadhariya
 
Challenges and opportunities for machine learning in biomedical research
Challenges and opportunities for machine learning in biomedical researchChallenges and opportunities for machine learning in biomedical research
Challenges and opportunities for machine learning in biomedical research
FranciscoJAzuajeG
 
tranSMART Community Meeting 5-7 Nov 13 - Session 2: Creating a Comprehensive ...
tranSMART Community Meeting 5-7 Nov 13 - Session 2: Creating a Comprehensive ...tranSMART Community Meeting 5-7 Nov 13 - Session 2: Creating a Comprehensive ...
tranSMART Community Meeting 5-7 Nov 13 - Session 2: Creating a Comprehensive ...
David Peyruc
 
Covid 19 detection using Transfer learning.pptx
Covid 19 detection using Transfer learning.pptxCovid 19 detection using Transfer learning.pptx
Covid 19 detection using Transfer learning.pptx
priyaghate2
 
The Personalized Health Risk Profile: A New Tool for Safety and Occupational ...
The Personalized Health Risk Profile: A New Tool for Safety and Occupational ...The Personalized Health Risk Profile: A New Tool for Safety and Occupational ...
The Personalized Health Risk Profile: A New Tool for Safety and Occupational ...
Richard Hartman, Ph.D.
 
Machine learning in Healthcare - WeCloudData
Machine learning in Healthcare - WeCloudDataMachine learning in Healthcare - WeCloudData
Machine learning in Healthcare - WeCloudData
WeCloudData
 
14 00-20171207 rance-piv_c
14 00-20171207 rance-piv_c14 00-20171207 rance-piv_c
14 00-20171207 rance-piv_c
Bertrand Tavitian
 
ESWC2019 - Injecting domain knowledge in electronic medical records to improv...
ESWC2019 - Injecting domain knowledge in electronic medical records to improv...ESWC2019 - Injecting domain knowledge in electronic medical records to improv...
ESWC2019 - Injecting domain knowledge in electronic medical records to improv...
Raphaël Gazzotti
 
Basen Network
Basen NetworkBasen Network
Basen Network
guestf7d226
 
Recent Advances in Deep Learning Techniques for Electronic Health Record
Recent Advances in Deep Learning Techniques for Electronic Health RecordRecent Advances in Deep Learning Techniques for Electronic Health Record
Recent Advances in Deep Learning Techniques for Electronic Health Record
kingstdio
 
Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Institute of Information Systems (HES-SO)
 
Basics of Data Analysis in Bioinformatics
Basics of Data Analysis in BioinformaticsBasics of Data Analysis in Bioinformatics
Basics of Data Analysis in Bioinformatics
Elena Sügis
 
NLP support for clinical tasks and decisions
NLP support for clinical tasks and decisionsNLP support for clinical tasks and decisions
NLP support for clinical tasks and decisions
CORIA-TALN 2018
 
Forum on Personalized Medicine: Challenges for the next decade
Forum on Personalized Medicine: Challenges for the next decadeForum on Personalized Medicine: Challenges for the next decade
Forum on Personalized Medicine: Challenges for the next decade
Joaquin Dopazo
 
Network medicine
Network medicineNetwork medicine
Network medicine
stepano7
 
Text mining and deep learning for biomedicine
Text mining and deep learning for biomedicineText mining and deep learning for biomedicine
Text mining and deep learning for biomedicine
Zhiyong Lu, PhD FACMI
 
Big Data in Pharma - Overview and Use Cases
Big Data in Pharma - Overview and Use CasesBig Data in Pharma - Overview and Use Cases
Big Data in Pharma - Overview and Use Cases
Josef Scheiber
 
ct_meeting_final_jcy (1).pdf
ct_meeting_final_jcy (1).pdfct_meeting_final_jcy (1).pdf
ct_meeting_final_jcy (1).pdf
ssuser2c7393
 
DISTANT-CTO: A Zero Cost, Distantly Supervised Approach to Improve Low-Resour...
DISTANT-CTO: A Zero Cost, Distantly Supervised Approach to Improve Low-Resour...DISTANT-CTO: A Zero Cost, Distantly Supervised Approach to Improve Low-Resour...
DISTANT-CTO: A Zero Cost, Distantly Supervised Approach to Improve Low-Resour...
Anjani Dhrangadhariya
 
Crowds Cure Canver: Annotating Data from The Cancer Imaging Archive
Crowds Cure Canver: Annotating Data from The Cancer Imaging ArchiveCrowds Cure Canver: Annotating Data from The Cancer Imaging Archive
Crowds Cure Canver: Annotating Data from The Cancer Imaging Archive
CancerImagingInforma
 

Similar to Deep learning application to medical imaging: Perspectives as a physician (20)

Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...
 
Challenges and opportunities for machine learning in biomedical research
Challenges and opportunities for machine learning in biomedical researchChallenges and opportunities for machine learning in biomedical research
Challenges and opportunities for machine learning in biomedical research
 
tranSMART Community Meeting 5-7 Nov 13 - Session 2: Creating a Comprehensive ...
tranSMART Community Meeting 5-7 Nov 13 - Session 2: Creating a Comprehensive ...tranSMART Community Meeting 5-7 Nov 13 - Session 2: Creating a Comprehensive ...
tranSMART Community Meeting 5-7 Nov 13 - Session 2: Creating a Comprehensive ...
 
Covid 19 detection using Transfer learning.pptx
Covid 19 detection using Transfer learning.pptxCovid 19 detection using Transfer learning.pptx
Covid 19 detection using Transfer learning.pptx
 
The Personalized Health Risk Profile: A New Tool for Safety and Occupational ...
The Personalized Health Risk Profile: A New Tool for Safety and Occupational ...The Personalized Health Risk Profile: A New Tool for Safety and Occupational ...
The Personalized Health Risk Profile: A New Tool for Safety and Occupational ...
 
Machine learning in Healthcare - WeCloudData
Machine learning in Healthcare - WeCloudDataMachine learning in Healthcare - WeCloudData
Machine learning in Healthcare - WeCloudData
 
14 00-20171207 rance-piv_c
14 00-20171207 rance-piv_c14 00-20171207 rance-piv_c
14 00-20171207 rance-piv_c
 
ESWC2019 - Injecting domain knowledge in electronic medical records to improv...
ESWC2019 - Injecting domain knowledge in electronic medical records to improv...ESWC2019 - Injecting domain knowledge in electronic medical records to improv...
ESWC2019 - Injecting domain knowledge in electronic medical records to improv...
 
Basen Network
Basen NetworkBasen Network
Basen Network
 
Recent Advances in Deep Learning Techniques for Electronic Health Record
Recent Advances in Deep Learning Techniques for Electronic Health RecordRecent Advances in Deep Learning Techniques for Electronic Health Record
Recent Advances in Deep Learning Techniques for Electronic Health Record
 
Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...Exploiting biomedical literature to mine out a large multimodal dataset of ra...
Exploiting biomedical literature to mine out a large multimodal dataset of ra...
 
Basics of Data Analysis in Bioinformatics
Basics of Data Analysis in BioinformaticsBasics of Data Analysis in Bioinformatics
Basics of Data Analysis in Bioinformatics
 
NLP support for clinical tasks and decisions
NLP support for clinical tasks and decisionsNLP support for clinical tasks and decisions
NLP support for clinical tasks and decisions
 
Forum on Personalized Medicine: Challenges for the next decade
Forum on Personalized Medicine: Challenges for the next decadeForum on Personalized Medicine: Challenges for the next decade
Forum on Personalized Medicine: Challenges for the next decade
 
Network medicine
Network medicineNetwork medicine
Network medicine
 
Text mining and deep learning for biomedicine
Text mining and deep learning for biomedicineText mining and deep learning for biomedicine
Text mining and deep learning for biomedicine
 
Big Data in Pharma - Overview and Use Cases
Big Data in Pharma - Overview and Use CasesBig Data in Pharma - Overview and Use Cases
Big Data in Pharma - Overview and Use Cases
 
ct_meeting_final_jcy (1).pdf
ct_meeting_final_jcy (1).pdfct_meeting_final_jcy (1).pdf
ct_meeting_final_jcy (1).pdf
 
DISTANT-CTO: A Zero Cost, Distantly Supervised Approach to Improve Low-Resour...
DISTANT-CTO: A Zero Cost, Distantly Supervised Approach to Improve Low-Resour...DISTANT-CTO: A Zero Cost, Distantly Supervised Approach to Improve Low-Resour...
DISTANT-CTO: A Zero Cost, Distantly Supervised Approach to Improve Low-Resour...
 
Crowds Cure Canver: Annotating Data from The Cancer Imaging Archive
Crowds Cure Canver: Annotating Data from The Cancer Imaging ArchiveCrowds Cure Canver: Annotating Data from The Cancer Imaging Archive
Crowds Cure Canver: Annotating Data from The Cancer Imaging Archive
 

Recently uploaded

Prix Galien International 2024 Forum Program
Prix Galien International 2024 Forum ProgramPrix Galien International 2024 Forum Program
Prix Galien International 2024 Forum Program
Levi Shapiro
 
Non-respiratory Functions of the Lungs.pdf
Non-respiratory Functions of the Lungs.pdfNon-respiratory Functions of the Lungs.pdf
Non-respiratory Functions of the Lungs.pdf
MedicoseAcademics
 
Hemodialysis: Chapter 3, Dialysis Water Unit - Dr.Gawad
Hemodialysis: Chapter 3, Dialysis Water Unit - Dr.GawadHemodialysis: Chapter 3, Dialysis Water Unit - Dr.Gawad
Hemodialysis: Chapter 3, Dialysis Water Unit - Dr.Gawad
NephroTube - Dr.Gawad
 
BRACHYTHERAPY OVERVIEW AND APPLICATORS
BRACHYTHERAPY OVERVIEW  AND  APPLICATORSBRACHYTHERAPY OVERVIEW  AND  APPLICATORS
BRACHYTHERAPY OVERVIEW AND APPLICATORS
Krishan Murari
 
The Normal Electrocardiogram - Part I of II
The Normal Electrocardiogram - Part I of IIThe Normal Electrocardiogram - Part I of II
The Normal Electrocardiogram - Part I of II
MedicoseAcademics
 
Phone Us ❤85270-49040❤ #ℂall #gIRLS In Surat By Surat @ℂall @Girls Hotel With...
Phone Us ❤85270-49040❤ #ℂall #gIRLS In Surat By Surat @ℂall @Girls Hotel With...Phone Us ❤85270-49040❤ #ℂall #gIRLS In Surat By Surat @ℂall @Girls Hotel With...
Phone Us ❤85270-49040❤ #ℂall #gIRLS In Surat By Surat @ℂall @Girls Hotel With...
Savita Shen $i11
 
Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdfAlcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
Dr Jeenal Mistry
 
Pharynx and Clinical Correlations BY Dr.Rabia Inam Gandapore.pptx
Pharynx and Clinical Correlations BY Dr.Rabia Inam Gandapore.pptxPharynx and Clinical Correlations BY Dr.Rabia Inam Gandapore.pptx
Pharynx and Clinical Correlations BY Dr.Rabia Inam Gandapore.pptx
Dr. Rabia Inam Gandapore
 
HOT NEW PRODUCT! BIG SALES FAST SHIPPING NOW FROM CHINA!! EU KU DB BK substit...
HOT NEW PRODUCT! BIG SALES FAST SHIPPING NOW FROM CHINA!! EU KU DB BK substit...HOT NEW PRODUCT! BIG SALES FAST SHIPPING NOW FROM CHINA!! EU KU DB BK substit...
HOT NEW PRODUCT! BIG SALES FAST SHIPPING NOW FROM CHINA!! EU KU DB BK substit...
GL Anaacs
 
For Better Surat #ℂall #Girl Service ❤85270-49040❤ Surat #ℂall #Girls
For Better Surat #ℂall #Girl Service ❤85270-49040❤ Surat #ℂall #GirlsFor Better Surat #ℂall #Girl Service ❤85270-49040❤ Surat #ℂall #Girls
For Better Surat #ℂall #Girl Service ❤85270-49040❤ Surat #ℂall #Girls
Savita Shen $i11
 
TEST BANK for Operations Management, 14th Edition by William J. Stevenson, Ve...
TEST BANK for Operations Management, 14th Edition by William J. Stevenson, Ve...TEST BANK for Operations Management, 14th Edition by William J. Stevenson, Ve...
TEST BANK for Operations Management, 14th Edition by William J. Stevenson, Ve...
kevinkariuki227
 
Lung Cancer: Artificial Intelligence, Synergetics, Complex System Analysis, S...
Lung Cancer: Artificial Intelligence, Synergetics, Complex System Analysis, S...Lung Cancer: Artificial Intelligence, Synergetics, Complex System Analysis, S...
Lung Cancer: Artificial Intelligence, Synergetics, Complex System Analysis, S...
Oleg Kshivets
 
ANATOMY AND PHYSIOLOGY OF URINARY SYSTEM.pptx
ANATOMY AND PHYSIOLOGY OF URINARY SYSTEM.pptxANATOMY AND PHYSIOLOGY OF URINARY SYSTEM.pptx
ANATOMY AND PHYSIOLOGY OF URINARY SYSTEM.pptx
Swetaba Besh
 
Flu Vaccine Alert in Bangalore Karnataka
Flu Vaccine Alert in Bangalore KarnatakaFlu Vaccine Alert in Bangalore Karnataka
Flu Vaccine Alert in Bangalore Karnataka
addon Scans
 
Triangles of Neck and Clinical Correlation by Dr. RIG.pptx
Triangles of Neck and Clinical Correlation by Dr. RIG.pptxTriangles of Neck and Clinical Correlation by Dr. RIG.pptx
Triangles of Neck and Clinical Correlation by Dr. RIG.pptx
Dr. Rabia Inam Gandapore
 
ACUTE SCROTUM.....pdf. ACUTE SCROTAL CONDITIOND
ACUTE SCROTUM.....pdf. ACUTE SCROTAL CONDITIONDACUTE SCROTUM.....pdf. ACUTE SCROTAL CONDITIOND
ACUTE SCROTUM.....pdf. ACUTE SCROTAL CONDITIOND
DR SETH JOTHAM
 
heat stroke and heat exhaustion in children
heat stroke and heat exhaustion in childrenheat stroke and heat exhaustion in children
heat stroke and heat exhaustion in children
SumeraAhmad5
 
ARTHROLOGY PPT NCISM SYLLABUS AYURVEDA STUDENTS
ARTHROLOGY PPT NCISM SYLLABUS AYURVEDA STUDENTSARTHROLOGY PPT NCISM SYLLABUS AYURVEDA STUDENTS
ARTHROLOGY PPT NCISM SYLLABUS AYURVEDA STUDENTS
Dr. Vinay Pareek
 
Pulmonary Thromboembolism - etilogy, types, medical- Surgical and nursing man...
Pulmonary Thromboembolism - etilogy, types, medical- Surgical and nursing man...Pulmonary Thromboembolism - etilogy, types, medical- Surgical and nursing man...
Pulmonary Thromboembolism - etilogy, types, medical- Surgical and nursing man...
VarunMahajani
 
Ozempic: Preoperative Management of Patients on GLP-1 Receptor Agonists
Ozempic: Preoperative Management of Patients on GLP-1 Receptor Agonists  Ozempic: Preoperative Management of Patients on GLP-1 Receptor Agonists
Ozempic: Preoperative Management of Patients on GLP-1 Receptor Agonists
Saeid Safari
 

Recently uploaded (20)

Prix Galien International 2024 Forum Program
Prix Galien International 2024 Forum ProgramPrix Galien International 2024 Forum Program
Prix Galien International 2024 Forum Program
 
Non-respiratory Functions of the Lungs.pdf
Non-respiratory Functions of the Lungs.pdfNon-respiratory Functions of the Lungs.pdf
Non-respiratory Functions of the Lungs.pdf
 
Hemodialysis: Chapter 3, Dialysis Water Unit - Dr.Gawad
Hemodialysis: Chapter 3, Dialysis Water Unit - Dr.GawadHemodialysis: Chapter 3, Dialysis Water Unit - Dr.Gawad
Hemodialysis: Chapter 3, Dialysis Water Unit - Dr.Gawad
 
BRACHYTHERAPY OVERVIEW AND APPLICATORS
BRACHYTHERAPY OVERVIEW  AND  APPLICATORSBRACHYTHERAPY OVERVIEW  AND  APPLICATORS
BRACHYTHERAPY OVERVIEW AND APPLICATORS
 
The Normal Electrocardiogram - Part I of II
The Normal Electrocardiogram - Part I of IIThe Normal Electrocardiogram - Part I of II
The Normal Electrocardiogram - Part I of II
 
Phone Us ❤85270-49040❤ #ℂall #gIRLS In Surat By Surat @ℂall @Girls Hotel With...
Phone Us ❤85270-49040❤ #ℂall #gIRLS In Surat By Surat @ℂall @Girls Hotel With...Phone Us ❤85270-49040❤ #ℂall #gIRLS In Surat By Surat @ℂall @Girls Hotel With...
Phone Us ❤85270-49040❤ #ℂall #gIRLS In Surat By Surat @ℂall @Girls Hotel With...
 
Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdfAlcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
Alcohol_Dr. Jeenal Mistry MD Pharmacology.pdf
 
Pharynx and Clinical Correlations BY Dr.Rabia Inam Gandapore.pptx
Pharynx and Clinical Correlations BY Dr.Rabia Inam Gandapore.pptxPharynx and Clinical Correlations BY Dr.Rabia Inam Gandapore.pptx
Pharynx and Clinical Correlations BY Dr.Rabia Inam Gandapore.pptx
 
HOT NEW PRODUCT! BIG SALES FAST SHIPPING NOW FROM CHINA!! EU KU DB BK substit...
HOT NEW PRODUCT! BIG SALES FAST SHIPPING NOW FROM CHINA!! EU KU DB BK substit...HOT NEW PRODUCT! BIG SALES FAST SHIPPING NOW FROM CHINA!! EU KU DB BK substit...
HOT NEW PRODUCT! BIG SALES FAST SHIPPING NOW FROM CHINA!! EU KU DB BK substit...
 
For Better Surat #ℂall #Girl Service ❤85270-49040❤ Surat #ℂall #Girls
For Better Surat #ℂall #Girl Service ❤85270-49040❤ Surat #ℂall #GirlsFor Better Surat #ℂall #Girl Service ❤85270-49040❤ Surat #ℂall #Girls
For Better Surat #ℂall #Girl Service ❤85270-49040❤ Surat #ℂall #Girls
 
TEST BANK for Operations Management, 14th Edition by William J. Stevenson, Ve...
TEST BANK for Operations Management, 14th Edition by William J. Stevenson, Ve...TEST BANK for Operations Management, 14th Edition by William J. Stevenson, Ve...
TEST BANK for Operations Management, 14th Edition by William J. Stevenson, Ve...
 
Lung Cancer: Artificial Intelligence, Synergetics, Complex System Analysis, S...
Lung Cancer: Artificial Intelligence, Synergetics, Complex System Analysis, S...Lung Cancer: Artificial Intelligence, Synergetics, Complex System Analysis, S...
Lung Cancer: Artificial Intelligence, Synergetics, Complex System Analysis, S...
 
ANATOMY AND PHYSIOLOGY OF URINARY SYSTEM.pptx
ANATOMY AND PHYSIOLOGY OF URINARY SYSTEM.pptxANATOMY AND PHYSIOLOGY OF URINARY SYSTEM.pptx
ANATOMY AND PHYSIOLOGY OF URINARY SYSTEM.pptx
 
Flu Vaccine Alert in Bangalore Karnataka
Flu Vaccine Alert in Bangalore KarnatakaFlu Vaccine Alert in Bangalore Karnataka
Flu Vaccine Alert in Bangalore Karnataka
 
Triangles of Neck and Clinical Correlation by Dr. RIG.pptx
Triangles of Neck and Clinical Correlation by Dr. RIG.pptxTriangles of Neck and Clinical Correlation by Dr. RIG.pptx
Triangles of Neck and Clinical Correlation by Dr. RIG.pptx
 
ACUTE SCROTUM.....pdf. ACUTE SCROTAL CONDITIOND
ACUTE SCROTUM.....pdf. ACUTE SCROTAL CONDITIONDACUTE SCROTUM.....pdf. ACUTE SCROTAL CONDITIOND
ACUTE SCROTUM.....pdf. ACUTE SCROTAL CONDITIOND
 
heat stroke and heat exhaustion in children
heat stroke and heat exhaustion in childrenheat stroke and heat exhaustion in children
heat stroke and heat exhaustion in children
 
ARTHROLOGY PPT NCISM SYLLABUS AYURVEDA STUDENTS
ARTHROLOGY PPT NCISM SYLLABUS AYURVEDA STUDENTSARTHROLOGY PPT NCISM SYLLABUS AYURVEDA STUDENTS
ARTHROLOGY PPT NCISM SYLLABUS AYURVEDA STUDENTS
 
Pulmonary Thromboembolism - etilogy, types, medical- Surgical and nursing man...
Pulmonary Thromboembolism - etilogy, types, medical- Surgical and nursing man...Pulmonary Thromboembolism - etilogy, types, medical- Surgical and nursing man...
Pulmonary Thromboembolism - etilogy, types, medical- Surgical and nursing man...
 
Ozempic: Preoperative Management of Patients on GLP-1 Receptor Agonists
Ozempic: Preoperative Management of Patients on GLP-1 Receptor Agonists  Ozempic: Preoperative Management of Patients on GLP-1 Receptor Agonists
Ozempic: Preoperative Management of Patients on GLP-1 Receptor Agonists
 

Deep learning application to medical imaging: Perspectives as a physician

  • 1. Deep learning application to medical imaging: Perspectives as a physician Hongyoon Choi Cheonan Public Health Center Department of Nuclear Medicine, Seoul National University Hospital
  • 2. CONTENTS INTRODUCTION Era of Medical Data Scientists DEEP LEARNING BASED BIOMARKER What clinicians want PRACTICAL ISSUES & PERSPECTIVES What to solve and future directions
  • 3. INTRODUCTION 15% of GDP Who gets this money? * For USA  Single target  Pharmaceutical companies  Systemic, multimodal, multidimensional  Tech companies Current Future
  • 4. Integrating multi-omics Genome Phenome Proteome Imaging Metabolome Biosensor CNN for imaging RNN for health care records Multidimensional data clustering INTRODUCTION + Deep learning Tsunami in medical fields
  • 5. INTRODUCTION 2016 1 billion USD 2024 AI Industry in Healthcare (from Global Market Insights) First Target
  • 6. INTRODUCTION “Deep learning killed the radio(logy) star”
  • 7. INTRODUCTION ROC curve - better than dermatologists Esteva, Andre, et al. Nature 2017
  • 8. INTRODUCTION Diabetic Retinopathy  Better or equivalent to ophthalmologists Normal DM Gulshan, Varun, et al. JAMA 2016 ChestXnet  Equivalent/Superior to radiologists (?) Pranav Rajpurkar, … Andrew Ng, 2017. Arxiv
  • 9. INTRODUCTION • Really replace? – Clinical unmet needs: What we want to DL – Unique characteristics in medical imaging data ?
  • 10. Deep learning-based biomarker Disease lesion detection Lesion segmentation Lesion classification Above or equivalent to human experts level What we eventually want Whether subjects die? If then, when? Appropriate treatment options What we want to DL is a ‘biomarker’
  • 11. Deep learning-based biomarker Issues in DL application to biomedical fields – Various purposes • More than simple diagnosis: Prognosis, Disease status monitor, Response prediction. – Ambiguous ground-truth • Diagnosis is not a simple classification
  • 12. Deep learning-based biomarker Clinical diagnosis: Spectrum instead of a clear-cut classification Blood pressure = a type of biomarker Cardiovascular eventrisk 140/90 HypertensionNormal Blood Pressure (one-dimensional) Quantitative biomarker reflecting prognostic outcome Direct Most diseases… Multiple domain and high-dimensional Not rely on single measurement
  • 13. Deep learning-based biomarker Brain imaging Symptoms Drug responses Lab Test • Prognosis: Whether patient will have cognitive dysfunction Deep neural network Quantitative biomarker reflecting prognostic outcome Low-dimensional features/ scores Qualitative/Empirical (high-dimensional Multimodal)
  • 14. Deep learning-based biomarker  Direct biomarker: Output of supervised learning  Indirectly enhance conventional biomarker  Biomarker beyond diagnosis
  • 15. Deep learning-based biomarker • Direct biomarker – Deep learning: y=f(x) where y: label. x: data f cat Direct biomarker
  • 16. Deep learning-based biomarker • Direct biomarker – Deep learning: y=f(x) where y: label. x: data f P(y|x) Cat : 0.98 Dog : 0.01 Cotton: 0.00 Real implementation Get probability label for given data Direct biomarker
  • 17. Deep learning-based biomarker • Direct biomarker – Deep learning: y=f(x) where y: label. x: data Parkinson 60% Dementia 10% In 10 Years A subject’s future report Depression 20% P(y|x) X: Image Y: Disease Direct biomarker A single parameter Direct biomarker
  • 18. Deep learning-based biomarker FDG and amyloid PET to predict future cognitive decline Normal MCI Alzheimer Slow progression Rapid progression FDG PET Amyloid PET AD Normal But, real case.. Direct biomarker
  • 19. Deep learning-based biomarker AD & NC MCI-converter & non-converter FDG and amyloid PET to predict future cognitive decline Choi H and Jin KH Arxiv 2017 3-D CNN architecture Direct biomarker
  • 20. Deep learning-based biomarker Output score measured by baseline PET & 3-year cognitive score f p(Alzheimer|X) Direct biomarker A single parameter Choi H and Jin KH Arxiv 2017 Direct biomarker
  • 21. Deep learning-based biomarker https://adfdgpet.appspot.com Online Demo Input file Web application Output: likelihood for AD & predicted cognitive score Output: Cognitive dysfunction-related map p(Alzheimer|X) Direct biomarker
  • 22. Deep learning-based biomarker SWEDD: Clinically PD, Image normal Expert1 Expert2 Conventional quantification SWEDD Deep learning- abnormal 2-year follow-up 80% Abnormal Deep learning : Redefine SWEDD diagnosis Choi, H., … ,Lee. D.S. NeuroImage: Clinical 2017 Dopamine transporter imaging for Parkinson’s disease diagnosis 3D CNN model Direct biomarker
  • 23. Deep learning-based biomarker  Direct biomarker: Output of supervised learning  Indirectly enhance conventional biomarker  Biomarker beyond diagnosis Enhancing biomarker
  • 24. Deep learning-based biomarker Enhancing biomarker Number of reports w.r.t. DL application to medical imaging Litjens G et al. Arxiv 2017 Medical Imaging Segmentation Conventional Biomarkers - Tumor volume - Area of necrosis - Combined with functional images
  • 25. Deep learning-based biomarker Enhancing biomarker Conventional image quantification - Measurement of radiotracer binding - Requiring accurate segmentation normal PD Noninvasive imaging of dopaminergic degeneration Combined with Functional Imaging
  • 26. Deep learning-based biomarker Brain tissue segmentation Enhancing biomarker Choi, H., & Jin, K. H. J Neurosci Methods 2016 de Brebisson, et al. CVPR 2015. Chen H, et al. Arxiv 2017
  • 27. Deep learning-based biomarker Enhancing biomarker Image Super-resolution Dahl et al. Arxiv 2017 Basic architecture - X: low image input, Y: high-resolution output - Loss: MSE(y-f(X)) - f: Deep convolutional layers
  • 28. Deep learning-based biomarker Image enhancement for better biomarker acquisition Enhancing biomarker Normal dose abdomen CT Low dose abdomen CT Low dose abdomen CT+CNN Chen H et al. Biomed Opt Exp 2017 Standard dose PET Low dose PET Low dose PET + CNN Xiang L, et al. Neurocomputing, 2017
  • 29. Deep learning-based biomarker • Image Generation Enhancing biomarker Discriminative features Features Common deep learning model Generative model z = f(x) where x: data, z: discriminative features f: classifier model x = g(z) where x: data, z: latent g: generation function
  • 30. Deep learning-based biomarker • Image Generation - Autoencoder Enhancing biomarker X f: encoder z g: decoder X X=g(f(X)) X f: encoder μ, σ Regularized latent features z~N(0,1) g: decoderμ+σZ X Variational autoencoder
  • 31. Deep learning-based biomarker • Image Generation – Generative adversarial network Enhancing biomarker z~N(0,1) G: generator Fake image Real image D: Discriminator 1: real 0: fake
  • 32. Deep learning-based biomarker • Image Generation – Generative adversarial network Enhancing biomarker z~N(0,1) G Fake image Real image D 1: real 0: fake Training D z~N(0,1) G Fake image D Fake, But 1 Training G
  • 33. Deep learning-based biomarker • Image Generation – Generative adversarial network Enhancing biomarker Structural MR generation from PET Florbetapir PET Generator: U-net Skip connection Generated MR PETandgeneratedMRPETandrealMR Discriminator Real or Fake Generative Adversarial Networks for MR generation z G(z) z & G(z) z & x Choi H and Lee DS, J Nucl Med 2017.
  • 34. Deep learning-based biomarker  Direct biomarker: Output of supervised learning  Indirectly enhance conventional biomarker  Biomarker beyond diagnosis Enhancing biomarker
  • 35. Deep learning-based biomarker Beyond diagnosis Most studies focuses on ‘classification’ Y = f(X) where f: classifier Real world Whether subjects die? If then, when? Appropriate treatment options Clinically, really require ‘special regression’ Y = f(X) where f: regressor Diagnosis Prognosis
  • 36. Deep learning-based biomarker Beyond diagnosis Deep learning and survival data Y~ f(X) Y  Target Model f trained by minimize L2 distance(MSE) between Y and f(X) Prognosis Common regression Regression of Survival Data Time and events (Labeled data: For a subject i, death or survived at time Ti ) A model estimates risk function, h(X) Where hazard function at time t, λ(t,x) = λ0(t) exp (h(x))
  • 37. Deep learning-based biomarker Beyond diagnosis Deep learning and survival data Prognosis λ(t,x) = λ0(t) exp (h(x)) h(x) = Σ βX for cox linear regression h(X) = f(X) where, f : neural network model. Training target : maximize hi(X) at death time Ti and minimize hj(X) other subjects still at risk (i.e. survival at Time Ti) Katzmann JL, et al. Arxiv 2016.
  • 38. Deep learning-based biomarker Beyond diagnosis Low risk group High risk group Application to transcriptome data of lung cancer Deep learning-based risk score Choi H and Na KJ, Biomed Res Int. In Press
  • 39. Deep learning-based biomarker Estimating normal population distribution - Disease : defined by distance from normal - Abnormal data are not always available - Particularly for rare disorders Beyond diagnosis Define normal is important
  • 40. Deep learning-based biomarker X f: encoder μ, σ g: decoderμ+σZ X Variational autoencoder Beyond diagnosis +y : Conditional input (covariates) Conditional variational autoencoder X Y: Cat f: encoder μ, σ μ+σZ Y: Cat g: decoder X
  • 41. Deep learning-based biomarker Beyond diagnosis VAE cVAE (y = 5) Z X Z XY
  • 42. Deep learning-based biomarker • Individual vs population – Aging of individual brain : Comparing with virtual population Population distribution of brain metabolism at each age by iterative generating from the VAE model Generator Latent features N(0,1) Random sampling from normal distribution + Age Age 50 55 60 65 70 75 • Evaluating individual brain’s aging compared with general population • Define ‘pathologic aging’ Choi H,… Lee DS. Biorxiv 2017 Beyond diagnosis
  • 43. Deep learning-based biomarker  Direct biomarker  Indirectly enhance conventional biomarker  Biomarker beyond diagnosis Use p(Y|X) as a single parameter Segmentation/Image enhancement/Image generation Targets different from simple classification/ Define normal population
  • 44. Practical issues & Perspectives Where to get Hospital • Best source • Practical model validation: Real world problems • Ethical issues and relatively small data size Public database • Easy to get • Red ocean • Limitation in real-world validation
  • 45. Practical issues & Perspectives • Public databases (medical images) – The Cancer Imaging Archives • Cancer imaging – Chest X-ray8 dataset – DDSM (mammography) – Kaggle diabetic retinopathy – MICCAI brain tumor segmentation challenges – ADNI (Alzheimer PET, MRI) – PPMI (Parkinson), ABIDE (Autism), ADHD-200 (ADHD), OASIS (Brain MR), HCP (Brain MR)
  • 46. Practical issues & Perspectives Data Size – Does deep learning require big data? – Always more than 100,000 images? Answer : No Answer : Yes
  • 47. Practical issues & Perspectives Data Size – Voxel-based training: Segmentation/Superresolution etc. – Image generation (Pix2Pix) :~100 3d volumes – Image augmentation : rotation/flipping (but, cautious. Need to consult to clinician) – Task and complexity dependent: AD vs normal was trained by ~300 cases.
  • 48. Practical issues & Perspectives Image labels Combined with natural language processing
  • 49. Practical issues & Perspectives Image labels – Unlabeled data >> Labeled data @ Hospital – Unbalanced label  Importance of unsupervised & semi- supervised learning
  • 50. Practical issues & Perspectives Certainty of DL-based decision – Recently combined with Bayesian approximation – Estimate p(θ|X,Y) (θ – model, X: image, Y: label) C Leibig, et al. biorxiv 2017 • Bayesian neural network • Dropout as a Bayesian approximation
  • 51. Practical issues & Perspectives Current decision Rule-base, Decision Tree
  • 52. Practical issues & Perspectives Future medical decision ConcatenatingFeatures • Diagnosis • Management Plan ex) Operation or chemotherapy 140/90 AbnormalNormal Integrated biomarker based on DL RiskatDeath
  • 53. Take Home Messages • Era of medical data scientists – Enormous roles of math/stats/c.s. • What clinician want? – DL aims at the most use of data and extract the best discriminative biomarker (feature) – DL can directly extract biomarker or help extract conventional biomarker – Beyond classification: Prognosis and defining normal • Future direction – Many things to solve (e.g. unsupervised, overcome small/unlabeled data, uncertainty)