![Computational map
25
Dense
Few
Millions
#ofVariables
Completeness of Data (Sampling)Sparse
More
• Compute
• Data
• Storage
• Bandwidth
Computationally
Intractable SpaceDeep Learning
Neural Nets
Statistical Analysis
Algorithms, Closed Form Solutions
Expert
Systems
Insufficient
data for analysis
[Un]supervised Learning
Models
Intuition](https://image.slidesharecdn.com/interpretabilityandinformaticsofdeeplearninginmedicalimages3-181228023044/75/Interpretability-and-informatics-of-deep-learning-in-medical-images3-21-2048.jpg)
![Convolutional Neural Networks (CNN)
A type of feed-forward neural network
Inspired by biological process
Weight sharing (convolution) + Subsampling
(pooling)
Reducing the number of parameters (Reduce over-fitting)
Translation invariance
Input
28 × 28
Feature maps
4@24 × 24
Feature maps
4@8 × 8
Feature maps
8@4 × 4
Feature maps
8@2 × 2
Feature maps
8 ⋅ 2 ⋅ 2 × 1
Output
10 × 1
Convolution
layer
Max-pooling
layer
Convolution
layer
Max-pooling
layer
Reshape Linear layer
[LeCun, 1998]](https://image.slidesharecdn.com/interpretabilityandinformaticsofdeeplearninginmedicalimages3-181228023044/75/Interpretability-and-informatics-of-deep-learning-in-medical-images3-24-2048.jpg)
![Comparison btw Brain and NN
112
1. 10 billion neurons
2. 60 trillion synapses
3. Distributed processing
4. Nonlinear processing
5. Parallel processing
6. Efficiency
(20~25W, 하루섭취량의 20~25%)
1. Faster than neuron (10-9 sec)
cf. neuron: 10-3 sec
3. Central processing
4. Arithmetic operation (linearity)
5. Relatively Sequential processing
6. Efficiency (Titan X : 250W)
cf. 1[kcal] = 1.16[Wh], 1W=1J=1Nm/s, 1cal=4.2J=1.163mWh
Brain Computer](https://image.slidesharecdn.com/interpretabilityandinformaticsofdeeplearninginmedicalimages3-181228023044/75/Interpretability-and-informatics-of-deep-learning-in-medical-images3-58-2048.jpg)
![[Parallel Worlds, Michio Kaku]
Temperature Precipitation
Wind …
Weather
E1 E2
E3 Ex
Longitude
C1
Landscape
C2
Altitude
C3
…
C4
!!
Disease
Predictability
??
Prediction (Complexity)](https://image.slidesharecdn.com/interpretabilityandinformaticsofdeeplearninginmedicalimages3-181228023044/75/Interpretability-and-informatics-of-deep-learning-in-medical-images3-61-2048.jpg)
Uploaded byNamkug Kim
Interpretability and informatics of deep learning in medical images3
1) The document discusses Namkug Kim's background and research interests which include medical imaging, artificial intelligence, and various grants and industry collaborations. 2) It provides an overview of how big data from sources like IoT devices, social media, and electronic health records can provide insights into public health trends and opportunities in healthcare. 3) The rise of artificial intelligence technologies like deep learning that now exceed human-level performance on certain tasks is also examined, as well as opportunities these techniques provide in medical applications.
More Related Content
Similar to Interpretability and informatics of deep learning in medical images3
![[Review] High-performance medicine: the convergence of human and artificial i...](https://cdn.slidesharecdn.com/ss_thumbnails/reviewhigh-performancemedicine-190728142338-thumbnail.jpg?width=640&height=640&fit=bounds)
Interpretability and informatics of deep learning in medical images3
- 1. Interpretability and informaticsof deep learning in medical images Namkug Kim, PhD Medical Imaging & Intelligent Reality Lab. Convergence Medicine/Radiology, University of Ulsan College of Medicine Asan Medical Center South Korea
- 2. Researches with LG Electronics CorelineSoft Inc. Osstem Implant CGBio VUNO Kakaobrain Conflict of Interests Stockholder Coreline Soft, Inc. AnyMedi Co-founder Somansa Inc. Cybermed Inc. Clinical Imaging Solution, Inc AnyMedi, Inc. Selected Grants as PI NRF, South Korea 7T용 4D 자기공명유속영상을이용한 심뇌혈관 질환의 in-vivo유동 정량화 SW개발, 2016 4D flow MRI을 이용한 심혈관 질환의 in-vivo유동 연구, 2015-7 자기공명분광영상 및 MRI의 통합 분석 소프트웨어 개발 KEIT, South Korea DigitalDentistry, 2018-2022 의료영상 인공지능 PACS 과제, 2016-20 3DP 척추 맞춤형 임플란트, 2016-20 3D 프린터 기반 무치악 및 두개악안면결손환자용 수복 보철물 제작, 재건 시스템 개발, 2015-9 근골격계 복구 수술 로봇 개발, 2012-7 영상중재시술 로봇시스템 개발, 2012-7 Spine및 Neurosurgery 수술보조용 항법 시스템 개발, 2001 의료용 3차원 모델 제작 S/W 기술 개발, 정통부, 2000 의료영상재구성에 의한 가상시술 소프트웨어 개발, 중소기업기술혁신개발, 중기청, 2001 KHIDI, South Korea 연구중심병원 육성과제, 2019-2028 인공지능 학습센터 과제, 2018-2023 영상 뇌졸중 예후 예측 및 치료방침 결정 시스템 개발, 2012-8 관동맥 관류 CT 의 자동 진단 프로그램을 활용한 허혈성 질환의 진단과 치료, 2013-6 RP를 이용한 척추나사못 삽입술 계획 프로그램 개발, 2000 Companies Fundings SiemensGermany, Hyundai Heavy Industry, OsstemImplant, S&G Biotech, Corelinesoft, MidasIT, AnyMedi, Hitachi Medical,Japan, Kakaobrain
- 3. Big data :IoT Thermometer IoT Thermometer Kinsa : Startup @ USA Real-time body temperature bigdata@USA Patients-derived health data Regional basis Influenza stats Kinsa : Realtime vs CDC : 3 week delay B2B model: Demands and production 독감 예방 접종 혹은 항생제, 살균제 등의 약 칫솔, 오렌지 쥬스, 수프 등 7 Influenza trends: comparison with CDC 2.5 y
- 4. Big data :Google Trends 8Nature 2008
- 5. Big data :Facebook 9 Correlation between Facebook usage vs drug addiction Accuracy : Tobacco(86%), Alcohol(81%), Drug(84%)
- 6. Healthcare Bigdata 7.5 ExaByte/day (30% of every data) 생물학적 특성, 건강 이력, 웰빙 상태, 가는 곳, 지출 내역, 수면 상태, 식사와 배 설 / 실험 기록, 의학 영상, 유전정보, 액체생체검사, 심전도 /보험금 청구서, 임 상시험, 처방전, etc 10IBM Healthcare
- 7. Precision Medicine 11 /46 • Bigdata • -Omics • Genomics, Transcriptomics, Proteonomics, Metabolomics, … • Personalized/Precision Backgrounds Big Data NGS Lab Imaging EMR
- 8. Opportunity 12 8 trillion exam: HealthcareIndustry 2 trillion : wastes in healthcare industry Better experience Imaging : Unnecessary tests Lower cost Oncology: Variability of Care Better outcomes Life sciences: Failed clinical trials Government: Fraud, Waste and Abuse Value Based Care: Cost of chronic disease 360 billion : total IT and healthcare market opportunity *IBM Watson
- 9. Beyond Human-level Performance •Now, Machines Beat Human in Tasks Once Considered Impossible 5:0 vs Fan Hui (Oct. 2015) 4:1 vs Sedol Lee (Mar. 2016) Modified by KH Jung, PhD
- 10. Beyond Human-level Performance •Now, Machines Beat Human in Tasks Once Considered Impossible TPU Server used against Lee Sedol TPU Board used against Ke Jie
- 11. Beyond Human-level Performance •Now, Machines Beat Human in Tasks Once Considered Impossible Libratus(Jan 30, 2017) DeepStack(Science, Mar 02, 2017)
- 12. Beyond Human-level Performance •Now, Machines Beat Human in Tasks Once Considered Impossible
- 13. AI vs MLvs DL 17
- 14. Machine Learning scikit-learn algorithmcheat-sheet.
- 15.
- 16. Comparison btw Brainand NN 20
- 17. Bio Plausible NeuralNetwork Mimic human visual recognition system Neocognitron, proposed by Hubel & Wiesel in 1959 Visual primary cortex by cascading from S-Cell to C-Cell Each unit connected to a small subset of other units Based on what it sees, it decides what it wants to say Units must learn to cooperate to accomplish the task 21From Gallant and van Esses, Simon Thorpe
- 18. CNN : MajorBreakthroughs in Feedforward NN K. Fukushima Yann Lecun G. Hinton, S. Ruslan Neocognitron (1979) • By Kunihiko Fukushima • First proposed CNN Convolutional Neural Networks (1989) • Yann Lecun et.al • Back propagation for CNN • Theoretically learn any function Neocognitron LeNet-5 architecture Alex krizhevsky , Hinton LeNet-5 (1998) • Convolutional networks Improved by Yann Lecun et.al • Classify handwritten digits D. Rumelhart, G. Hinton, R. Wiliams 1960 1970 1980 1990 2000 2010 2012 Perceptron XOR Problem Golden Age 1957 1969 1986 F. RosenblattM. Minsky, S. Papert • Adjustable weights • Weights are not learned • XOR problem is not linearly separble • Solution to nonlinearity separable problems • Big computation, local optima and overfiting CNN Breakthrough (2012) • By Alex Krizhevsky et al. • Winner of ILSVRC2012 by large marginDark Age (AI winter) Back propagation (1981) • Train multiple layers Multi-layer Perceptron (1986) 1950 Neocognitron (1959) • Hubel & Wiesel • by cascading from S- Cell to C-Cell
- 19. Feature Engineering vsFeature Learning Modified From Yann LeCun Knowledge-driven Feature Engineering Conventional Radiomics Data-driven Feature Learning Deep Radiomics •Feature Learning instead of Conventional Feature Engineering Removes Barriers for Multi-modal Studies and Data-driven Approaches in Medical Data Analysis
- 20. Machine Learning vsDeep Learning — Scale Matters — Millions to Billions of parameters — Data Matters — Regularize using more data — Productivity Matters — It’s simple, so we can make tools Data & Compute Accuracy Deep Learning Many previous methods Deep learning is most useful for large problems Modified by Nvidia DLI
- 21. Computational map 25 Dense Few Millions #ofVariables Completeness ofData (Sampling)Sparse More • Compute • Data • Storage • Bandwidth Computationally Intractable SpaceDeep Learning Neural Nets Statistical Analysis Algorithms, Closed Form Solutions Expert Systems Insufficient data for analysis [Un]supervised Learning Models Intuition
- 22.
- 23.
- 24. Convolutional Neural Networks(CNN) A type of feed-forward neural network Inspired by biological process Weight sharing (convolution) + Subsampling (pooling) Reducing the number of parameters (Reduce over-fitting) Translation invariance Input 28 × 28 Feature maps 4@24 × 24 Feature maps 4@8 × 8 Feature maps 8@4 × 4 Feature maps 8@2 × 2 Feature maps 8 ⋅ 2 ⋅ 2 × 1 Output 10 × 1 Convolution layer Max-pooling layer Convolution layer Max-pooling layer Reshape Linear layer [LeCun, 1998]
- 25.
- 26. Feature Extraction byCNN 30
- 27.
- 28.
- 29. AI Medical Device 33 •Verily@Google: Normal vs Abn, Anti-aging, Life Prolongation • IBM: 트루벤 인수등 40B USD • Apple: GSK, EMR +iPhone Healthcare Platform • Facebook: Incurable dx, Human cell atlas, 5000M USD • Zebra Medical Vision • AI medical imaging Dx : 1st place of investment • Medical imaging reading cloud service/1 USD
- 30. Clinical Unmet Needson Deep Learning 효율적인 데이터 비식별화, Curation, 및 인공지능 기술을 이용한 스마트 레이블링 기술 다양한 장비와 병원마다의 차이를 극복하는 도메인 적응(Domain adaptation) 기술 블랙박스 성격을 완화하기 위한 인공지능 판단 해석(Interpretability) 및 시각화(Visualization) 연 구 의료 데이터가 가지고 있는 불확실성(Uncertainty), 인공지능의 판단이 가지고 있는 불확실성을 평가하는 기술 질환별 편향(imbalance) 문제를 해결하기 위한 전처리 및 인공지능 학습 기술 한번도 보지 못한 새로운 클래스를 미지판단(Novelty)을 통해서 검출해서 추후 의사가 따로 평가 희귀 데이터나 소수의 데이터를 학습하기 위한 One/Multi-shot Learning 기술 개발 반복측정된 영상데이터를 이용하여 딥러닝이 가지고 있는 재현가능성(Reproducibility) 연구 Adversarial Attack에 강인한 인공지능 연구 여러 물리 및 의학 법칙 등을 이용하여 기계학습(Physics Induced Machine Learning)의 효율을 증가하는 연구 34
- 31.
- 32.
- 33.
- 34. 해석력 강화 모델 38 PatrickHall et al. (O’REILLY, 2017) Macro Tulio Ribeiro et al. (O’REILLY, 2016) Surrogate models Local-Interpretable-Model-agnostic Explanations (Perturbation)
- 35. 해석력 강화 모델 39 YinLou et al, ACM 2012 GAM, Generalized Additive Models Everything should be made as simple as possible, but not simpler. — Albert Einstein. • 변수간 상호작용 효과를 배제 • 설명력 제고 • 개별 변수별로 복잡한 구조의 알고리즘을 적용한 후 이를 더하기 형태로 종합
- 36. Analysis on DeepLearning Methods for Predicting Patient Survival Basic CNN Modified CNN Multi-layer CNN Residual NetworkFeed-forward NN
- 37. Interpretability : MachineOperable, Human Readable Visual attention Category – feature mapping Sparsity and diversity 41
- 38. Machine Operable, HumanReadable Visualization of salient region in bone x-ray 42
- 39. Machine Operable, HumanReadable Evidence hotpot for lesion visualization “SpineNet: Automatically Pinpointing Classification Evidence in Spinal MRIs” 43
- 40. Bias representation inCNN 44 Zhang et al., 2018b
- 41. Knowledge hierarchy hiddenin conv-layers 45 Zhang et al., 2018a
- 42. Knowledge hierarchy hiddenin conv-layers 46 Zhang et al., 2018a
- 43. Knowledge hierarchy hiddenin conv-layers 47 Zhang et al., 2018a
- 44.
- 45.
- 46.
- 47. Visualization of interpretablefilters in top conv-layer 51
- 48. Parse tree 52 Latent parsetree and configuration as quantitatively extractive rationale in detection
- 49.
- 50.
- 51. Uncertainty Uncertainty of trainingdata In clinical situation, it is common Deep Bayesian Modeling Uncertainty of classification/prediction of Machine Learning 55
- 52. Novelty (Untrained catergory) Inclinical situation Novelty is everywhere, especially supervised learning Rare diseases, but well known to medical doctors Hard to training How to determine novel (untrained) category Unsupervised learning Semi-unsupervised learning Normal vs abnormal Abnormality Detection 56
- 53. Reproducibility Test-retest is oneof most important issues of biomarker Multiple scanning within similar date Evaluate reliability of AI/Deep learning Chest PA (Nodule) Nodule Size on Chest PA with YOLO – 56% : Variability of Size Marker – Chest PA 50 pairs DILD CT 57
- 54. Case Orchestration 58 Reading orderingby AI for efficient reading Agfa, IBM, Philips, etc
- 55. Big data PACSplatform Bigdata PACS Quantifying every data Transforming PACS into big data platform Advanced processing service (APS) Lung nodules – Detection : location – Segmentation : boundary drawing » Quantifying size, long/short axis, volume, etc 59 Zebra Medical Vision + Google, EnvoyAI+TeraRecon, Arterys, Fujifilm Nuance + Partners HC, Philips, Siemens, etc
- 56. Normality/Abnormality Filtering BlockChain powered? AbnormalityFiltering DeepRadiology, Inc. by Le Cun 60
- 57. AI apps Platform AIapps platform Philips, Nuance, Siemens, etc Radiologists’ tasks : 23,000 Follow/up development of imaging modality 61
- 58. Comparison btw Brainand NN 112 1. 10 billion neurons 2. 60 trillion synapses 3. Distributed processing 4. Nonlinear processing 5. Parallel processing 6. Efficiency (20~25W, 하루섭취량의 20~25%) 1. Faster than neuron (10-9 sec) cf. neuron: 10-3 sec 3. Central processing 4. Arithmetic operation (linearity) 5. Relatively Sequential processing 6. Efficiency (Titan X : 250W) cf. 1[kcal] = 1.16[Wh], 1W=1J=1Nm/s, 1cal=4.2J=1.163mWh Brain Computer
- 59. AI AI ANTHROPOMORPHIC (X) ARTIFICIALINTELLIGENCE (X) ->AUGMENTED INTELLIGENCE !
- 60. Strength of SouthKorea 주민번호 세계적 의료수준 예) 장기이식 생존률 대형병원 대부분의 질환의 Survival 데이 터를 쉽게 모을수 있음 ICT 산업 인프라 국내 산업 인프라 의료 인공지능은 수준이 떨어지 지 않음 114
- 61. [Parallel Worlds, MichioKaku] Temperature Precipitation Wind … Weather E1 E2 E3 Ex Longitude C1 Landscape C2 Altitude C3 … C4 !! Disease Predictability ?? Prediction (Complexity)
- 62. Job Opening @MI2RL_AMC, Seoul, SouthKorea Post-doc research fellow, PhD Students, Researchers 116 AMC, UoU Seoul South Korea Contact (namkugkim@gmail.com)
- 63. Collaborators Radiology Joon Beom Seo,SangMin LeeA,B, Dong Hyun, Yang, Hyung Jin Won, Ho Sung Kim, Seung Chai Jung, Ji Eun Park, So Jung Lee,Jeong Hyun Lee, Gilsun Hong Neurology Dong-Wha Kang, Chongsik Lee, Jaehong Lee, Sangbeom Jun, Misun Kwon, Beomjun Kim Cardiology Jaekwan Song, Jongmin Song, Young-Hak Kim Emergency Medicine Dong-Woo Seo Pathology Hyunjeong Go, Gyuheon Choi, Gyungyub Gong, Dong Eun Song Surgery Beom Seok Ko, JongHun Jeong, Songchuk Kim, Tae-Yon Sung Internal Medicine Jeongsik Byeon, Kang Mo Kim Anesthesiology Sung-Hoon Kim, Eun Ho Lee