2019.8.20 서울대학교 딥러닝 동아리 Deepest에서 강연한 슬라이드입니다.

1. Deep learning in Medicine
- from the perspective of CNN
Ryoungwoo Jang, M.D.
University of Ulsan, Asan Medical Center

2. Outline 2
Personal History
Introduction
Questions
What others have done
What Google have done
Deep Learning in Medicine - Korea
Unsupervised Learning for Medicine
Concerns for Deep Learning in Medicine
What we are doing
What I am doing
What we are doing
Questions Again
Future of Deep Learning in Medicine

3. Personal History

4. Personal History 4
1. Born in July 21th, 1993
2. Yeungnam University, College of Medicine, Medical Doctor
(2012.03∼2019.02)
3. University of Ulsan, Biomedical Engineering, Masters
Student(2019.03∼)

5. Introduction

6. Introduction 6
Deep Learning in Medicine?
How?

7. Introduction 6
How Deep Learning is used in Medicine?

8. Introduction 6
How Deep Learning is used in Medicine?
Classification?

9. Introduction 6
How Deep Learning is used in Medicine?
Classification?
Segmentation?

10. Introduction 6
How Deep Learning is used in Medicine?
Classification?
Segmentation?
Detection?

11. Introduction 6
How Deep Learning is used in Medicine?
Classification?
Segmentation?
Detection?
Are these all?

12. Questions 7
Questions I got :
Credibility of Deep Learning based Algorithm.
Workflow of Cooperation between Deep Learning
Reasercher and Medical Doctor.
Using GAN to Medicine

13. What others have done

14. What Google have done 9
Journal of the American Medical Association, 2016

15. What Google have done 10
Normal Fundus Photograph(Source : Wikipedia)

16. What Google have done 11
Diabetic Retinopathy(Source : Google Search)

17. What Google have done 12
Development and Validation of a Deep Learning
Algorithm for Detection of Diabetic Retinopathy in
Retinal Fundus Photographs
Used Convolutional Neural Network(CNN)
128,175 Retinal Fundus Photographs
54 US licensed Opthamologists graded Photographs
7 US boarder-certificated Opthamologists validated graded
Photographs

18. What Google have done 13
So, what end?

19. What Google have done 14
Nature NPJ Digital Medicine, 2019.04

20. What Google have done 15
Deep learning versus human graders for classifying
diabetic retinopathy severity in a nationwide screening
program
Only 1500 Ophthalmologists, 200 Retinal Specialists v.s.
4.5 million Diabetes Patients in Thailand
Half of Ophthalmologists and Retinal Specialists are in
Bankok
Validated Google’s algorithm on 7,517 Patients, 29,943
Retinal Images.

21. What Google have done 16

22. What Google have done 17
Hmm· · · , Classification?

23. What Google have done 18
arXiv, 2017

24. What Google have done 19

25. What Google have done 20
Detecting Cancer Metastases on Gigapixel Pathology
Images
Size of Pathology Image is about 1∼10 Gigapixels.
Thus, it is unable to put Original Pathology Image to CNN.
Google used Multiscale Patch(Cropping) for CNN training.
Google Team achieved AUC of 0.96.

26. What Google have done 21
Classification again?

27. What Google have done 22
Google AI Healthcare, 2019.08.20

28. Deep Learning in Medicine - Korea 23
VUNO - Bone Age

29. Deep Learning in Medicine - Korea 24
Lunit - Lunit Insight

30. Unsupervised Learning for Medicine

31. Unsupervised Learning for Medicine 26
2016.8.5 Forbes

32. Unsupervised Learning in Medicine 27
“This(=GAN), and the variations that are now being proposed
is the most interesting idea in the last 10 years in ML, in my
opinion.”
- Yann Lecun

33. Unsupervised Learning in Medicine 28
Generative Adversaial Network

34. Unsupervised Learning in Medicne 29
Generation of Fake Images? What for?
And, Why Not?

35. Unsupervised Learning in Medicine 30
arXiv, 2018

36. Unsupervised Learning in Medicine 31

37. Unsupervised Learning in Medicine 32

38. Unsupervised Learning in Medicine 33

39. Unsupervised Learning in Medicine 34

40. Unsupervised Learning in Medicine 35

41. Concerns for Deep Learning in Medicine

42. Concerns for Deep Learning in Medicine 37
1. Deep Learning requires Time-consuming,
Hard-Labored Labeled Dataset

43. Concerns for Deep Learning in Medicine 38
1. Deep Learning requires Time-consuming,
Labor-Intensive Labeled Dataset
Domain Specialists, especially Radiologists have to label
Data by Data.
Inaccurate Dataset causes decrease in Accuracy, which
cannot be tolerated in Medicine.

44. Concerns for Deep Learning in Medicine 39
2. Validation of Deep Learning Algorithm in different
Circumstances

45. Concerns for Deep Learning in Medicine 40
2. Validation of Deep Learning Algorithm in different
Circumstances
Asan Medical Center : 3rd Hospital, Severe Patients(Lung
Cancer)
Local Hospitals : 1st, 2nd Hospital, Mild
Patients(Tuberculosis)
Discrepancy between Patient Distribution - How to
overcome?

46. Concerns for Deep Learning in Medicine 41
3. Using Accuracy is Improper

47. Concerns for Deep Learning in Medicine 42
3. Using Accuracy is Improper
Sensitivity(Recall), Specificity(Precision)
Receiver Operating Characteristic(ROC), Area Under the
Curve(AUC) are more widely used.
Confusion Table

48. Concerns for Deep Learning in Medicine 43
4. Limitation of Deep Learning based Diagnosis

49. Concerns for Deep Learning in Medicine 44
4. Limitation of Deep Learning based Diagnosis
More Accurate Diagnosis - Why not?
No improvement of Treatment

50. Concerns for Deep Learning in Medicine 45
5. Discrepancy between Hospital-visiting Patients and
Real World People

51. Concerns for Deep Learning in Medicine 46
5. Discrepancy between Hospital-visiting Patients and
Real World People
Hospital-visiting People are usually Patients. In contrast,
Most People are Normal.
Is it right to train Deep Learning Model with Patients?

52. Concerns for Deep Learning in Medicine 47
6. Legal Issues

53. Concerns for Deep Learning in Medicine 48
6. Legal Issues
Technical Development of existing approved Algorithm -
Should one go through regulatory system again?
Responsibility of Decision Making - AI or Doctor?

54. Concerns for Deep Learning in Medicine 49
7. Data Preprocessing

55. Concerns for Deep Learning in Medicine 50
7. Data Preprocessing
Format of Medical Images(DICOM) is 12-bit.
Not easy to normalize - Unable to divide with 255.0
Standardization? MinMax Normalization?

56. What we are doing

57. What I am doing 52

58. What I am doing 53
CheXNet

59. What I am doing 54
CheXNet

60. What I am doing 55
Radiological Society of North America, 2019, Chicago

61. What I am doing 56
Y Label Noise - AUC curves

62. What I am doing 57
Y Label Noise - AUC curves

63. What I am doing 58
NIH dataset, Kaggle

64. What I am doing 59
Y Label Noise - AUC curves

65. What I am doing 60
Y Label Noise - AUC curves

66. What I am doing 61
CheXpert dataset, Stanford

67. What I am doing 62
Y Label Noise - AUC curves

68. What I am doing 63
Y Label Noise - AUC curves

69. What I am doing 64
“· · · quite obvious that we should stop training
radiologists· · · ,
· · · the coyote already over the edge of the cliff who
hasn’t yet looked down· · · ”
- Geoffrey Hinton, 2016

70. What I am doing 65
Contents Based Image Retrieval using Variational AutoEncoder

71. What I am doing 66
Solitary Pulmonary Nodule(Source : Wikipedia)

72. What I am doing 67
Pneumothorax(Source : Google Search)

73. What I am doing 68
CBIR using VAE
1024 × 1024 −→ 512
Latent Vector does not contain Information of Small
Lesions.
Problem of Similarity Matching between Latent Vectors

74. What we are doing 69
Scientific Reports, 2019

75. What we are doing 70
Scientific Reports, 2019

76. What we are doing 71
Journal of Digital Imaging, 2019

77. What we are doing 72
Journal of Digital Imaging, 2019

78. What we are doing 73
Journal of Digital Imaging, 2019

79. What we are doing 74
Medical Image Analysis, 2019

80. What we are doing 75
Medical Image Analysis, 2019

81. What we are doing 76
Medical Image Analysis, 2019

82. Questions Again 77
Questions I got :
Credibility of Deep Learning based Algorithm.
Workflow of Cooperation between Deep Learning
Reasercher and Medical Doctor.
Using GAN to Medicine

83. Questions Again 78
Creadibility of Deep Learning based Algorithm

84. Questions Again 79
Workflow of Cooperation between Deep Learning Reasercher
and Medical Doctor

85. Questions Again 80
Using GAN to Medicine

86. Future of Deep Learning in Medicine

87. Future of Deep Learning in Medicine 82
In Two Words

88. Future of Deep Learning in Medicine 83
In Two Words
⇒GAN, RL

89. “Medicine is a science of uncertainty and an art of
probability.”
-William Osler