Deep Learning in Healthcare
•
2 likes•847 views
In this presentation we will make an attempt to answer the question of how far AI from revolutionising healthcare and what is the current progress in this area. I have looked into the latest groundbreaking medical innovations driven by deep learning and evaluate their potential impact on medical practice. I have discussed the main challenges that deep learning engineers face and recent advances that have been proposed in deep learning in order to address these challenges. Most of the presentation is based on the recently accepted review paper – Computational biology – deep learning by William Jones, Kaur Alasoo, Dmytro Fishman et al.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Deep Learning in Healthcare
Similar to Deep Learning in Healthcare (20)
More from Dmytro Fishman
More from Dmytro Fishman (14)
Recently uploaded
Recently uploaded (20)
Deep Learning in Healthcare
- 1. Deep Learning in Medicine and Computational Biology Dmytro Fishman (dmytro@ut.ee)
- 3. Cat
- 4. Cat
- 8. Cat Representation Cat Not a cat Machine Learning
- 10. 1.2 million images 1000 categories
- 13. Errors 2010 2011 2012 2013 2014 2015 2016 28% 26% 16% 12% 7% 3% <3% AlexNet (A. Krizhevsky et al. 2012) http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/
- 14. Errors 2010 2011 2012 2013 2014 2015 2016 28% 26% 16% 12% 7% 3% <3% Hypothetical super- dedicated fine- grained expert ensemble of human labelers AlexNet (A. Krizhevsky et al. 2012) http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/
- 15. Errors 2010 2011 2012 2013 2014 2015 2016 28% 26% 16% 12% 7% 3% <3% Hypothetical super- dedicated fine- grained expert ensemble of human labelers AlexNet (A. Krizhevsky et al. 2012) https://www.semanticscholar.org/paper/Fine-grained-Categorization-Short-Summary-of-our-E-G%C3%B6ring-Freytag/0f3b7d252c236d47cf4185fd81bbb40767baf3d8 Different breeds The same breed
- 23. Medicine is really complex
- 25. ReconMap 2.0
- 26. ReconMap 2.0
- 28. Deep Learning Super rapid growth
- 29. Deep Learning Fever Super rapid growth
- 31. Bloodletting
- 34. Nevertheless…
- 35. https://pancreaticcanceraction.org/wp-content/uploads/2012/10/Trends-in-cancer-survival-by-tumour-site-1971_2011.png Year1970 2010 Trends in cancer survival Skin Breast Prostate Leukaemia Bowel Kidney Myeloma Ovarian Pancreatic
- 36. Merck Molecular Activity Challenge 2012
- 38. Diabetic Retinopathy Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs Dermatologist-level classification of skin cancer with deep neural networks Skin Cancer
- 39. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs Dermatologist-level classification of skin cancer with deep neural networks Skin Cancer Diabetic Retinopathy
- 45. Diagnostics done manually 9 - 12 minutes per patient
- 46. 128 175 images for training
- 47. 128 175 images for training 54 US licensed ophthalmologists
- 48. 128 175 images for training 54 US licensed ophthalmologists Classify into healthy, mild and severe
- 49. Sample annotation 1st doctor 2nd doctor 3rd doctor 4th doctor Healthy Disease
- 50. 0 0 0 0 1st doctor 2nd doctor 3rd doctor 4th doctor Healthy Disease Sample annotation
- 51. Healthy Disease 0 1st doctor 2nd doctor 3rd doctor 4th doctor Sample annotation
- 52. Healthy Disease 0 1 4 3 4 1st doctor 2nd doctor 3rd doctor 4th doctor Sample annotation
- 53. Healthy Disease 0 Major vote is used 1 4 3 4 1st doctor 2nd doctor 3rd doctor 4th doctor Sample annotation
- 54. Healthy Disease 0 4 Algorithm might only marginally outperform doctors 1st doctor 2nd doctor 3rd doctor 4th doctor Sample annotation
- 56. Algorithm vs Ophthalmologists Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4%
- 57. Algorithm vs Ophthalmologists Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4% The black curve is ROC for the Deep Learning algorithm
- 58. Algorithm vs Ophthalmologists Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4% Points on ROC are performances of individual ophthalmologists The black curve is ROC for the Deep Learning algorithm
- 59. Performances are very similar Algorithm vs Ophthalmologists Points on ROC are performances of individual ophthalmologists The black curve is ROC for the Deep Learning algorithm Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4%
- 60. Deep Learning algorithm can operate in any point on the curve Algorithm vs Ophthalmologists Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4%
- 61. Deep Learning algorithm can operate in any point on the curve Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4% Algorithm vs Ophthalmologists High specificity mode (diagnosis)
- 62. Deep Learning algorithm can operate in any point on the curve Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4% Algorithm vs Ophthalmologists High specificity mode (diagnosis) High sensitivity mode (screening)
- 63. Deep Learning algorithm can operate in any point on the curve Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4% Algorithm vs Ophthalmologists High specificity mode (diagnosis) High sensitivity mode (screening) While ophthalmologists’s mode is fixed by experience
- 64. Diabetic Retinopathy Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs Dermatologist-level classification of skin cancer with deep neural networks Skin Cancer
- 65. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs Dermatologist-level classification of skin cancer with deep neural networks Diabetic Retinopathy Skin Cancer
- 67. ?!
- 72. 0 1
- 73. 0 1
- 74. 0 1 90% VS 14% Difference in survival rates is drastic
- 75. 0 1 90% VS 14%
- 77. 129 450 images for training set
- 78. 129 450 images for training set 21 board- certified dermatologists
- 80. Skin disease
- 89. 9% 1% 5% 4.5% 8.5% 2% 2% 4% 1.5% 0.5% 1.5% 4.5% 1% 3% 7% 15% 10% 1%3.3% 6.3% 0.4% 4% 5%
- 90. 9% 1% 5% 4.5% 8.5% 2% 2% 4% 1.5% 0.5% 1.5% 4.5% 1% 3% 7% 15% 10% 1%3.3% 6.3% 0.4% 4% 5% 15% Sum leaf nodes
- 91. 9% 1% 5% 4.5% 8.5% 2% 2% 4% 1.5% 0.5% 1.5% 4.5% 1% 3% 7% 15% 10% 1%3.3% 6.3% 0.4% 4% 5% 15% 13% 8% 10% 5.5% 2% 20% 25% 15%
- 95. Algorithm vs Dermatologists Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25)
- 96. Each of the cases was verified by biopsy Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25) Algorithm vs Dermatologists
- 97. Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25) Algorithm vs Dermatologists
- 98. Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25) Algorithm vs Dermatologists Performance of the algorithm was compared to dermatologists
- 99. Performance of the algorithm was compared to dermatologists Average dermatologist’s performance was marked as Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25) Algorithm vs Dermatologists
- 100. Performance of the algorithm was compared to dermatologists Average dermatologist’s performance was marked as Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25) Algorithm vs Dermatologists
- 101. Specificity,% Sensitivity, % AUC of 96% Specificity,% Sensitivity, % AUC of 94% Specificity,% Sensitivity, % AUC of 91% Carcinoma: 135 images Melanoma: 130 images Melanoma: 111 images Dermatologists (25) Dermatologists (22) Dermatologists (21) Algorithm vs Dermatologists
- 102. Specificity,% Sensitivity, % AUC of 96% Specificity,% Sensitivity, % AUC of 94% Specificity,% Sensitivity, % AUC of 91% Carcinoma: 135 images Melanoma: 130 images Melanoma: 111 images Dermatologists (25) Dermatologists (22) Dermatologists (21) Algorithm vs Dermatologists Across all biopsy verified datasets Deep Neural Network was superior
- 103. Diabetic Retinopathy Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs Dermatologist-level classification of skin cancer with deep neural networks Skin Cancer https://jamanetwork.com/journals/jama/fullarticle/2588763 https://www.nature.com/nature/journal/v542/n7639/full/ nature21056.html
- 104. Few more interesting applications
- 105. Diagnosing Parkinson from voice (Al-Fatlawi et al., 2016) Detection of hypoglycemic episodes in children (San et al., 2016) HemoglobinA1c 03.2010 Timeline 07.2010 12.2010 02.2011 04.2011 ? Painintensity Frames Pain estimation from video (Zhou et al., 2016) Predicting subsequent hospitalisation (Choi et al., 2016)
- 108. Sleep
- 109. Sleep Hunger
- 110. Sleep Pain Hunger
- 111. Seems like revolution did not happened
- 112. Why Deep Learning has not revolutionised medicine yet?
- 113. Chart of possible reasons why deep learning may fail to revolutionise medicineLikelihood Effect UnlikelyHighlylikely Not nice, but ok Terrible consequences
- 114. We may fail to compose large enough datasets
- 115. Collecting data in medicine is very expensive We may fail to compose large enough datasets
- 116. Collecting data in medicine is very expensive Medical data is often protected (for a good reason) We may fail to compose large enough datasets
- 117. We may fail to compose large enough datasets
- 118. We may fail to compose large enough datasets
- 119. We may fail to compose large enough datasets
- 120. We can build a model that can distinguish them from other objects We may fail to compose large enough datasets
- 121. We can build a model that can distinguish them from other objects We may fail to compose large enough datasets
- 122. We can build a model that can distinguish them from other objects We cannot build a robust representation for all of them We may fail to compose large enough datasets
- 123. We can build a model that can distinguish them from other objects We would need a separate ImageNet for each type We may fail to compose large enough datasets We cannot build a robust representation for all of them
- 126. Chart of possible reasons why deep learning may fail to revolutionise medicineLikelihood Effect UnlikelyHighlylikely Not nice, but ok Terrible consequences Data
- 128. There is a ABCD rule they learned in college How doctors diagnose melanomas?
- 129. There is a ABCD rule they learned in college Melanomas are Asymmetrical How doctors diagnose melanomas?
- 130. There is a ABCD rule they learned in college Melanomas are Asymmetrical How doctors diagnose melanomas?
- 131. There is a ABCD rule they learned in college Melanomas are Asymmetrical How doctors diagnose melanomas? Their Borders are uneven
- 132. There is a ABCD rule they learned in college Melanomas are Asymmetrical Their Borders are uneven How doctors diagnose melanomas?
- 133. There is a ABCD rule they learned in college Melanomas are Asymmetrical Colour can be patchy and variegated How doctors diagnose melanomas? Their Borders are uneven
- 134. There is a ABCD rule they learned in college Melanomas are Asymmetrical Colour can be patchy and variegated Their Borders are uneven How doctors diagnose melanomas?
- 135. There is a ABCD rule they learned in college Melanomas are Asymmetrical Colour can be patchy and variegated How doctors diagnose melanomas? Their Borders are uneven
- 136. There is a ABCD rule they learned in college Colour can be patchy and variegated their Diameter is usually > 6 millimetres How doctors diagnose melanomas? Melanomas are Asymmetrical Their Borders are uneven
- 137. There is a ABCD rule they learned in college Melanomas are Asymmetrical Their Borders are uneven Colour can be patchy and variegated their Diameter is usually > 6 millimetres How doctors diagnose melanomas?
- 138. How computers diagnose melanomas?
- 139. How computers diagnose melanomas?
- 140. How computers diagnose melanomas? Melanoma
- 141. How computers diagnose melanomas? Melanoma
- 143. Chart of possible reasons why deep learning may fail to revolutionise medicineLikelihood Effect UnlikelyHighlylikely Not nice, but ok Terrible consequences DataInterpretability
- 144. Your computer
- 145. Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
- 146. Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more… Genome
- 147. Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
- 148. Risks Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
- 149. Risks Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
- 150. Application Cool company Risks Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
- 151. Application Cool company Risks Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
- 152. Risks Application Cool company Risks Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
- 153. Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
- 154. Encrypting your genome Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
- 155. !#$#*(#$@&#$@^@ %#@%@^#@#$*(@#) $#@&#$@^#@$)@# $#@*#@$#@####@@!! Read 3.5 B more… Encrypting your genome Your computer
- 156. Encrypted genome Gene Technology !#$#*(#$@&#$@^@ %#@%@^#@#$*(@#) $#@&#$@^#@$)@# $#@*#@$#@####@@!! Read 3.5 B more… Your computer
- 157. #@#$!!@@# - #?@% #@$(%&&& - ?@#% $&@))#^%? - ??@% Encrypted genome Gene Technology !#$#*(#$@&#$@^@ %#@%@^#@#$*(@#) $#@&#$@^#@$)@# $#@*#@$#@####@@!! Read 3.5 B more… Your computer
- 158. Encrypted risks #@#$!!@@# - #?@% #@$(%&&& - ?@#% $&@))#^%? - ??@% Encrypted genome Gene Technology !#$#*(#$@&#$@^@ %#@%@^#@#$*(@#) $#@&#$@^#@$)@# $#@*#@$#@####@@!! Read 3.5 B more… Your computer
- 159. Encrypted risks #@#$!!@@# - #?@% #@$(%&&& - ?@#% $&@))#^%? - ??@% Encrypted genome Gene Technology !#$#*(#$@&#$@^@ %#@%@^#@#$*(@#) $#@&#$@^#@$)@# $#@*#@$#@####@@!! Read 3.5 B more… Your computer Decrypt your risks with your private key
- 160. Encrypted risks #@#$!!@@# - #?@% #@$(%&&& - ?@#% $&@))#^%? - ??@% Encrypted genome Gene Technology Your computer Decrypt your risks with your private key Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01%
- 161. and start panicking changing your lifestyle Encrypted risks #@#$!!@@# - #?@% #@$(%&&& - ?@#% $&@))#^%? - ??@% Encrypted genome Gene Technology Your computer Decrypt your risks with your private key Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01%
- 162. Chart of possible reasons why deep learning may fail to revolutionise medicineLikelihood Effect UnlikelyHighlylikely Not nice, but ok Terrible consequences DataInterpretability ! # $ # * ( # $ @ & # $ @ ^ @ % # @ % @ ^ # @ # $ * ( @ # ) $ # @ & # $ @ ^ # @ $ ) @ # $ # @ * # @ $#@####@@!! Read 3.5 B more… Privacy
- 163. Original image
- 164. Original image Correct segmentation Segmentation Road Cars Trees
- 165. Original image Correct segmentation Segmentation Road Cars Trees Original image Adversarial example
- 166. Original image Correct segmentation Segmentation Road Cars Trees Original image Adversarial example Altered image
- 167. Original image Correct segmentation Segmentation Road Cars Trees Original image Adversarial example Altered image New funny segmentation
- 173. Chart of possible reasons why deep learning may fail to revolutionise medicineLikelihood Effect UnlikelyHighlylikely Not nice, but ok Terrible consequences DataInterpretability ! # $ # * ( # $ @ & # $ @ ^ @ % # @ % @ ^ # @ # $ * ( @ # ) $ # @ & # $ @ ^ # @ $ ) @ # $ # @ * # @ $#@####@@!! Read 3.5 B more… Privacy Adversarial attacks
- 174. This is all great stuff, what is next?
- 175. kaggle.com
- 176. http://www.datasciencebowl.com/competitions/turning-machine-intelligence-against-lung-cance Team: Lauri Listak Supervisor: Dmytro Fishman Turning Machine Intelligence Against Lung Cancer
- 177. Turning Machine Intelligence Against Lung Cancer http://www.datasciencebowl.com/competitions/turning-machine-intelligence-against-lung-cance Team: Lauri Listak Supervisor: Dmytro Fishman
- 178. Turning Machine Intelligence Against Lung Cancer http://www.datasciencebowl.com/competitions/turning-machine-intelligence-against-lung-cance Team: Lauri Listak Supervisor: Dmytro Fishman 20% of lung cancer deaths can be reduced with early detection
- 179. High False Positives rates lead to interventional treatments, additional costs and patient anxiety 20% of lung cancer deaths can be reduced with early detection Turning Machine Intelligence Against Lung Cancer http://www.datasciencebowl.com/competitions/turning-machine-intelligence-against-lung-cance Team: Lauri Listak Supervisor: Dmytro Fishman
- 182. 2023
- 187. References • Series of blog posts “Do machines actually beat doctors?” by Luke Oakden-Rayner (https://lukeoakdenrayner.wordpress.com/ 2016/11/27/do-computers-already-outperform-doctors/) • Opportunities and obstacles for deep learning in biology and medicine by Ching et al. (http://www.biorxiv.org/content/biorxiv/ early/2017/05/28/142760.full.pdf) • Computational biology - deep learning by William Jones, Kaur Alasoo, Dmytro Fishman et al. (accepted)