Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Scientists meet Entrepreneurs - AI & Machine Learning, Dima Fishman, University of Tartu

60 views

Published on

Scientists meet Entrepreneurs - AI & Machine Learning, Dima Fishman, University of Tartu

Published in: Health & Medicine
  • Be the first to comment

  • Be the first to like this

Scientists meet Entrepreneurs - AI & Machine Learning, Dima Fishman, University of Tartu

  1. 1. Deep Learning in Medicine and Computational Biology Dmytro Fishman (dmytro@ut.ee)
  2. 2. Cat
  3. 3. Cat
  4. 4. Cat Representation
  5. 5. Cat Representation
  6. 6. Cat Representation Cat
  7. 7. Cat Representation Cat Not a cat Machine Learning
  8. 8. http://www.asimovinstitute.org/wp-content/uploads/2016/09/neuralnetworks.png
  9. 9. 1.2 million images 1000 categories
  10. 10. Errors 2010 2011 28% 26% http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/
  11. 11. Errors 2010 2011 2012 28% 26% 16% http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/ AlexNet (A. Krizhevsky et al. 2012)
  12. 12. Errors 2010 2011 2012 2013 2014 2015 2016 28% 26% 16% 12% 7% 3% <3% AlexNet (A. Krizhevsky et al. 2012) http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/
  13. 13. Errors 2010 2011 2012 2013 2014 2015 2016 28% 26% 16% 12% 7% 3% <3% Hypothetical super- dedicated fine- grained expert ensemble of human labelers AlexNet (A. Krizhevsky et al. 2012) http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/
  14. 14. Errors 2010 2011 2012 2013 2014 2015 2016 28% 26% 16% 12% 7% 3% <3% Hypothetical super- dedicated fine- grained expert ensemble of human labelers AlexNet (A. Krizhevsky et al. 2012) https://www.semanticscholar.org/paper/Fine-grained-Categorization-Short-Summary-of-our-E-G%C3%B6ring-Freytag/0f3b7d252c236d47cf4185fd81bbb40767baf3d8 Different breeds The same breed
  15. 15. http://thelibertarianrepublic.com/wp-content/uploads/2016/01/self-driving-car1.jpg
  16. 16. http://static.dnaindia.com/sites/default/files/styles/half/public/2016/03/09/435253-skype-translator.jpg?itok=1kAl-u1D
  17. 17. http://static.dnaindia.com/sites/default/files/styles/half/public/2016/03/09/435253-skype-translator.jpg?itok=1kAl-u1D
  18. 18. http://static.dnaindia.com/sites/default/files/styles/half/public/2016/03/09/435253-skype-translator.jpg?itok=1kAl-u1D
  19. 19. https://github.com/deepmind/pysc2 Atari Starcraft Go Dota 2
  20. 20. How about medical field?
  21. 21. Medicine is complex
  22. 22. Medicine is really complex
  23. 23. ReconMap 2.0
  24. 24. ReconMap 2.0
  25. 25. Medicine is massive
  26. 26. Deep Learning Super rapid growth
  27. 27. Deep Learning Fever Super rapid growth
  28. 28. Medicine is weird
  29. 29. Bloodletting
  30. 30. Bloodletting Soothing
  31. 31. Bloodletting Lobotomy Soothing
  32. 32. Nevertheless…
  33. 33. https://pancreaticcanceraction.org/wp-content/uploads/2012/10/Trends-in-cancer-survival-by-tumour-site-1971_2011.png Year1970 2010 Trends in cancer survival Skin Breast Prostate Leukaemia Bowel Kidney Myeloma Ovarian Pancreatic
  34. 34. Merck Molecular Activity Challenge 2012
  35. 35. Diabetic Retinopathy Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs Dermatologist-level classification of skin cancer with deep neural networks Skin Cancer
  36. 36. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs Dermatologist-level classification of skin cancer with deep neural networks Skin Cancer Diabetic Retinopathy
  37. 37. Diabetic Retinopathy
  38. 38. Diabetic Retinopathy
  39. 39. Diabetic Retinopathy
  40. 40. Diabetic Retinopathy
  41. 41. Diagnostics done manually 9 - 12 minutes per patient
  42. 42. 128 175 images for training
  43. 43. 128 175 images for training 54 US licensed ophthalmologists
  44. 44. 128 175 images for training 54 US licensed ophthalmologists Classify into healthy, mild and severe
  45. 45. Sample annotation 1st doctor 2nd doctor 3rd doctor 4th doctor Healthy Disease
  46. 46. 0 0 0 0 1st doctor 2nd doctor 3rd doctor 4th doctor Healthy Disease Sample annotation
  47. 47. Healthy Disease 0 1st doctor 2nd doctor 3rd doctor 4th doctor Sample annotation
  48. 48. Healthy Disease 0 1 4 3 4 1st doctor 2nd doctor 3rd doctor 4th doctor Sample annotation
  49. 49. Healthy Disease 0 Major vote is used 1 4 3 4 1st doctor 2nd doctor 3rd doctor 4th doctor Sample annotation
  50. 50. Healthy Disease 0 4 Algorithm might only marginally outperform doctors 1st doctor 2nd doctor 3rd doctor 4th doctor Sample annotation
  51. 51. Algorithm vs Ophthalmologists
  52. 52. Algorithm vs Ophthalmologists Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4%
  53. 53. Algorithm vs Ophthalmologists Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4% The black curve is ROC for the Deep Learning algorithm
  54. 54. Algorithm vs Ophthalmologists Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4% Points on ROC are performances of individual ophthalmologists The black curve is ROC for the Deep Learning algorithm
  55. 55. Performances are very similar Algorithm vs Ophthalmologists Points on ROC are performances of individual ophthalmologists The black curve is ROC for the Deep Learning algorithm Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4%
  56. 56. Deep Learning algorithm can operate in any point on the curve Algorithm vs Ophthalmologists Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4%
  57. 57. Deep Learning algorithm can operate in any point on the curve Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4% Algorithm vs Ophthalmologists High specificity mode (diagnosis)
  58. 58. Deep Learning algorithm can operate in any point on the curve Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4% Algorithm vs Ophthalmologists High specificity mode (diagnosis) High sensitivity mode (screening)
  59. 59. Deep Learning algorithm can operate in any point on the curve Sensitivity,% 100 - Specificity, % 0100 0 100 AUC of 97.4% Algorithm vs Ophthalmologists High specificity mode (diagnosis) High sensitivity mode (screening) While ophthalmologists’s mode is fixed by experience
  60. 60. Diabetic Retinopathy Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs Dermatologist-level classification of skin cancer with deep neural networks Skin Cancer
  61. 61. Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs Dermatologist-level classification of skin cancer with deep neural networks Diabetic Retinopathy Skin Cancer
  62. 62. ?!
  63. 63. 0 1
  64. 64. 0 1
  65. 65. 0 1 90% VS 14% Difference in survival rates is drastic
  66. 66. 0 1 90% VS 14%
  67. 67. 129 450 images for training set
  68. 68. 129 450 images for training set 21 board- certified dermatologists
  69. 69. Skin disease
  70. 70. Non-neoplastic Benign Malignant
  71. 71. DermalEpidermal Melanocytic GenodermatosisInflammatory Epidermal Lymphoma Melanoma Dermal
  72. 72. DermalEpidermal Melanocytic Inflammatory Epidermal Lymphoma Melanoma Dermal Genodermatosis
  73. 73. Google Inception v3
  74. 74. Google Inception v3
  75. 75. 9% 1% 5% 4.5% 8.5% 2% 2% 4% 1.5% 0.5% 1.5% 4.5% 1% 3% 7% 15% 10% 1%3.3% 6.3% 0.4% 4% 5%
  76. 76. 9% 1% 5% 4.5% 8.5% 2% 2% 4% 1.5% 0.5% 1.5% 4.5% 1% 3% 7% 15% 10% 1%3.3% 6.3% 0.4% 4% 5% 15% Sum leaf nodes
  77. 77. 9% 1% 5% 4.5% 8.5% 2% 2% 4% 1.5% 0.5% 1.5% 4.5% 1% 3% 7% 15% 10% 1%3.3% 6.3% 0.4% 4% 5% 15% 13% 8% 10% 5.5% 2% 20% 25% 15%
  78. 78. 9% 1% 5% 4.5% 8.5% 2% 2% 4% 0.5% 1.5% 4.5% 1% 3% 7% 15% 10% 1%3.3% 6.3% 0.4% 4% 5% 36% 45% 19%
  79. 79. 9% 1% 5% 4.5% 8.5% 2% 2% 4% 0.5% 1.5% 4.5% 1% 3% 7% 15% 10% 1%3.3% 6.3% 0.4% 4% 5% 36% 45% 19%
  80. 80. Algorithm vs Dermatologists
  81. 81. Algorithm vs Dermatologists Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25)
  82. 82. Each of the cases was verified by biopsy Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25) Algorithm vs Dermatologists
  83. 83. Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25) Algorithm vs Dermatologists
  84. 84. Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25) Algorithm vs Dermatologists Performance of the algorithm was compared to dermatologists
  85. 85. Performance of the algorithm was compared to dermatologists Average dermatologist’s performance was marked as Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25) Algorithm vs Dermatologists
  86. 86. Performance of the algorithm was compared to dermatologists Average dermatologist’s performance was marked as Specificity,% Sensitivity, % AUC of 96% Carcinoma: 135 images Dermatologists (25) Algorithm vs Dermatologists
  87. 87. Specificity,% Sensitivity, % AUC of 96% Specificity,% Sensitivity, % AUC of 94% Specificity,% Sensitivity, % AUC of 91% Carcinoma: 135 images Melanoma: 130 images Melanoma: 111 images Dermatologists (25) Dermatologists (22) Dermatologists (21) Algorithm vs Dermatologists
  88. 88. Specificity,% Sensitivity, % AUC of 96% Specificity,% Sensitivity, % AUC of 94% Specificity,% Sensitivity, % AUC of 91% Carcinoma: 135 images Melanoma: 130 images Melanoma: 111 images Dermatologists (25) Dermatologists (22) Dermatologists (21) Algorithm vs Dermatologists Across all biopsy verified datasets Deep Neural Network was superior
  89. 89. Diabetic Retinopathy Development and Validation of a Deep Learning Algorithm for Detection of Diabetic Retinopathy in Retinal Fundus Photographs Dermatologist-level classification of skin cancer with deep neural networks Skin Cancer https://jamanetwork.com/journals/jama/fullarticle/2588763 https://www.nature.com/nature/journal/v542/n7639/full/ nature21056.html
  90. 90. Few more interesting applications
  91. 91. Diagnosing Parkinson from voice (Al-Fatlawi et al., 2016) Detection of hypoglycemic episodes in children (San et al., 2016) HemoglobinA1c 03.2010 Timeline 07.2010 12.2010 02.2011 04.2011 ? Painintensity Frames Pain estimation from video (Zhou et al., 2016) Predicting subsequent hospitalisation (Choi et al., 2016)
  92. 92. Sleep
  93. 93. Sleep Hunger
  94. 94. Sleep Pain Hunger
  95. 95. Seems like revolution did not happened
  96. 96. Why Deep Learning has not revolutionised medicine yet?
  97. 97. Chart of possible reasons why deep learning may fail to revolutionise medicineLikelihood Effect UnlikelyHighlylikely Not nice, but ok Terrible consequences
  98. 98. We may fail to compose large enough datasets
  99. 99. Collecting data in medicine is very expensive We may fail to compose large enough datasets
  100. 100. Collecting data in medicine is very expensive Medical data is often protected (for a good reason) We may fail to compose large enough datasets
  101. 101. We may fail to compose large enough datasets
  102. 102. We may fail to compose large enough datasets
  103. 103. We may fail to compose large enough datasets
  104. 104. We can build a model that can distinguish them from other objects We may fail to compose large enough datasets
  105. 105. We can build a model that can distinguish them from other objects We may fail to compose large enough datasets
  106. 106. We can build a model that can distinguish them from other objects We cannot build a robust representation for all of them We may fail to compose large enough datasets
  107. 107. We can build a model that can distinguish them from other objects We would need a separate ImageNet for each type We may fail to compose large enough datasets We cannot build a robust representation for all of them
  108. 108. http://langlotzlab.stanford.edu/projects/medical-image-net/ http://langlotzlab.stanford.edu/projects/medical-image Possible solution
  109. 109. http://langlotzlab.stanford.edu/projects/medical-image-net/
  110. 110. Chart of possible reasons why deep learning may fail to revolutionise medicineLikelihood Effect UnlikelyHighlylikely Not nice, but ok Terrible consequences Data
  111. 111. How doctors diagnose melanomas?
  112. 112. There is a ABCD rule they learned in college How doctors diagnose melanomas?
  113. 113. There is a ABCD rule they learned in college Melanomas are Asymmetrical How doctors diagnose melanomas?
  114. 114. There is a ABCD rule they learned in college Melanomas are Asymmetrical How doctors diagnose melanomas?
  115. 115. There is a ABCD rule they learned in college Melanomas are Asymmetrical How doctors diagnose melanomas? Their Borders are uneven
  116. 116. There is a ABCD rule they learned in college Melanomas are Asymmetrical Their Borders are uneven How doctors diagnose melanomas?
  117. 117. There is a ABCD rule they learned in college Melanomas are Asymmetrical Colour can be patchy and variegated How doctors diagnose melanomas? Their Borders are uneven
  118. 118. There is a ABCD rule they learned in college Melanomas are Asymmetrical Colour can be patchy and variegated Their Borders are uneven How doctors diagnose melanomas?
  119. 119. There is a ABCD rule they learned in college Melanomas are Asymmetrical Colour can be patchy and variegated How doctors diagnose melanomas? Their Borders are uneven
  120. 120. There is a ABCD rule they learned in college Colour can be patchy and variegated their Diameter is usually > 6 millimetres How doctors diagnose melanomas? Melanomas are Asymmetrical Their Borders are uneven
  121. 121. There is a ABCD rule they learned in college Melanomas are Asymmetrical Their Borders are uneven Colour can be patchy and variegated their Diameter is usually > 6 millimetres How doctors diagnose melanomas?
  122. 122. How computers diagnose melanomas?
  123. 123. How computers diagnose melanomas?
  124. 124. How computers diagnose melanomas? Melanoma
  125. 125. How computers diagnose melanomas? Melanoma
  126. 126. https://arxiv.org/pdf/1708.08296v1.pdf
  127. 127. Chart of possible reasons why deep learning may fail to revolutionise medicineLikelihood Effect UnlikelyHighlylikely Not nice, but ok Terrible consequences DataInterpretability
  128. 128. Your computer
  129. 129. Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
  130. 130. Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more… Genome
  131. 131. Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
  132. 132. Risks Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
  133. 133. Risks Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
  134. 134. Application Cool company Risks Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
  135. 135. Application Cool company Risks Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
  136. 136. Risks Application Cool company Risks Genome Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01% Gene Technology Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
  137. 137. Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
  138. 138. Encrypting your genome Your computer ACCCTTAAGGAGATCCTT TAACCGAACCTCACCCTT AAGGAGATCCTTTAACCG CCCTTTTATTCCTATTACGT Read 3.5 B more…
  139. 139. !#$#*(#$@&#$@^@ %#@%@^#@#$*(@#) $#@&#$@^#@$)@# $#@*#@$#@####@@!! Read 3.5 B more… Encrypting your genome Your computer
  140. 140. Encrypted genome Gene Technology !#$#*(#$@&#$@^@ %#@%@^#@#$*(@#) $#@&#$@^#@$)@# $#@*#@$#@####@@!! Read 3.5 B more… Your computer
  141. 141. #@#$!!@@# - #?@% #@$(%&&& - ?@#% $&@))#^%? - ??@% Encrypted genome Gene Technology !#$#*(#$@&#$@^@ %#@%@^#@#$*(@#) $#@&#$@^#@$)@# $#@*#@$#@####@@!! Read 3.5 B more… Your computer
  142. 142. Encrypted risks #@#$!!@@# - #?@% #@$(%&&& - ?@#% $&@))#^%? - ??@% Encrypted genome Gene Technology !#$#*(#$@&#$@^@ %#@%@^#@#$*(@#) $#@&#$@^#@$)@# $#@*#@$#@####@@!! Read 3.5 B more… Your computer
  143. 143. Encrypted risks #@#$!!@@# - #?@% #@$(%&&& - ?@#% $&@))#^%? - ??@% Encrypted genome Gene Technology !#$#*(#$@&#$@^@ %#@%@^#@#$*(@#) $#@&#$@^#@$)@# $#@*#@$#@####@@!! Read 3.5 B more… Your computer Decrypt your risks with your private key
  144. 144. Encrypted risks #@#$!!@@# - #?@% #@$(%&&& - ?@#% $&@))#^%? - ??@% Encrypted genome Gene Technology Your computer Decrypt your risks with your private key Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01%
  145. 145. and start panicking changing your lifestyle Encrypted risks #@#$!!@@# - #?@% #@$(%&&& - ?@#% $&@))#^%? - ??@% Encrypted genome Gene Technology Your computer Decrypt your risks with your private key Schizofrenia - 0.15% Diabetes - 0.05% Cancer - 0.01%
  146. 146. Chart of possible reasons why deep learning may fail to revolutionise medicineLikelihood Effect UnlikelyHighlylikely Not nice, but ok Terrible consequences DataInterpretability ! # $ # * ( # $ @ & # $ @ ^ @ % # @ % @ ^ # @ # $ * ( @ # ) $ # @ & # $ @ ^ # @ $ ) @ # $ # @ * # @ $#@####@@!! Read 3.5 B more… Privacy
  147. 147. Original image
  148. 148. Original image Correct segmentation Segmentation Road Cars Trees
  149. 149. Original image Correct segmentation Segmentation Road Cars Trees Original image Adversarial example
  150. 150. Original image Correct segmentation Segmentation Road Cars Trees Original image Adversarial example Altered image
  151. 151. Original image Correct segmentation Segmentation Road Cars Trees Original image Adversarial example Altered image New funny segmentation
  152. 152. Chart of possible reasons why deep learning may fail to revolutionise medicineLikelihood Effect UnlikelyHighlylikely Not nice, but ok Terrible consequences DataInterpretability ! # $ # * ( # $ @ & # $ @ ^ @ % # @ % @ ^ # @ # $ * ( @ # ) $ # @ & # $ @ ^ # @ $ ) @ # $ # @ * # @ $#@####@@!! Read 3.5 B more… Privacy Adversarial attacks
  153. 153. This is all great stuff, what is next?
  154. 154. kaggle.com
  155. 155. http://www.datasciencebowl.com/competitions/turning-machine-intelligence-against-lung-cance Team: Lauri Listak Supervisor: Dmytro Fishman Turning Machine Intelligence Against Lung Cancer
  156. 156. Turning Machine Intelligence Against Lung Cancer http://www.datasciencebowl.com/competitions/turning-machine-intelligence-against-lung-cance Team: Lauri Listak Supervisor: Dmytro Fishman
  157. 157. Turning Machine Intelligence Against Lung Cancer http://www.datasciencebowl.com/competitions/turning-machine-intelligence-against-lung-cance Team: Lauri Listak Supervisor: Dmytro Fishman 20% of lung cancer deaths can be reduced with early detection
  158. 158. High False Positives rates lead to interventional treatments, additional costs and patient anxiety 20% of lung cancer deaths can be reduced with early detection Turning Machine Intelligence Against Lung Cancer http://www.datasciencebowl.com/competitions/turning-machine-intelligence-against-lung-cance Team: Lauri Listak Supervisor: Dmytro Fishman
  159. 159. 2023
  160. 160. github.com/concept-to-clinic/concept-to- clinic
  161. 161. http://dreamchallenges.org/
  162. 162. References • Series of blog posts “Do machines actually beat doctors?” by Luke Oakden-Rayner (https://lukeoakdenrayner.wordpress.com/ 2016/11/27/do-computers-already-outperform-doctors/) • Opportunities and obstacles for deep learning in biology and medicine by Ching et al. (http://www.biorxiv.org/content/biorxiv/ early/2017/05/28/142760.full.pdf) • Computational biology - deep learning by William Jones, Kaur Alasoo, Dmytro Fishman et al. (accepted)
  163. 163. Contact: dmytro@ut.ee

×