Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Medical Imaging at UPC - Elisa Sayrol - UPC Barcelona 2018

247 views

Published on

https://telecombcn-dl.github.io/2018-dlcv/

Deep learning technologies are at the core of the current revolution in artificial intelligence for multimedia data analysis. The convergence of large-scale annotated datasets and affordable GPU hardware has allowed the training of neural networks for data analysis tasks which were previously addressed with hand-crafted features. Architectures such as convolutional neural networks, recurrent neural networks and Q-nets for reinforcement learning have shaped a brand new scenario in signal processing. This course will cover the basic principles and applications of deep learning to computer vision problems, such as image classification, object detection or image captioning.

Published in: Health & Medicine
  • Be the first to comment

Medical Imaging at UPC - Elisa Sayrol - UPC Barcelona 2018

  1. 1. Associate Professor Universitat Politecnica de Catalunya Technical University of Catalonia Medical Imaging @UPC Day 2 Lecture 5 #DLUPC http://bit.ly/dlcv2018 Elisa Sayrol Clols elisa.sayrol@upc.edu At GPI with Verónica Vilaplana, Míriam Bellver, Xavier Giró
  2. 2. bit.ly/DLCV2018 #DLUPC 2 Acknowledgments Working in this area @UPC (GPI and BSC) Verónica Vilaplana veronica.vilaplana@upc.edu Xavier Giro-i-Nieto xavier.giro@upc.edu Míriam Bellver miriam.bellver@bsc.es Adrià Casamitjana ….and plenty of other students
  3. 3. bit.ly/DLCV2018 #DLUPC Outline ● Why DL for medical imaging? ● Challenges ● Some applications at UPC ○ Segmentation 1: brain tumor segmentation ○ Segmentation 2: brain tumor segmentation ○ Classification (Exam)1: Skin Classification ○ Super-Resolution 1: Brain MRI super-resolution ○ Segmentation 2: Liver Lesion Segmentation ○ Segmentation 3: Parasite Segmentation ○ Segmentation 4: Active Learning & Segmentation ○ Classification (Exam) 2: Impact of Segmentation in Exam (Skin) ● Datasets, challenges 3
  4. 4. bit.ly/DLCV2018 #DLUPC Why deep learning for medical imaging? Deep learning is providing exciting solutions for medical image analysis problems and is seen as a key method for future applications Papers that use deep learning in areas of health informatics / medical imaging Ravi et al, Deep Learning for Health Informatics, JBHI, 2017 Litjens et al, A Survey on Deep Learning in Medical Image Analysis, June 4th , 2017 4
  5. 5. bit.ly/DLCV2018 #DLUPC Image modalities 5Litjens et al, A Survey on Deep Learning in Medical Image Analysis, June 4th , 2017 Why deep learning for medical imaging? 2015 - 2017
  6. 6. bit.ly/DLCV2018 #DLUPC Why deep learning for medical imaging? Anatomical application areas 6 Litjens et al, A Survey on Deep Learning in Medical Image Analysis, June 4th , 2017 2015 - 2017
  7. 7. bit.ly/DLCV2018 #DLUPC Deep learning uses in medical imaging Classification: Image / exam classification, object or lesion classification Detection: organ, region and landmark localization, object or lesion detection Segmentation: organ and substructure segmentation, lesion segmentation Other tasks: registration, content based image retrieval, image generation and enhancement, combining image data with reports 7 Litjens et al, A Survey on Deep Learning in Medical Image Analysis, June 4th , 2017
  8. 8. bit.ly/DLCV2018 #DLUPC Challenges 1. Problem definition and expertise ● Relevance (clinician vs ML scientist) ● Complexity of human physiology ● Difficulty in modeling implicit knowledge and skills of clinicians ● Discrepancy between experts and algorithms ● DL seen as a black-box which may lead to a lack of trust 2. Data ● Lack of large annotated datasets ○ Availability of datasets ○ Stringent regulations ○ Annotations from experts (expensive, time consuming) ● Large variability in images due to sensors and other factors ● Class imbalance: normal class is usually over represented 8
  9. 9. bit.ly/DLCV2018 #DLUPC Segmentation 1: brain tumor segmentation Gliomas are the most frequent primary brain tumors in adults Challenges: lesions are defined through intensity changes relative to surrounding normal tissue, tumor structures vary in size, extension, localization (MRImages) BRATS: Brain Tumor Segmentation Challenge ( MICCAI Int. Conf. on Medical Image Computing and computer assisted intervention) 9 sagittal coronal axial
  10. 10. bit.ly/DLCV2018 #DLUPC Segmentation 1: brain tumor segmentation Multimodal MRI images (different biological information, 3D) BRATS challenge: 4 tumor regions + background 10 Whole tumor visible in FLAIR (A), the tumor core visible in T2 (B), enhancing tumor visible in T1c (blue), necrotic components of core (green). Segmentations combined to generate final labels (D): edema (yellow), non-enhancing solid core (red), necrotic/cystic core (green), enhancing core (blue) T1 T1* T2 FLAIR
  11. 11. bit.ly/DLCV2018 #DLUPC Segmentation 1: brain tumor segmentation Fully 3D Convolutional NN: to account for 3D correlation Hybrid training: take image patches and densely train them (due to memory constraints) Dense inference: segment the whole volume in a single forward pass. Pre-processing: normalize each input channel (mean, std). Post-processing: remove small connected components (100 voxels) Architectures: - 3D-Net1: 3D extension of FCN8 Long - 3D-Net2: 3D extension of U-Net - 3D-Net3: based on two-paths 2D (1), DeepMedic 3D (2) 11 (1) Havaei et al, Brain tumor segmentation with Deep Neural Networks, 2016 (2) Kamnitsas, Efficient Multi Scale 3D CNN with fully connected CRF for Accurate Brain Lesion Segmentation, 2016 UPCPaper: Casamitjana et al, 3DConvolutional Networks for Brain Tumor Segmentation: a comparison of multiresolution architectures, 2016.
  12. 12. bit.ly/DLCV2018 #DLUPC Segmentation 1: brain tumor segmentation 12 3D-Net1
  13. 13. bit.ly/DLCV2018 #DLUPC Segmentation 1: brain tumor segmentation 13 3D-Net2
  14. 14. bit.ly/DLCV2018 #DLUPC Segmentation 1: brain tumor segmentation 14 3D-Net3
  15. 15. bit.ly/DLCV2018 #DLUPC Segmentation 1: brain tumor segmentation 15 Y: Ground Truth. X: Predicted Segmentation
  16. 16. bit.ly/DLCV2018 #DLUPC Segmentation 2: brain tumor segmentation 16 UPCPaper: Casamitjana et al, Cascaded V-Net Using ROI Masks for BrainTumor Segmentation, 2018 Whole Image (dense training) Smaller filter size 3x3x3 ReLU Residual Connections ROI Masks
  17. 17. bit.ly/DLCV2018 #DLUPC Segmentation 2: brain tumor segmentation 17
  18. 18. bit.ly/DLCV2018 #DLUPC Classification (Exam)1: Skin Classification 18 UPCPaper: Combalia M. Vilaplana V., Monte-CarloSampling applied toMultiple Instance Learning for whole Slide Image Classification, 2018 Task: Classify between Sun Exposed and Not Sun Exposed Pieces of Skin Whole Slide Tissue Image: High Resolution Image, high computational cost to evaluate, to reduce it MIL is used. Multiple Instance Learning (MIL): An input set of N bags, where each bag contains M instances Bag: Images / Instances: Patches Sampling of Patches: Grid- Sampling Uniform Sampling Monte-Carlo Sampling Architecture: - Shallow ResNet (8 Layers) - Patch Size (50x50)
  19. 19. bit.ly/DLCV2018 #DLUPC Classification (Exam)1: Skin Classification 19
  20. 20. bit.ly/DLCV2018 #DLUPC Super-Resolution 1: Brain MRI super-resolution 20 UPCPaper: Sánchez I. Vilaplana V., Brain MRI super-resolution using 3Dgenerative adversarial networks, 2018 Task: Generate Brain MRI High Resolution Images from low Resolution Images Generative Adversarial Network: SRGAN Model (1) extended to 3D Discriminator: least squares adversarial loss
  21. 21. bit.ly/DLCV2018 #DLUPC Super-Resolution 1: Brain MRI super-resolution 21 Generator: least squares adversarial loss and a content term
  22. 22. bit.ly/DLCV2018 #DLUPC Super-Resolution 1: Brain MRI super-resolution 22 Results
  23. 23. bit.ly/DLCV2018 #DLUPC Segmentation 2: Liver Lesion Segmentation 23 UPCPaper: Bellver M. et al. Detection-aided liver lesion segmentation using deep learning, 2017 Task: Segmentation of Lesions of Liver in CT Images Architecture: Cascade networks (based on DRIU, 1) for Liver and Lesion Segmentation (1) Maninis, K. K., et al. (2016, October). Deep retinal image understanding. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 140-148). Springer International Publishing
  24. 24. bit.ly/DLCV2018 #DLUPC Segmentation 2: Liver Lesion Segmentation 24 Used to keep context around lesion tissues Pre Trained Resnet 50 for Image Net without classification layer Single Neuron determining if it’s a healthy tissue or not Reduces false positives Detection Network
  25. 25. bit.ly/DLCV2018 #DLUPC Segmentation 2: Liver Lesion Segmentation 25
  26. 26. bit.ly/DLCV2018 #DLUPC Segmentation 3: Leishmaniasis Parasite Segmentation 26 UPCPaper: Górriz M. et al., Leishmaniasis Parasite Segmentation and Classification using Deep Learning, AMDO2018
  27. 27. bit.ly/DLCV2018 #DLUPC Segmentation 3: Parasite Segmentation 27 Annotation tool
  28. 28. bit.ly/DLCV2018 #DLUPC Segmentation 3: Parasite Segmentation 28
  29. 29. bit.ly/DLCV2018 #DLUPC Segmentation 3: Active Learning & Segmentation 29 UPCPaper: Górriz M. et al. Active Deep Learning for Medical Imaging Segmentation, NIPS 2017 Task: Use Active Learning when not enough labeled data is available or is explensive Active Learning: auto-selection of useful instances to be labeled in order to achieve similar performance with few data Cost Effective Methodology: Automatically Select and Pseu-Annotate good Unlabeled samples using the output of the network
  30. 30. bit.ly/DLCV2018 #DLUPC Segmentation 3: Active Learning & Segmentation 30 Architecture: U-NET
  31. 31. bit.ly/DLCV2018 #DLUPC Classification (Exam) 2: Impact of Segmentation in Exam 31 UPCPaper: Burdick J. et al. The Impact of Segmentation on the Accuracy and Sensitivity of a Melanoma Classifier Based on Skin Lesion Images , SIIM 2017 Original VGG. This method uses a single sigmoid for classification
  32. 32. bit.ly/DLCV2018 #DLUPC Classification (2) : Impact of Segmentation in Classification 32 Perfect Segmentation PS+25 Unsegmented
  33. 33. bit.ly/DLCV2018 #DLUPC Resources ● Datasets ○ The cancer imaging archive ○ Alzheimer’s Disease Neuroimaging Initiative (ADNI) ○ Open-access Medical Image repositories ● Challenges ○ Grand challenges in Biomedical Image Analysis ○ Medical Image Computing and Computer Assisted Interventions (MICCAI) ○ Int. Symposyum on Medical Imaging (ISBI) ● Survey papers: DL in Medical imaging ○ Ravi et al, Deep Learning for Health Informatics, JBHI, 2017 ○ Litjens et al, A Survey on Deep Learning in Medical Image Analysis, 2017 ○ Zhou et al Ed., Deep learning for Medical Image Analysis, Elsevier, 2017 33
  34. 34. Thank You!! Questions? 34

×