Download to read offline
Uploaded byAlican Bozkurt
Delineation of Skin Strata in Reflectance Confocal Microscopy Images with Recurrent Convolutional Networks
Presented in CVPR Workshop on Computer Vision in Microscopy Image Analysis, Honolulu, 21 July 2017
Delineation of Skin Strata in Reflectance Confocal Microscopy Images with Recurrent Convolutional Networks
- 1. Delineation of SkinStrata in Reflectance Confocal Microscopy Images With Recurrent Convolutional Networks Alican Bozkurt*, Trevor Gale*, Kivanc Kose, Christi Alessi-Fox, Dana Brooks, Milind Rajadhyaksha, Jennifer Dy
- 2.
- 3. The Skin &Dermal-Epidermal Junction (DEJ) • Outer shell of the body • Exposed to outside effects, e.g. sun, chemicals, etc... • Skin cancer is the most common cancer in USA • Melanoma is among the deadliest. https://openi.nlm.nih.gov/detailedresult.php?img=PMC2914370_cde0002-0103-f03&req=4 https://www.cancer.org/cancer/melanoma-skin-cancer/detection-diagnosis-staging/survival-rates-for-melanoma-skin-cancer-by-stage.html • Most skin cancers emerge from dermal-epidermal junction (basal layer) • Early detection is important • Early stage→5 Year survival 98% • Late stage → 5 year Survival 18%
- 4.
- 5.
- 6.
- 7.
- 8. Current Practice -Biopsy • Invasive - > Scarring • Incisional biopsies may miss • %80 biopsies turn out to be benign
- 9. Lesions overall body Clinical Examination Clinically Suspicious Dermoscopy Dermoscopically Suspicious Biopsy Cancer or Not CurrentPractice • Invasive - > Scarring • Incisional biopsies may miss • %80 biopsies turn out to be benign
- 10.
- 11. Lesions overall body Clinical Examination Clinically Suspicious Dermoscopy Dermoscopically Suspicious Reflectance Confocal Microscopy RCM Suspicious Biopsy Canceror Not Current Practice • Non-invasive-> No scarring • RCM Improves “Number Need to Treat” by 2x (But in the hands of a trained clinician)
- 12. Reflectance Confocal Microscopy 1kx 1k pixels= 0.5mm x 0.5mm 12k x 12k pixels= 6mm x 6mm 0.2mm
- 13. Skin Imaging: BiopsyVs RCM https://openi.nlm.nih.gov/detailedresult.php?img=PMC2914370_cde0002-0103-f03&req=4 Epidermis Dermis
- 14. Skin Imaging: BiopsyVs RCM https://openi.nlm.nih.gov/detailedresult.php?img=PMC2914370_cde0002-0103-f03&req=4
- 15. Data & Objective •Given a stack, label each image Epidermis, DEJ, or Dermis
- 16. Classification using singleimages • Classify images independently (not using sequence information) • Inception v3 (Szegedy et al.,2015) • Trained from scratch • Other methods • Hames et al. (2016): BoW + linear SVM • Kaur et al. (2016): MR8 filters + histogram + MLP Hames, Samuel C., et al. "Automated segmentation of skin strata in reflectance confocal microscopy depth stacks." PloS one 11.4 (2016): e0153208. Kaur, Parneet, et al. "Hybrid deep learning for Reflectance Confocal Microscopy skin images." Pattern Recognition (ICPR), 2016 23rd International Conference on. IEEE, 2016.
- 17. Results • Non-sequential methodscan’t capture structure of the problem • Epidermis → DEJ → Dermis Accuracy (%) Error types Total # ErrorEpidermis → Dermis DEJ → Epidermis Dermis → Epidermis Dermis → DEJ Inception-v3 84.87 3 25 8 32 68 Hames et al. 84.48 14 59 11 56 140 Kaur et al. 64.33 32 255 16 99 402
- 18.
- 19. Incorperating sequential information-RNN • Recurrent: 𝑠𝑡 = 𝑓 𝑥 𝑡, 𝑠𝑡−1 • LSTM (Hochreiter & Schmidhuber, 1997) • GRU (Chung et al., 2014)
- 20.
- 21.
- 22. Results • Sequential methodsimprove performance drastically • Is it overkill? Accuracy (%) Error types Total # ErrorEpidermis → Dermis DEJ → Epidermis Dermis → Epidermis Dermis → DEJ Full seq. RCN 87.97 0 4 0 3 7 Inception-v3 84.87 3 25 8 32 68 Hames et al. 84.48 14 59 11 56 140 Kaur et al. 64.33 32 255 16 99 402
- 23. Partial Sequence RCN •Motivation: Experts do not look at all the images in a stack while labeling an image
- 24. Results • Sequential methodsimprove performance drastically Accuracy (%) Error types Total # ErrorEpidermis → Dermis DEJ → Epidermis Dermis → Epidermis Dermis → DEJ Full seq. RCN 87.97 0 4 0 3 7 Par. Seq. RCN 87.52 3 10 5 5 23 Inception-v3 84.87 3 25 8 32 68 Hames et al. 84.48 14 59 11 56 140 Kaur et al. 64.33 32 255 16 99 402
- 25.
- 26. Conclusions • RCM asbeneficial screening before biopsy • Convolutional + recurrent units for highly structured problem • Deep learning based methods beat hand-tuned features • Adding sequential information improves performance • RCNs beat vanilla CNNs • Using only 3 images gives similar accuracy to best model
- 27. Acknowledgements • This workwas supported in part by the National Cancer Institute (NCI) under grants R01CA156773 and R01CA199673, the National Institute of Biomedical Imaging and Bioengineering’s (NIBIB) Image Guided Interventions Program under grant R01EB012466, the NCI Core Center grant P30CA008748, and NIBIB grant R01EB020029.