Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

MediaEval 2017 - Medical Multimedia Task: Multimedia for Medicine: The Medico Task at MediaEval 2017 (Overview)

140 views

Published on

Presenter: Konstantin Pogorelov, Simula Research Laboratory, University of Oslo, Norway

Paper: http://ceur-ws.org/Vol-1984/Mediaeval_2017_paper_3.pdf

Video: https://youtu.be/V2vFNXKSFrM

Authors: Michael Riegler, Konstantin Pogorelov, Pål Halvorsen, Carsten Griwodz, Thomas de Lange, Kristin Ranheim Randel, Sigrun Losada Eskeland, Duc-Tien Dang-Nguyen, Mathias Lux, Concetto Spampinato

Abstract: The Multimedia for Medicine Medico Task, running for the first time as part of MediaEval 2017, focuses on detecting abnormalities, diseases and anatomical landmarks in images captured by medical devices in the gastrointestinal tract. The task characteristics are described, including the use case and its challenges, the dataset with ground truth, the required participant runs and the evaluation metrics.

Published in: Science
  • Be the first to comment

  • Be the first to like this

MediaEval 2017 - Medical Multimedia Task: Multimedia for Medicine: The Medico Task at MediaEval 2017 (Overview)

  1. 1. Multimedia for Medicine: The Medico Task Michael Riegler, Konstantin Pogorelov, Pål Halvorsen, Carsten Griwodz, Thomas de Lange, Kristin Ranheim Randel, Sigrun Losada Eskeland, Duc-Tien Dang-Nguyen, Mathias Lux, Concetto Spampinato Email: michael@simula.no, konstantin@simula.no
  2. 2. Medico  Bringing together IT and medicine, focusing on detecting abnormalities, diseases and anatomical landmarks in images captured by medical devices in the gastrointestinal tract  Goals of the task: Help to improve the health care system using multimedia methods to reach the next level of multimedia-assisted computer diagnosis, detection and interpretation of abnormalities  Attract more researchers to medical use cases
  3. 3. GI Tract Challenges  Many types of diseases can affect the human digestive system  Screening of the gastrointestinal (GI) tract using different types of traditional endoscopy – is costly (colonoscopy: US - $1100/patient, $10 billion dollars) – consumes valuable medical personnel time (1-2 hours) – does not scale to large populations – is intrusive to the patient  Current developments in technology may potentially enable automatic algorithmic screening and assisted examinations  a true interdisciplinary activity with high chances of societal impact
  4. 4. Colorectal Cancer Mortality 2012  Most common cancer for men in Norway  Second most common cancer for women in Norway
  5. 5. Live Automatic Detection  System to assist doctors during live traditional endoscopy procedures  Second pair of eyes  Support for inexperienced doctors  Automatic tagging of lesions  Automated reports generation  Better procedure documentation [1] van Rijn, J. C., Reitsma, J. B., Stoker, J., Bossuyt, P. M., van Deventer, S. J., and Dekker, E. Polyp miss rate determined by tandem colonoscopy: a systematic review. The American journal of gastroenterology 101, 2 (2006)
  6. 6. Wireless Video Capsule (Capsular Endoscopy)  better scale  less intrusive  possible to combine examinations!?  less expensive? (detection might lead to an endoscopy)  expensive  does not scale  intrusive
  7. 7. Our Goals  A complete system for: – Live Traditional Endoscopy – Capsular Endoscopy  Automatic detection of different abnormalities in the digestive system – HD sources – Real-time and faster – High recall and precision – Automated reports generation
  8. 8. EIR Overview
  9. 9. Common Challenges  Blurry images due to camera motion  Objects too close to camera  Under or over scene lighting  Flares  Artificial objects and natural "contaminants“  Low resolution of Capsular Endoscopes  No proper support for medical reports generation
  10. 10. Common Challenges
  11. 11. Subtasks  Detection  Detection of different diseases and landmarks  Use as few images in the training dataset as possible  Efficient detection  Solve the classification problem as fast as possible  Efficient detection (fare evaluation, the same hardware)  Evaluate detection algorithms on the same hardware  Report generation (Experimental)  Automatically create a text-report for a medical doctor for three video cases
  12. 12. Dataset  Hard to find annotated data allowed to use (even) for research  Two data sets have been published at the MMSys 2017 open dataset track: Kvasir and Nerthus (bowel preparation quality)  Kvasir: the open source dataset consist of 8,000 annotated GI tract images in 8 different classes: 500 in dev. + 500 in test set per class
  13. 13. Participants and submissions  12 teams registered and requested the data  5 teams successfully submitted the results  Austria, China, India, Italy, Norway, Pakistan, US…  46 submissions in total  Detection – Successful  Efficient detection – Partially Successful  Efficient detection – No-Show  Report generation – No-Go
  14. 14. The Approaches  Machine Learning – Manifold learning – Support vector machine – Random forest and random tree – Logistic model tree – Regression-based – Unsupervised clustering  Convolutional Neural Networks – Direct classification – Transfer learning  Features Extraction – Deep features – Global and Local features
  15. 15. Metrics  Multi-class generalization of Matthews correlation coefficient (R_K statistic) – Correlation coefficient between the observed and predicted classifications – Perfect for unbalanced multi-class case – Maximum value +1 for perfect prediction – Minimum value between -1 and 0 depending on the true distribution • Frames per second (FPS)
  16. 16. The Results – Best Performing Team Training set size Multi-class MCC F1 FPS SCL-UMD 3200 0.827 0.848 1.3 FAST-NU-DS 4000 0.736 0.767 2.3 ITEC-AAU 3600 0.724 0.755 1.4 HKBU 1000 0.663 0.703 2.2 SIMULA 4000 0.802 0.826 46.0 Random - -0.001 0.124 - ZeroR - 0 0.125 -
  17. 17. The Results – Best Submission Confusion Matrix Predicted class Actual class polyps normal- cecum normal-z- line normal- pylorus esophagitis dyed- resection- margins dyed- lifted- polyps ulcerative -colitis polyps 448 13 1 0 0 0 23 33 normal-cecum 22 478 0 0 0 0 0 27 normal-z-line 0 0 427 8 202 0 0 0 normal-pylorus 4 0 5 480 5 0 0 0 esophagitis 0 0 67 10 293 0 0 2 dyed-resection- margins 0 0 0 0 0 406 55 1 dyed-lifted- polyps 2 0 0 0 0 94 421 0 ulcerative-colitis 24 9 0 2 0 0 1 437
  18. 18. The Results – Smallest Training Set Team Training set size Multi-class MCC F1 ITEC-AAU 400 0.607 0.649 FAST-NU-DS 732 0.649 0.689 HKBU 800 0.648 0.692 SCL-UMD 3200 0.827 0.848 SIMULA 4000 0.802 0.826 Random - -0.001 0.124 ZeroR - 0 0.125
  19. 19. Conclusions 1. Good classification performance achieved 2. Small amount of training data would not stop us! 3. Real-time is still challenging 4. Combined approaches are required for medical image analysis 5. Medical task is interesting for the community
  20. 20. Future of the task 1. Localization/segmentation of findings and lesions 2. Exploiting domain expert knowledge – more data! 3. Integration of various data, multi-modality – new sensors, doctors’ records, audio recordings, patient context information 4. Automated reporting
  21. 21. Thank You! Questions? All the data is released publicly and available at: http://datasets.simula.no/kvasir/

×