Presenter: Konstantin Pogorelov, Simula Research Laboratory, University of Oslo, Norway Paper: http://ceur-ws.org/Vol-1984/Mediaeval_2017_paper_48.pdf Video: https://youtu.be/M4VcDocqHac Authors: Konstantin Pogorelov, Michael Riegler, Pål Halvorsen, Carsten Griwodz, Thomas de Lange, Kristin Ranheim Randel, Sigrun Losada Eskeland, Duc-Tien Dang-Nguyen, Olga Ostroukhova, Mathias Lux, Concetto Spampinato Abstract: This paper presents our approach for the 2017 Multimedia for Medicine Medico Task of the MediaEval 2017 Benchmark. We propose a system based on global features and deep neural networks, and preliminary results comparing the approaches are presented.

1. A Comparison of Deep Learning with
Global Features for
Gastrointestinal Disease Detection
Konstantin Pogorelov, Michael Riegler, Pål Halvorsen,
Carsten Griwodz, Thomas de Lange, Kristin Ranheim Randel,
Sigrun Losada Eskeland, Duc-Tien Dang-Nguyen,
Olga Ostroukhova, Mathias Lux, Concetto Spampinato
Email: konstantin@simula.no

2. Used approaches
 Global Features (GF) + Machine Learning
 Simple Convolutional Neural Network (CNN)
 6 layers
 Pretrained CNN + Transfer Learning (TL)
 Deep Features + Machine Learning
 Machine learning classification:
 Random tree (RT)
 Random forest (RF)
 Logistic model tree (LMT)

3. Feature-based approaches
Features
extractor
 Training
 Classifying
Features
Features
Polyps
…
Features
extractor
Features
Normal
Supervised
machine learning
classifier
Training set
Image
classFeatures

4. Global Features
 LIRE for features extraction
 Used features: JCD, Tamura, Color Layout, Edge
Histogram, Auto Color Correlogram and Pyramid
Histogram of Oriented Gradients
 Supervised machine learning for classification (WEKA)

5. Transfer Learning
Inception v3

6. Deep Features
 TensorFlow as backend
 Pre-trained on ImageNet dataset
Inception v3 and ResNet50 models
 Concepts
 Classify into 1000 classes
 Classes’ weights as features vector
 Features
 Top layer input as features vector (16384 for
Inception v3 and 2048 for ResNet50)
 Supervised machine learning for
classification (WEKA)
Pretrained
model
Output or
top-layer
input weights
WEKA for
classification

7. Initial Evaluation

8. Runs submitted
 Performance
 Confusion matrix

9. Conclusions
Thank You! Questions?
 Deep features are the best for medical images (Surprise!)
 Real-time multi-class detection is possible
 Time-consuming models retraining is useless
 Deep features are the best for coarse classification
 Fine sub-classification step should be added