In this paper, we describe our approach for the MediaEval 2015 Retrieving Diverse Social Images Task. The proposed approach exploits available user-generated textual descriptions and the visual content of the images in a combination with common, unsupervised clustering techniques in order to increase the diversification of retrieval results. Preliminary experiments indicate that the approach generalizes well for different datasets and achieves comparable results for single-and multi-topic queries. http://ceur-ws.org/Vol-1436/ http://www.multimediaeval.org

1. MIS @ Retrieving
Diverse Social Images Task 2015
Lukas Diem and Maia Zaharieva
MediaEval 2015 Benchmark Workshop, September 14-15, Wurzen, Germany

2. Features
• Visual-based
• CNN adapted features provided by the organizers
• Text-based
• LDA (Latent Dirichlet allocation)
• Title and tags
• 50 topics per query
• TF-IDF
• Title, tags, and description
• Cleanup step and handling of missing text

3. Approach
• Data preprocessing
• Face and GPS filter
• Relevance reranking
• Visual-based similarity to wiki or top 10 images
• CNN outperforms other features significantly
+15% P@20 compared to CN3x3
• Image clustering and selection
• AHC with Ward’s aggregation and 50 clusters
• Round-Robin selection approach

4. Lessons Learned
Lukas Diem, MediaEval 2015 Benchmark Workshop, September 14-15, Wurzen, Germany

5. Lessons Learned
Lukas Diem, MediaEval 2015 Benchmark Workshop, September 14-15, Wurzen, Germany

6. Lessons Learned
Lukas Diem, MediaEval 2015 Benchmark Workshop, September 14-15, Wurzen, Germany

7. Lessons Learned
Lukas Diem, MediaEval 2015 Benchmark Workshop, September 14-15, Wurzen, Germany
LDA
TF-IDF

8. Lessons Learned
Lukas Diem, MediaEval 2015 Benchmark Workshop, September 14-15, Wurzen, Germany
LDA
TF-IDF
CNN

9. Lessons Learned
Lukas Diem, MediaEval 2015 Benchmark Workshop, September 14-15, Wurzen, Germany
LDA
TF-IDF
CNN
LDA
TF-IDF

10. Thank you
Contact: Lukas Diem, l.diem@univie.ac.at
MediaEval 2015 Benchmark Workshop, September 14-15, Wurzen, Germany