This paper presents the RECOD team experience in the Retrieving Diverse Social Images Task at MediaEval 2015. The teams were required to develop a diversification approach for social photo retrieval. Our proposal is based on irrelevant image filtering, reranking, rank aggregation, and diversity promotion. We proposed a multimodal approach and exploited image metadata and user credibility information. http://ceur-ws.org/Vol-1436/ http://www.multimediaeval.org

1. Recod @ MediaEval 2015:
Diverse Social Images Retrieval
Rodrigo T. Calumby, Iago B. A. do C. Araujo, Vinícius P. Santana
Javier A. V. Munoz, Otávio A. B. Penatti, Lin T. Li, Jurandy Almeida
Giovani Chiachia, Marcos A. Gonçalves, and Ricardo da S. Torres
Acknowledgments: UEFS/PROBIC, FAPESP, and MediaEval 2015 Organizers
UEFS
contact: rtcalumby@ecomp.uefs.br
Daniel Moreira
on behalf of

2. Face detectionGeographic
Visual / Textual / Credibility / Geo
input
output
Clustering
Representative selection
Filtering
Reranking
Diversification
Blur
Genetic programming
Fusion
Relevance-based
Selection
(up to 150-top ranked)
Filtering
Approach
Recod @ MediaEval 2015: Diverse Social Images Retrieval

3. Geo Filter
Face Filter
#faces > 1 → non-relevant
(location: Christ the Redeemer, Rio de Janeiro)
10km radius
limit from
reference
lat/long of the
location
Blur Filter
(location: Iguazu Falls, Brazil/Argentina)
α > 0.8 → non-relevant
Filtering
Recod @ MediaEval 2015: Diverse Social Images Retrieval

4. Face detectionGeographic
Visual / Textual / Credibility / Geo
input
output
Clustering
Representative selection
Filtering
Reranking
Diversification
Blur
Genetic programming
Fusion
Relevance-based
Selection
(up to 150-top ranked)
Filtering
Approach
Recod @ MediaEval 2015: Diverse Social Images Retrieval

5. Recod @ MediaEval 2015: Diverse Social Images Retrieval
Reranking
Fusion
(Genetic Programming)
{ VisualRank1 TextRank CredRank GeoRank
CombSUM MRA
BordaCount
VisualRank2
CombSUM
Ranked List
INPUT
OUTPUT {
Original Re-ranked
(location: Casa Batlló, Barcelona)
Location representatives:
VISUAL
Textual query: Locality text
Credibility: user scores
GEOGRAPHIC

6. Recod @ MediaEval 2015: Diverse Social Images Retrieval
Parameter Value
Number of
generations
30
Genetics
operators
Reproduction, Mutation,
Crossover
Fitness
functions
FFP1, WAS, MAP, NDCG
Rank Agg.
methods
CombMAX, CombMIN,
CombSUM, CombMED,
CombANZ, CombMNZ,
RLSim, BordaCount, RRF,
MRA

7. Face detectionGeographic
Visual / Textual / Credibility / Geo
input
output
Clustering
Representative selection
Filtering
Reranking
Diversification
Blur
Genetic programming
Fusion
Relevance-based
Selection
(up to 150-top ranked)
Filtering
Approach
Recod @ MediaEval 2015: Diverse Social Images Retrieval

8. Clustering
Selection
- Descending cluster size
- Best connected items
from each cluster
(location: Arc de Triomphe, Paris)
kMedoids: 30 to 40 clusters
Initial medoids: rank offset positions
Output list
Recod @ MediaEval 2015: Diverse Social Images Retrieval

9. Run Filtering Reranking Diversity
1 Geo*, face, blur Visual* BIC
2 Geo*, face, blur Textual Cosine + Jaccard
3 Geo*, face, blur Visual*, textual Jaccard
4 Geo*, face, blur Credibility Users
5 Geo*, face*, blur Visual*, textual, credibility, geo* Jaccard
*only for one-topic queries
Submitted Runs
Recod @ MediaEval 2015: Diverse Social Images Retrieval

10. Recod @ MediaEval 2015:
Diverse Social Images Retrieval
Rodrigo T. Calumby, Iago B. A. do C. Araujo, Vinícius P. Santana
Javier A. V. Munoz, Otávio A. B. Penatti, Lin T. Li, Jurandy Almeida
Giovani Chiachia, Marcos A. Gonçalves, and Ricardo da S. Torres
Acknowledgments: UEFS/PROBIC, FAPESP, and MediaEval 2015 Organizers
UEFS
contact: rtcalumby@ecomp.uefs.br
Thank you!