This paper details the participation of the UNED-UV group at the 2015 Retrieving Diverse Social Images Task. This year, our proposal is based on a multi-modal approach that firstly applies a textual algorithm based on Formal Concept Analysis (FCA) and Hierarchical Agglomerative Clustering (HAC) to detect the latent topics addressed by the images to diversify them according to these topics. Secondly, a Local Logistic Regression model, which uses the low level features and some relevant and non-relevant samples, is adjusted and estimates the relevance probability for all the images in the database. http://ceur-ws.org/Vol-1436/ http://www.multimediaeval.org

1. A. Castellanos1, X. Benavent2, A. García-Serrano1, E. De Ves2, J. Cigarrán1
(1): ETSI Informática, UNED.
(2): Dept. d’Informàtica, ETSE, Univ. de València.
{acastellanos, agarcia, juanci}@lsi.uned.es
{xaro.benavent, esther.deves}@uv.es
ABSTRACT
This paper details the participation of the UNED-UV group at the 2015 Retrieving Diverse
Social Images Task. This year, our proposal is based on a multi-modal approach that firstly
applies a textual algorithm based on Formal Concept Analysis (FCA) and Hierarchical
Agglomerative Clustering (HAC) to detect the latent topics addressed by the images to
diversify them according to these topics. Secondly, a Local Logistic Regression model, which
uses the low level features and some relevant and non-relevant samples, is adjusted and
estimates the relevance probability for all the images in the database.
CONTENT-BASED INFORMATION RETRIEVAL
Content-Based Image retrieval models the user preferences by using a
relevance feedback algorithm. The general methodology involves five steps:
STEP 1: Reduction of the data dimensionality: the low-level features, x, are
reduced by a Principal Component Analysis, PCA, which retains only the first
components that account for 80% of the data variability, xr-
STEP 2: Selecting the relevant and non-relevant sets:
• Set of relevant/positive images, IP:
• One-topic: images given by Wikipedia
• Multi-topic: first five image result list.
• Set of non-relevant/negative images, Is: a generated set taken into
account the guidelines (photos with regular people as main subject, photos
with riots and protests).
STEP 3: Parameter estimation of the Local Logistic Regression Models:
• The reduced feature vector, xr is dynamically splitted in m groups of non-
fixed size.
• The xr of the IP and Is images are the inputs of the models.
• The output is the relevance probability, p, of image Ij.
STEP 4: Fusion of the probability vector p in a value (scalar) for each
image using a weighted average:
• The weights, w, for a given probability are obtain by the amount of
variance accounted for the group of components used to adjust the model.
STEP 5: Ranking of the database: The final diversity similarity ranks is
generated by selecting the highest probability image at each textual cluster
(run3, run5).
For run1, automatic using only visual information, the clusters are made by a
k-means (k=50) over the PCA components xr.
UNED-UV @ Retrieving Diverse Social Images Task
METHODOLOGY
Our multimedia system has two sub-systems:
• Textual-Based Information Retrieval system, TBIR, that works with textual information
and generates clusters for diversity
• Content-Based Information Retrieval system, CBIR, that estimates the relevance of
each of the images of the generated clusters.
TEXTUAL-BASED INFORMATION RETRIEVAL
STEP 1: Kullback Leiber Divergence (KLD) to obtain the image representation using the
most divergent terms: weights in the first tertile (not only the most relevant but also the most
“different and original”).
STEP 2: A theory (Formal Concept Analysis, FCA) to organize the images in a lattice
according its shared KLD-features (for 123 locations).
STEP 3: HAC approach to identify the most Formal Concepts, as they will contain the most
diverse images:
• The Zero-Induces measure used to identify the similarity between lattice nodes (Formal
Concepts).
• Regrouping of the lattice nodes maximizing diversity.
Metrics run1 run2 run3 run5
One-topic
P@20 0.6362
0.7094
0.7051
0.7645
CR@20 0.3704
0.4082
0.3995
0.4194
F1@20 0.4618
0.5068
0.4988
0.5240
Multi-topic
P@20 0.7300
0.6393
0.7207
0.7886
CR@20 0.4257
0.4407
0.4116
0.4491
F1@20 0.5130
0.5025
0.5001
0.5519
Overall
P@20 0.6835
0.6741
0.7129
0.7766
CR@20 0.3983
0.4246
0.4056
0.4344
F1@20 0.4876
0.5046
0.4994
0.5380
Table 2. Official Metrics for Retrieving Diverse Social Images Task. Best result for each topic set is in
bold, and best result for the automatic runs is in blue. First block results are for the one-topic subset,
the second for the multi-topic subset, and the third for the overall set.
RESULTS
CONCLUSSIONS AND FUTURE WORK
• The best results are achieved by the manual version of the multimedia
approach.
• Our challenge: is to make the automatic approach to be able to select
the relevant and non-relevant images as a human being.
• Our best result for all metrics is with the multimedia human-based
approach, run5.
• For the automatic runs, the best result for run2 at the one-topic
subsest, run1 and run2 for multi-topic, and run2 and run3 for the
Overall.
Runs/
Algorithms Details of the two algorithms run1 run2 run3 run5
TBIR
Clusters with FCA ✓ ✓ ✓
Ranking by selecting the first ranked image at each FCA cluster ✓
CBIR
Clusters with k-means using PCA visual low-level features ✓
Automated image selection sets:
• Relevant: Wikipedia images or first five ranked images
• Non-relevant: generated set from non-relevance guidelines.
✓ ✓
Manual selection of relevant and non-relevant image sets. ✓
Ranking by selecting highest probability at each FCA cluster ✓ ✓
Ranking by selecting highest probability at each K-means cluster ✓
Table 1. Details of the TBIR and CBIR algorithms used for the submitted runs.
EXPERIMENTS
ACKNOWLEDGEMENTS
This work has been partially supported by VOXPOPULI
(TIN2013-47090C3-1-P) and DPI2013- 47279-C2-1-R Spanish projects.
EXPERIMENT RESULT: (run5, multimedia with relevance feedback algorithm)
Topic 249 Pamplona bull run (P@20=0.9000, CR@20=0.8333, F1@20= 0.8654)