The document presents a non-parametric unsupervised approach for content-based image retrieval and clustering. It uses spectral clustering to group images retrieved from Flickr based on keyword searches. The method removes outlier images from the initial retrieval set and clusters the remaining visually similar images without using metadata or tags. It was tested on queries for 3 cultural heritage sites and was able to cleanly cluster hundreds of images for each site with no prior dataset knowledge.

1. A Non-parametric Unsupervised Approach
for Content Based Image Retrieval and
Clustering
Technical
University
of
Crete
Authors:
Konstantinos Makantasis (Technical University of Crete)
Anastasios Doulamis (Technical University of Crete)
Nikolaos Doulamis (Cyprus University of Technology)
4th Workshop on Analysis and Retrieval of Tracked Events and Motion in Imagery Streams (ARTEMIS 2013)

2. OUTLINE
 System Overview
 Web Query
 Image Retrieval
 Outliers Removal
 Image Clustering
 Results
4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS (ARTEMIS
2013)

3. SYSTEM OVERVIEW
Web Query
Image Retrieval
Image Clustering
Fully Automatic CBIR
4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

4. WEB QUERY – IMAGE RETRIEVAL
 Query
 Flickr Image Database
 Flickr API in Python
 Query keywords are
associated with images’ title
 Retrieval
 Retrieved Set: hundreds to
thousands of photos
 Retrieved Content: cultural
heritage monuments
4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

5. OUTLIERS REMOVAL (1)
 Retrieved dataset contains many
outliers (visually dissimilar images)
 Inconsistent human generated tags
 Uninformative machine generated
tags
 Absence of camera generated meta-
data
(Retrieved dataset for “Porta Nigra”)
4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

6. OUTLIERS REMOVAL (2)

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

7. OUTLIERS REMOVAL (3)

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

8. OUTLIERS REMOVAL (4)
Initial
Retrieved Set
Visually
Similar Images
Outliers
4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

9. SPECTRAL CLUSTERING

4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

10. EXPERIMENTAL RESULTS (1)
 Queries for 3 cultural heritage monuments
 Porta Nigra
 Parthenon, Athens
 Descobrimentos
Initial Set
Outliers
Final Set
Number of Clusters
Porta Nigra
500
227 (45.4%)
273 (54.6%)
2
Parthenon
500
321 (64.2%)
179 (35.8%)
1
Descobrimentos
500
104 (20.8%)
396 (79.2%)
2
4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

11. EXPERIMENTAL RESULTS (2)
Porta Nigra
Initial
retrieved
Set
4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

12. EXPERIMENTAL RESULTS (3)
Parthenon
Initial
retrieved
Set
4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

13. EXPERIMENTAL RESULTS (4)
Descobrimentos
Initial
retrieved
Set
4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

14. CONTRIBUTION
 Fully Automatic and Non-parametric algorithm
 Handle digital and analog “born” images
 Handle historic images
 Based only on visual information
 No a priori knowledge of the dataset
4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)

15. QUESTIONS
4TH WORKSHOP ON ANALYSIS AND RETRIEVAL OF TRACKED EVENTS AND MOTION IN IMAGERY STREAMS
(ARTEMIS 2013)