This document proposes using a Deep Convolutional Embedding method to cluster Pablo Picasso's artworks in an unsupervised manner based on visual features. It trains an autoencoder to learn a nonlinear mapping of the artwork images to a latent space, then performs k-means clustering in that space. Testing on a database of 439 Picasso paintings, it achieved good clustering performance according to evaluation metrics. Future work will label clusters based on art literature and analyze the encoder to identify distinguishing objects that drove the clustering.

1. Deep Convolutional Embedding for Painting
Clustering: Case Study on Picasso’s Artworks
Giovanna Castellano, Gennaro Vessio
CILAB, Computer Science Department, University of Bari, Italy
gennaro.vessio@uniba.it

2. Context
Our cultural heritage is of inestimable
importance for the cultural, historical
and economic growth of our society
It is important for:
● art historians
● economists
● museum curators
● …
● computer scientists!
2

3. The role of
Computer Science
Research by computer scientists in this context is
mainly focused on applying machine learning/pattern
recognition for:
● classification and categorization
● link prediction
● information retrieval
● knowledge discovery
● …
This growing interest has been motivated by the
increasing availability of large-scale digitized art
collections (e.g., WikiArt)
3

4. Motivations
Human beings find similarity relationships
among paintings based on their aesthetic
perception
This perception (which can also be influenced by
subjective experience) is:
● extremely hard to conceptualize
● difficult to translate into features and labels
Our goal is to develop an automatic tool to group
digital paintings based on:
● “visual” features
● an unsupervised approach
4

5. Limitations of traditional methods
1. Applying traditional algorithms like k-means on the high-dimensional raw pixel
space can be ineffective
2. The application of reduction techniques, such as PCA, can ignore nonlinear
relationships between the original input and the latent feature space
3. Some variants of k-means (e.g., spectral clustering) are computationally
expensive as the data grows
4. Engineering meaningful features based on domain knowledge is extremely
difficult
5

6. We propose to use a refinement of the Deep Convolutional Embedding Clustering
(DCEC) method recently proposed by Guo et al.
Main benefits:
1. deep learning algorithms are good at mapping input to output data due to their
exceptional ability to express nonlinear representations
2. conv layers are even better when the input is complex image data
6
Key points of the proposed method

7. Overall network architecture
7

8. Model training works in two phases:
1. parameter initialization
a. use a convolutional autoencoder to learn a nonlinear mapping between the original space X
and a latent spaze Z
2. parameter optimization
a. initialize k cluster centroids with k-means
b. compute a soft assignment between the embedded points and cluster centroids
c. compute an auxiliary target distribution
d. minimize the KL divergence with respect to the computed target distribution
Input reconstruction and cluster assignment are jointly optimized
8
Key points

9. Case study
We used a database that collects 439 artworks
by a very popular artist: Pablo Picasso
This was done to evaluate the effectiveness of
the method in finding meaningful clusters within
the artist’s production
9

10. Experimental setting
● Input resized to 128×128 pixels and normalized to [0, 1]
● Autoencoder pre-training:
○ epochs: 200
○ mini-batch size: 128
○ optimizer: AdaMax
● Overall training:
○ delta: 0.001
○ optimizer: AdaMax
● Evaluation metrics:
○ silhouette score
○ Calinski-Harabasz index
10

11. Clustering performance
11
# clusters silhouette score Calinski-Harabasz index
2 0.933 0.737
3 0.936 0.771
4 0.951 0.768
5 0.965 1.000
6 0.962 0.812

12. Qualitative evaluation
12

13. Conclusion
Encouraging preliminary results were obtained, which confirm the effectiveness of
the deep clustering approach to address highly complex image domains, such as
the artistic one
Future work will use much of the existing literature on Picasso to try to label
paintings to perform a much more systematic evaluation, even according to
external clustering criteria
Finally, the convolutional layers of the encoder will be analyzed to find out which
are the distinctive objects in the paintings that led to their clustering
13