A Brief Overview of Deep Learning Approaches to Paintings and Drawings
•
0 likes•46 views
Our paper presented at the FAPER workshop we organized.
Recommended
Recommended
More Related Content
Similar to A Brief Overview of Deep Learning Approaches to Paintings and Drawings
Similar to A Brief Overview of Deep Learning Approaches to Paintings and Drawings (20)
More from Gennaro Vessio
More from Gennaro Vessio (10)
Recently uploaded
Recently uploaded (20)
A Brief Overview of Deep Learning Approaches to Paintings and Drawings
- 1. A Brief Overview of Deep Learning Approaches to Paintings and Drawings Giovanna Castellano and Gennaro Vessio name.surname@uniba.it
- 2. History of Artificial Intelligence Research into the extraction and discovery of patterns in visual artworks has essentially witnessed the same revolution 2
- 3. From feature engineering to Deep Learning Acquiring explicit knowledge to be associated with a particular artist or artwork can be extremely difficult This knowledge, in fact, is typically associated with implicit and subjective skills human observers may find difficult to conceptualize and verbalize Representation learning approaches, such as those on which DL models are based, can be the key to addressing the problem of extracting useful representations from the raw-level pixel features 3
- 5. Main datasets ➔ Classification and retrieval ◆ WikiArt ◆ WikiArt Emotions ◆ Rijksmuseum ◆ The MET ➔ Object detection ◆ People-Art ◆ BAM! ◆ Brueghel ➔ Cross-modal retrieval ◆ SemArt ◆ Artpedia 5
- 6. Some milestones Karayev et al. Transfer learning from traditional photographic domains can be effective for visual style classification 2013 Crowley and Zissermann Cross-depiction problem in object detection in artworks 2014 Tan et al. Visualization techniques showed that a CNN is able to find key objects to classify paintings 2016 Saleh and Elgammal A machine can express semantic judgments related to aesthetics 2015 6
- 7. Some milestones Elgammal et al. A form of art can be generated by using a GAN trained on prior human art 2017 Strezoski and Worring Multi-task models can lead to better performance than unimodal models 2017 Garcia and Vogiatzis Paintings can be retrieved according to an artistic text and vice versa 2018 Cetinic et al. Pre-trained model initialization strongly influences fine-tuning performance 2018 7
- 8. Some milestones Gonthier et al. Iconographic elements that are very specific to art history cannot be easily learned from natural images 2018 Cetinic et al. The emotion evoked in the observer and the memorability of an artwork can be predicted with visual features 2019 Tomei et al. To reduce the feature distribution gap between artistic and natural images, paintings can be translated to photos 2020 Shen et al. Duplicated visual patterns in artwork collections can be automatically discovered 2019 8
- 9. Some milestones Castellano et al. A graph of influences between artists can be constructed based on visual links between artworks 2020 Garcia et al. Injecting contextual information into the learning process can improve style and retrieval performance 2020 You New progress will be made thanks to your contribution… but no spoilers 😁 2021 Garcia et al. First baseline on visual question answering on art 2020 9
- 10. Future challenges? ➔ Domain generalization: agnostic models that generalize well regardless of the test domain ➔ Neuro-symbolic learning: hybrid models to explain the ratio behind the decision made by the machine ➔ Social robots: deployment of CV techniques into robots for several applications (e.g. remote museum visit tours during lockdown) 10