Réunion du GdR ISIS Titre : Traitement du signal et des images pour l'art et le patrimoine Dates : 2016-05-13 Lieu : Télécom Paristech, amphi B310 http://gdr-isis.fr/index.php?page=reunion&idreunion=305

1. Introduction
Proposed Work
Results and Discussions
Exploring Global Reflection Symmetry in Visual
Arts
M. ELAWADY1
, P. COLANTONI2
, C. DUCOTTET1
, C. BARAT1
1
Universit´e de Lyon, CNRS, UMR 5516, Laboratoire Hubert Curien,
Universit´e de Saint-´Etienne, Jean-Monnet, F-42000 Saint-´Etienne, France
2
Universit´e Jean Monnet, CIEREC EA n0
3068, Saint-´Etienne, France
GdR-ISIS, May 2016
Traitement du signal et des images pour l’art et le patrimoine
UMR • CNRS • 5516 • SAINT-ETIENNE
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

2. Introduction
Proposed Work
Results and Discussions
Table of Contents
1 Introduction
Problem Definition
Motivation
2 Proposed Work
Related Work
Our Method
3 Results and Discussions
Datasets
Results
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

3. Introduction
Proposed Work
Results and Discussions
Problem Definition
Motivation
Table of Contents
1 Introduction
Problem Definition
Motivation
2 Proposed Work
Related Work
Our Method
3 Results and Discussions
Datasets
Results
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

4. Introduction
Proposed Work
Results and Discussions
Problem Definition
Motivation
What is Art?
Pulitzer Prize for Feature Photography1
(1963, 1973, 1983; 1994, 2003,
2013)
1http://www.pulitzer.org/prize-winners-by-category/217
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

5. Introduction
Proposed Work
Results and Discussions
Problem Definition
Motivation
Art Elements and Principles
The Elements of Design
(the tools to make art)
Line
Shape
Form
Colour
Texture
Space
Horizontal, vertical, diagonal,
straight, curved, dotted, broken
thick, thin.
2D (two dimensional)/ flat
Geometric (square, circle, oval, triangle)
Organic (all other shapes)
3D (three dimensional),
Geometric (cube, sphere, cone),
Organic (all other forms such as: people,
animals, tables, chairs, etc).
Refers to the wavelengths of light.
Refers to hue (name), value (lightness/darkness),
intensity (saturation, or amount of pigment),
and temperature (warm and cool).
Relates to tint, tone and shade.
The feel, appearance, thickness,
or stickiness of a surface
(for example: smooth, rough, silky, furry).
The area around, within, or between
images or parts of an image (relates
to perspective). Positive and negative space.
Value
The lightness or darkness of an image
(or part of an image).
The Principles of Design
(how to use the tools to make art)
Pattern
Contrast
Emphasis
Balance
Scale
Harmony
Rhythm/
Movement
A regular arrangement of alternated or
repeated elements (shapes, lines, colours)
or motifs.
The juxtaposition of different elements of design
(for example: rough and smooth textures, dark and light values)
in order to highlight their differences and/or create
visual interest, or a focal point.
Special attention/importance given to one part of a work of
art (for example, a dark shape in a light composition).
Emphasis can be acheived through placement, contrast, colour,
size, repetition... Relates to focal point.
A feeling of balance results when the elements of design
are arranged symmetrically or asymmetrically to create
the impression of equality in weight or importance.
The relationship between objects with respect to
size, number, and so on, including the relation
between parts of a whole.
The arrangement of elements to give the viewer
the feeling that all the parts of the piece form a
coherent whole.
The use of recurring elements to direct the eye through the
image; the way the elements are organized to lead the eye to
the focal area. The eye can be directed, for example, along
edges and by means of shape and colour.
Unity All parts of an image work together to be
seen as a whole.
Variety Using different elements in an image to
create visual interest.
1http://splitcomplementary.blogspot.fr/2012/08/new-and-improved-elements-and.html
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

6. Introduction
Proposed Work
Results and Discussions
Problem Definition
Motivation
Composition Rules
1http://stevemccurry.com/
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

7. Introduction
Proposed Work
Results and Discussions
Problem Definition
Motivation
Aesthetic Measure
1Qualitative results of [1] on the CUHK dataset [2].
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

8. Introduction
Proposed Work
Results and Discussions
Problem Definition
Motivation
General Framework
Problem
Describing correlation between low-level visual primitives and formal
principles used in photographs.
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

9. Introduction
Proposed Work
Results and Discussions
Problem Definition
Motivation
Balance and Symmetry
There are three different types of balance 1
: symmetrical, asymmetrical
and radial.
Objectives
Developing a symmetry measure using high-level descriptors characterizing
“Balance” art-principle.
Embedding this measure to retrieve symmetrical images from large-scale
datasets.
1http://www.school-of-digital-photography.com/2014/01/photography-
composition-tips-balance.html
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

10. Introduction
Proposed Work
Results and Discussions
Related Work
Our Method
Table of Contents
1 Introduction
Problem Definition
Motivation
2 Proposed Work
Related Work
Our Method
3 Results and Discussions
Datasets
Results
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

11. Introduction
Proposed Work
Results and Discussions
Related Work
Our Method
Reflection Symmetry
2 4 6 8
1 3 5 7
1Images from AVA dataset[6].
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

12. Introduction
Proposed Work
Results and Discussions
Related Work
Our Method
Related Work I
The general scheme (Loy and Eklundh 2006 [3]) (feature sparsity)
consists of:
1 Extraction of local-feature points (i.e. Lowe’s SIFT).
2 Matching of pairs of those keypoints based on the similarity of
corresponding mirror feature descriptors.
3 Use of the pairs in Hough-voting space to find the best symmetrical
candidate.
Loy Measure [5] = Strength of maximum symmetry coefficient.
1Images from [3]
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

13. Introduction
Proposed Work
Results and Discussions
Related Work
Our Method
Related Work II
The general idea (Cicconet et al. 2014 [4]) consists of:
1 Extracting a regular set of feature points with local edge amplitude and
orientation.
2 Given any pair of points, a axis candidate is defined within a symmetry
coefficient.
3 Defining a mirror symmetry voting histogram as the sum of the
contribution of all pairs of points for a given axis.
4 Extracting the best candidate representing the maximum over the mirror
symmetry histogram.
Disadvantages:
Lacking neighborhood’s information inside the feature representation.
Depending on the scale parameter of the edge detector
For example: (high texture objects with noisy background)
inferior symmetrical info =⇒ incorrect global symmetry.
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

14. Introduction
Proposed Work
Results and Discussions
Related Work
Our Method
Detection
Problem definition:
Investigating Cicconet’s insufficient edge features [4] within Loy’s scheme
[3] by adding neighboring-pixel information.
Contributions:
1 Introducing a new local edge descriptor.
2 Using multiscale edge extraction exploiting the full resolution image.
3 Solving the orientation discontinuity problem in the voting space.
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

15. Introduction
Proposed Work
Results and Discussions
Related Work
Our Method
Measures
1 OurMax = Strength of maximum symmetry coefficient / Number of
symmetry pairs.
2 OurArea = Area of convex hull of symmetry points / Image area.
3 OurMix = Sum of inverse weighted rank of (1) and (2).
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

16. Introduction
Proposed Work
Results and Discussions
Datasets
Results
Table of Contents
1 Introduction
Problem Definition
Motivation
2 Proposed Work
Related Work
Our Method
3 Results and Discussions
Datasets
Results
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

17. Introduction
Proposed Work
Results and Discussions
Datasets
Results
Dataset I - AVA
From DPChallenge photo contest website, Murray et al. [6] introduces
different annotations1
(aesthetic, semantic and photographic style) to
more than 250,000 images for Aesthetic Visual Analysis “AVA”.
10 relevant images from three challenges of “Symmetry”: photographs
composing symmetrical balance.
90 non-relevant images from a challenges of “Abstract”: photographs
composing symmetrical balance.
1http://www.lucamarchesotti.com/
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

18. Introduction
Proposed Work
Results and Discussions
Datasets
Results
Dataset II - SM
High resolution photo collection (Siemens: ∼ 1000 images) in CIEREC 1
projects (i.e. VIVA-ARTS 2
).
Description: documenting national man-made (industrial and
architectural) environments and life style of local families.
Points of view: wide, close-up shot, frontal, side viewing angles,
horizontal, top, down viewing heights.
5 relevant and 15 non-relevant images describing indoor factory views
(first attempt on this collection; studied by art scientists).
1https://www.univ-st-etienne.fr/fr/cierec.html
2https://viva-arts.univ-st-etienne.fr/
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

19. Introduction
Proposed Work
Results and Discussions
Datasets
Results
Quantitative Results
Evaluation Metrics:
AP(X): Average Precision in X Predictions.
NumRel(X): Number of Relevant in X Predictions.
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

20. Introduction
Proposed Work
Results and Discussions
Datasets
Results
Qualitative Results - SM Top5
Rows: Loy, OurMax, OurArea, OurMix
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

21. Introduction
Proposed Work
Results and Discussions
Conclusion
Summary:
1 A balance measure is firstly introduced in terms of both geometrical
and symmetrical properties inside an image.
2 Superior performance among different photography datasets.
Future work:
1 Further investigation is required to get a stable and generalized
measure.
2 Computation optimization and integration can be achieved within
existing retrieval systems of visual arts.
3 Introducing other measures to represent unexplored art principles
(Vanishing Points → Emphasis, Repetitive Patterns → Pattern and
Rhythm).
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

22. Introduction
Proposed Work
Results and Discussions
References
[1] L. Marchesotti, F. Perronnin, D. Larlus, and G. Csurka, “Assessing the aesthetic
quality of photographs using generic image descriptors,” in Computer Vision
(ICCV), 2011 IEEE International Conference on, pp. 1784–1791, IEEE, 2011.
[2] Y. Ke, X. Tang, and F. Jing, “The design of high-level features for photo quality
assessment,” in Computer Vision and Pattern Recognition, 2006 IEEE Computer
Society Conference on, vol. 1, pp. 419–426, IEEE, 2006.
[3] G. Loy and J.-O. Eklundh, “Detecting symmetry and symmetric constellations of
features,” in Computer Vision–ECCV 2006, pp. 508–521, Springer, 2006.
[4] M. Cicconet, D. Geiger, K. C. Gunsalus, and M. Werman, “Mirror symmetry
histograms for capturing geometric properties in images,” in Computer Vision and
Pattern Recognition (CVPR), 2014 IEEE Conference on, pp. 2981–2986, IEEE,
2014.
[5] S. Zhao, Y. Gao, X. Jiang, H. Yao, T.-S. Chua, and X. Sun, “Exploring
principles-of-art features for image emotion recognition,” in Proceedings of the
ACM International Conference on Multimedia, pp. 47–56, ACM, 2014.
[6] N. Murray, L. Marchesotti, and F. Perronnin, “Ava: A large-scale database for
aesthetic visual analysis,” in Computer Vision and Pattern Recognition (CVPR),
2012 IEEE Conference on, pp. 2408–2415, IEEE, 2012.
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

23. Introduction
Proposed Work
Results and Discussions
Thanks for your attention!
Questions?
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

24. Introduction
Proposed Work
Results and Discussions
Appendix - Loy
1 LoyMax [5] = Strength of maximum symmetry coefficient.
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)

25. Introduction
Proposed Work
Results and Discussions
Appendix - AVA Top5
Rows: Loy, OurMax, OurArea, OurMix
M. ELAWADY, P. COLANTONI, C. DUCOTTET, C. BARAT Image Analysis and Understanding (Hubert Curien Lab, FR)