Successfully reported this slideshow.
Your SlideShare is downloading. ×

Wavelet-based Reflection Symmetry Detection via Textural and Color Histograms

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 27 Ad

Wavelet-based Reflection Symmetry Detection via Textural and Color Histograms

Download to read offline

Conference: ICCV 2017 Workshop: Detecting Symmetry in the Wild, Venice, Italy

Source Code: http://github.com/mawady/ColorSymDetect/

Authors: M. Elawady, C. Ducottet, O. Alata, C. Barat, & P. Colantoni

Affiliation: Universite de Lyon, CNRS, UMR 5516, Laboratoire Hubert Curien, Universite de Saint-Etienne, Jean-Monnet, F-42000 Saint-Etienne, France

Conference: ICCV 2017 Workshop: Detecting Symmetry in the Wild, Venice, Italy

Source Code: http://github.com/mawady/ColorSymDetect/

Authors: M. Elawady, C. Ducottet, O. Alata, C. Barat, & P. Colantoni

Affiliation: Universite de Lyon, CNRS, UMR 5516, Laboratoire Hubert Curien, Universite de Saint-Etienne, Jean-Monnet, F-42000 Saint-Etienne, France

Advertisement
Advertisement

More Related Content

Slideshows for you (18)

Similar to Wavelet-based Reflection Symmetry Detection via Textural and Color Histograms (20)

Advertisement

More from Mohamed Elawady (16)

Advertisement

Wavelet-based Reflection Symmetry Detection via Textural and Color Histograms

  1. 1. Introduction Related Work Methodology Results and Discussion Wavelet-based Reflection Symmetry Detection via Textural and Color Histograms M. Elawady1 , C. Ducottet1 , O. Alata1 , C. Barat1 , P. Colantoni2 1 Universit´e de Lyon, CNRS, UMR 5516, Laboratoire Hubert Curien, Universit´e de Saint-´Etienne, Jean-Monnet, F-42000 Saint-´Etienne, France 2 Universit´e Jean Monnet, CIEREC EA n0 3068, Saint-´Etienne, France ICCV 2017 Workshop: Detecting Symmetry in the Wild UMR • CNRS • 5516 • SAINT-ETIENNE M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 1 / 27
  2. 2. Introduction Related Work Methodology Results and Discussion Table of Contents 1 Introduction Background Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 2 / 27
  3. 3. Introduction Related Work Methodology Results and Discussion Background Problem Definition Table of Contents 1 Introduction Background Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 3 / 27
  4. 4. Introduction Related Work Methodology Results and Discussion Background Problem Definition Bilateral Symmetry 1Image from book: The Photographer’s Eye by Michael Freeman M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 4 / 27
  5. 5. Introduction Related Work Methodology Results and Discussion Background Problem Definition Detection of Global Symmetries Axis Legend: Strong, Weak M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 5 / 27
  6. 6. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods Table of Contents 1 Introduction Background Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 6 / 27
  7. 7. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods Baseline (Loy 2006) and its Successor (Mo 2011) The general scheme (Loy and Eklundh 2006 [1]) consists of: Disadvantages: Depending mainly on the properties of hand-crafted features (i.e. SIFT). For example: (smooth objects with noisy background) little feature points =⇒ lost symmetry. M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 7 / 27
  8. 8. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods First Work (Cic 2014) Instead of SIFT, the general idea (Cicconet et al. 2014 [2]) is extracting a regular set of wavelet segments with local edge amplitude and orientation. Disadvantages: Lacking neighborhood’s information inside the feature representation. Depending on the scale parameter of the edge detector. For example: (high texture objects with noisy background) inferior symmetrical info =⇒ incorrect symmetry. M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 8 / 27
  9. 9. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods State-of-Art (Ela 2016) (Single Symmetry) Investigating Cicconet’s edge features [2] within Loy’s scheme [1] by adding neighboring-pixel information. (1) Mul�scale Edge Segment Extrac�on (2) Triangula�on based on Local Symmetry Weights: • Geometry Edge Orienta�ons (Cic) • Local Texture Histogram (Loy) (3) Vo�ng Space for Peak Detec�on with Handling Orienta�on Discon�nuity. θ ρ 0 π Legend: Groundtruth, Our2016, Loy2006, Mo2011, Cic2014 M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 9 / 27
  10. 10. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Table of Contents 1 Introduction Background Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 10 / 27
  11. 11. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Proposed Idea Contribution: Feature extraction using Log-Gabor filters. Similarity measure based on textural and color image information. (a) Multiscale Edge Segment Extraction (b) Triangulation based on Symmetry Weights θ ρ 0 π (c) Voting Space for Peak Detection M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 11 / 27
  12. 12. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Feature Extraction I: Log-Gabor filter Logarithmic transformation of a Gabor filter in the Fourier domain: ˆG(η, α; s, o) = exp(− (log( η ηs ))2 2(log(ση))2 ) exp(− (α − αo)2 2σ2 α ) (1) Advantages: Better resolution in orientations Avoid over-representation of low frequencies Uniform cover of the Fourier domain Fourier Real Imaginary M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 12 / 27
  13. 13. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Feature Extraction II: Log-Gabor filter The modulus of complex wavelet coefficients Is,o(x, y) are computed on an image I (width W and height H) over multiple scales s ∈ {1, . . . , S} and orientations o ∈ {zπ O , z = 0, . . . , O − 1} as follows: I → GS IGS FT → ˆIGS (2) Is,o(x, y) = |FT−1 (ˆIGS × ˆG)| (3) I J(x, y) = max s,o Is,o(x, y) φ(x, y) = argo max s,o Is,o(x, y) M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 13 / 27
  14. 14. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Similarity Measure I Di : Neighboring cell of pi Ji : Maximum wavelet response inside Di φi : Orientation of Ji ψi : HSV Color of Ji hi : Neighboring textural histogram [3] M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 14 / 27
  15. 15. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Similarity Measure II: Color Local HSV histogram gi of size C with sub-sampling rate (Chu : Csa : Cva ) gi (c) = r∈D∗ i 1Ψc (ψr ), (4) c = (chu , csa , cva ), chu ∈ {0, . . . , Chu − 1}, csa ∈ {0, . . . , Csa − 1}, cva ∈ {0, . . . , Cva − 1}, Ψc = [2chu π Chu , 2(chu +1)π Chu [ × [ csa Csa , csa +1 Csa [ × [ cva Cva , cva +1 Cva [ where D∗ i is the neighborhood window around feature point pi , ψc is a sub-sampled set of HSV space, in terms of three components: hue (hu), saturation (sa) and value (va). l1 normalization is applied to gi (.) for bin-wise histogram comparison. M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 15 / 27
  16. 16. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Symmetry Triangulation I A set of feature pairs (pi , pj ) of size P(P−1) 2 are elected such that i = j, and P is the number of feature points. The candidate axis is parametrized by angle θi,j , and displacement ρi,j and has a symmetry weight ωi,j defined as follows: ωi,j = ω(pi , pj ) = m(i, j) t(i, j) q(i, j) (5) m(i, j) = |τi R(T⊥ ij )τj | (6) t(i, j) = N n=1 min(hi (n), ˜hj (n)) (7) q(i, j) = C c=1 min(gi (c), gj (c)) (8) where τi = [cos(φi ), sin(φi )]T , R(T⊥ ij ) is the reflection matrix with respect to the perpendicular of the line connecting (pi , pj ) [2, 3], and ˜hj is the mirror version of hj histogram. M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 16 / 27
  17. 17. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Symmetry Triangulation II A symmetry histogram H(ρ, θ) is defined as the sum of the symmetry weights of all pairs of feature points such as: H(ρ, θ) = pi ,pj i=j ωi,j δρ−ρi,j δθ−θi,j (9) where δ is the Kronecker delta. The voting histogram H(ρ, θ) is smoothed using a Gaussian kernel to output a proper density representation. A1 A2 B3 B4 B1 B2 Input+GT A1 A2 A2 B1 B2 B3 B4 B3 B4 Voting Space M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 17 / 27
  18. 18. Introduction Related Work Methodology Results and Discussion Table of Contents 1 Introduction Background Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 18 / 27
  19. 19. Introduction Related Work Methodology Results and Discussion Datasets and Evaluation Datasets PSU AVA NY ICCV Year 2010-2013 2016 2017 2017 #Images (Single) 157 253 176 100 #Images/#Symmetries (Multiple) 142/479 -/- 63/188 100/384 Evaluation γ (angle) ζ (distance) CVPR2011 [4] 10◦ 20% × len(GT) CVPR2013 [5] 10◦ 20% × min{len(MT), len(GT)} ICCV2017 3◦ 2.5% × min{W , H} M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 19 / 27
  20. 20. Introduction Related Work Methodology Results and Discussion Quantitative Results I - PR Curves PSU (2010-2013) ICCV (2017) X-axis: Recall, Y-axis: Precision Evaluation Metrics: CVPR2013 [5] M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 20 / 27
  21. 21. Introduction Related Work Methodology Results and Discussion Quantitative Results II - TP Datasets Loy[1] Cic[2] Ela[3] Lg LgC PSU(157) 81 90 97 109 116 AVA(253) 174 124 170 191 197 NY(176) 98 92 109 125 132 ICCV17(100) 52 53 52 70 69 PSUm(142) 69 68 67 74 79 NYm(63) 32 36 36 39 40 ICCV17m(100) 54 39 53 53 56 Evaluation Metrics: CVPR2013 [5] M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 21 / 27
  22. 22. Introduction Related Work Methodology Results and Discussion Qualitative Results I - Single Columns: (1) GT, (2) Loy2006 [1], (3) Ela2016 [3] (4) Our(Lg), and (5) Our(LgC) Top 5 detections: red, yellow, green, blue, and magenta. M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 22 / 27
  23. 23. Introduction Related Work Methodology Results and Discussion Qualitative Results II - Multiple Columns: (1) GT, (2) Loy2006 [1], (3) Ela2016 [3] (4) Our(Lg), and (5) Our(LgC) Top 5 detections: red, yellow, green, blue, and magenta. M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 23 / 27
  24. 24. Introduction Related Work Methodology Results and Discussion Conclusion Summary: 1 A reliable detection framework for global symmetries using Log-Gabor wavelet response. 2 Introduction of symmetric metrics using textural and color information around wavelet features. 3 Superior performance over single and multiple symmetries over all public datasets. Future work: 1 The proposed detection can be improved using a continuous maximal-seeking technique to avoid over-extended axes. 2 Possibility of integration within retrieval systems for artistic photographs and paintings in museums. Source Code: http://github.com/mawady/ColorSymDetect AVA Symmetry Dataset: http://github.com/mawady/AvaSym M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 24 / 27
  25. 25. Introduction Related Work Methodology Results and Discussion Comments about ICCV Reflection Symmetry Typo Error for Loy algorithm in symmetry competition results (ECCV 2006 not ECCV 2004) Test image set is not published public as in competition phase 2 Evaluation metric is not complete (just TPR) and not correct (wrong calculation in angle difference and removing candidate duplicates) The two baseline codes are not published yet on ICCV2017 for validation: Loy2006: using PSU2013 version with same parameters? Atadjanov2016: Not available! Not unified! Two different setups for candidates thresholding for single (¡= 5) and multiple (¡= 10) as stated in ECCV 2016 paper M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 25 / 27
  26. 26. Introduction Related Work Methodology Results and Discussion References I [1] G. Loy and J.-O. Eklundh, “Detecting symmetry and symmetric constellations of features,” in Computer Vision–ECCV 2006, pp. 508–521, Springer, 2006. [2] M. Cicconet, D. Geiger, K. C. Gunsalus, and M. Werman, “Mirror symmetry histograms for capturing geometric properties in images,” in Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pp. 2981–2986, IEEE, 2014. [3] M. Elawady, C. Barat, C. Ducottet, and P. Colantoni, “Global bilateral symmetry detection using multiscale mirror histograms,” in International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 14–24, Springer, 2016. [4] I. Rauschert, K. Brocklehurst, S. Kashyap, J. Liu, and Y. Liu, “First symmetry detection competition: Summary and results,” tech. rep., Technical Report CSE11-012, Department of Computer Science and Engineering, The Pennsylvania State University, 2011. [5] J. Liu, G. Slota, G. Zheng, Z. Wu, M. Park, S. Lee, I. Rauschert, and Y. Liu, “Symmetry detection from realworld images competition 2013: Summary and results,” in Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 IEEE Conference on, pp. 200–205, IEEE, 2013. M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 26 / 27
  27. 27. Introduction Related Work Methodology Results and Discussion Questions? M. Elawady, C. Ducottet, O. Alata, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 27 / 27

×