Successfully reported this slideshow.
Your SlideShare is downloading. ×

Global Bilateral Symmetry Detection Using Multiscale Mirror Histograms

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 31 Ad

More Related Content

Slideshows for you (18)

Advertisement

Similar to Global Bilateral Symmetry Detection Using Multiscale Mirror Histograms (20)

More from Mohamed Elawady (16)

Advertisement

Recently uploaded (20)

Global Bilateral Symmetry Detection Using Multiscale Mirror Histograms

  1. 1. Introduction Related Work Methodology Results and Discussion Global Bilateral Symmetry Detection Using Multiscale Mirror Histograms M. ELAWADY1 , C. BARAT1 , C. DUCOTTET1 and P. COLANTONI2 1 Universit´e de Lyon, CNRS, UMR 5516, Laboratoire Hubert Curien, Universit´e de Saint-´Etienne, Jean-Monnet, F-42000 Saint-´Etienne, France 2 Universit´e Jean Monnet, CIEREC EA n0 3068, Saint-´Etienne, France ACIVS Conference, October 2016 UMR • CNRS • 5516 • SAINT-ETIENNE M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 1 / 31
  2. 2. Introduction Related Work Methodology Results and Discussion Table of Contents 1 Introduction Background Applications Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 2 / 31
  3. 3. Introduction Related Work Methodology Results and Discussion Background Applications Problem Definition Table of Contents 1 Introduction Background Applications Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 3 / 31
  4. 4. Introduction Related Work Methodology Results and Discussion Background Applications Problem Definition Bilateral Symmetry 1Image from book: The Photographer’s Eye by Michael Freeman M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 4 / 31
  5. 5. Introduction Related Work Methodology Results and Discussion Background Applications Problem Definition Bilateral Symmetry in Computer Vision I Medial Image Compression [1] Depth Estimation [2] M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 5 / 31
  6. 6. Introduction Related Work Methodology Results and Discussion Background Applications Problem Definition Bilateral Symmetry in Computer Vision II Object Segmentation [3] Aesthetic Analysis [4] M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 6 / 31
  7. 7. Introduction Related Work Methodology Results and Discussion Background Applications Problem Definition Detection of Main Symmetry Axis Axis Legend: Strong, Weak M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 7 / 31
  8. 8. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods Table of Contents 1 Introduction Background Applications Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 8 / 31
  9. 9. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods Baseline and its Successors I The general scheme (Loy and Eklundh 2006 [5]) consists of: Example: 1Second figure from [5] M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 9 / 31
  10. 10. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods Baseline and its Successors II Disadvantages: Depending mainly on the properties of hand-crafted features (i.e. SIFT). For example: (smooth objects with noisy background) little feature points =⇒ lost symmetry. (Mo and Draper 2011 [6]) proposed refinements in the general scheme in: 1 Selecting all symmetry candidate pairs instead of finding closest matches for each point. 2 Using less complex hough voting scheme. M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 10 / 31
  11. 11. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods State of Art Instead of SIFT, the general idea (Cicconet et al. 2014 [7]) is extracting a regular set of wavelet segments with local edge amplitude and orientation. Disadvantages: Lacking neighborhood’s information inside the feature representation. Depending on the scale parameter of the edge detector. For example: (high texture objects with noisy background) inferior symmetrical info =⇒ incorrect symmetry. M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 11 / 31
  12. 12. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Table of Contents 1 Introduction Background Applications Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 12 / 31
  13. 13. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Proposed Idea Investigating Cicconet’s edge features [7] within Loy’s scheme [5] by adding neighboring-pixel information. Contributions: 1 Introducing a new local edge descriptor. 2 Using multiscale edge extraction exploiting the full resolution image. 3 Solving the orientation discontinuity problem in the voting space. 4 Introducing a symmetry dataset based on aesthetic analysis. M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 13 / 31
  14. 14. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Symmetry Detection Algorithm Main Steps: (1) Mul�scale Edge Segment Extrac�on (2) Triangula�on based on Local Symmetry Weights: • Geometry Edge Orienta�ons (Cic) • Local Texture Histogram (Loy) (3) Vo�ng Space for Peak Detec�on with Handling Orienta�on Discon�nuity. θ ρ 0 π M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 14 / 31
  15. 15. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Multiscale Edge Segment Extraction I A feature point p and its local edge characteristics (Jp, τp) are extracted within each cell using a Morlet wavelet ψk,σ of constant scale σ and varying orientation {τk , k = 1 . . . n}. M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 15 / 31
  16. 16. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Multiscale Edge Segment Extraction II Jk (p) denote the modulus of wavelet coefficients at point p, in which local edge characteristics Jp and τp are obtained by seeking the maximum wavelet response and orientation over all orientations. M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 16 / 31
  17. 17. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Multiscale Edge Segment Extraction III Histogram count at a given orientation τk is: hp(k) = r∈N(p) Jr δφk −φr (1) where φk and φr are angles associated with τk and τr , and δx is the Kronecker delta. hp is subsequently 1 normalized and circular shifted so as the first bin corresponds to τp. 0 36 72 108 144 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 #106 Magnitude Histogram 108 144 0 36 72 0 0.1 0.2 0.3 0.4 0.5 0.6 Histogram Count (hp) 0 36 72 108 144 0 500 1000 1500 2000 2500 3000 Frequency Histogram M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 17 / 31
  18. 18. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Multiscale Edge Segment Extraction IV In most images, relevant information about the visual content may appear at different scales. Feature points are computed with respect to a set of regular grids at different scales and a corresponding set of wavelet scales {σl , l = 1..m}. M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 18 / 31
  19. 19. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Triangulation: Local Texture Histogram (Textural Information) Symmetry degree of the two regions around p and q can be measured by comparing their corresponding local orientation histogram hp and hq. Texture-based symmetry measure is given by: dI (hp, h∗ q) = n k=1 min(hp(k), h∗ q(k)) (2) 108 144 0 36 72 0 0.1 0.2 0.3 0.4 0.5 0.6 Histogram Count (hp) 72 36 0 144 108 0 0.1 0.2 0.3 0.4 0.5 0.6 Histogram Count (hq*) 1 2 3 4 5 0 0.1 0.2 0.3 0.4 0.5 0.6 Histogram Intersection M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 19 / 31
  20. 20. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Triangulation: Geometry Edge Orientation (Edge Information) Pairwise symmetry coefficient f (p, q) is defined as [7]: f (p, q) = |τqS(T⊥ pq)τp| (3) M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 20 / 31
  21. 21. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Triangulation: Symmetry Weights Given a pair of feature points (p, q), the candidate axis T⊥ pq perpendicular to (pq) is parametrized by the orientation of its normal θpq and its distance to the origin ρpq. Mirror symmetry histogram HS (ρ, θ) is defined as the sum of the contribution of all pairs of feature points such as: H(cx , cy , θ) = p,q p=q JpJqf (p, q)dI (hp, hq)δ(cx ,cy )− p+q 2 δθ−θpq (4) HS (ρ, θ) = cx ,cy H(cx , cy , θ)δρ−ρpq (5) M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 21 / 31
  22. 22. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Voting Space and Peak Detection A1 A2 A3 A4 A5 B A1 A2 A3 A4 A5 B B A1 A2 A3 A4 A5 M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 22 / 31
  23. 23. Introduction Related Work Methodology Results and Discussion Table of Contents 1 Introduction Background Applications Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 23 / 31
  24. 24. Introduction Related Work Methodology Results and Discussion Algorithm Evaluation From Real-World Images Competition CVPR 2013 [10], a symmetry detection is correct if: (1) θ < 15◦ and (2) d < 0.2 ∗ min(lenGT , lenR ). R GT d θ M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 24 / 31
  25. 25. Introduction Related Work Methodology Results and Discussion Quantitative Results$FFXUDFuD(t M CM iM cM OM uM rM M M M CMM 'DWDVHWV 368iMCM 368iMCC 368iMCc $9$iMCr /RiMMr 0RiMCC LFiMCO 2XUiMCr M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 25 / 31
  26. 26. Introduction Related Work Methodology Results and Discussion Qualitative Results on PSU Datasets (http://www.flickr.com/), around 200 images from PSU symmetry detection challenges [9, 10] in ECCV2010, CVPR2011 and CVPR2013. Legend: Groundtruth, Our2016, Loy2006, Mo2011, Cic2014 M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 26 / 31
  27. 27. Introduction Related Work Methodology Results and Discussion Qualitative Results on AVA Dataset (http://www.dpchallenge.com/), around 250 images from Aesthetic Visual Analysis “AVA” [8] with our global-axis symmetry groundtruth. Legend: Groundtruth, Our2016, Loy2006, Mo2011, Cic2014 M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 27 / 31
  28. 28. Introduction Related Work Methodology Results and Discussion Conclusion Summary: 1 A reliable global symmetry detection is developed among variants of visual cues. 2 A groundtruth of global symmetry axis is introduced and extracted from large scale Aesthetic Visual Analysis (AVA) dataset. Future work: 1 Real-world images is required to handle with large degrees of perspective view. 2 The proposed detection can be improved to avoid over-extended axes. 3 A stable balance measure can be introduced to describe the existence and degree of global axes inside an image. 4 Possibility of integration within retrieval systems of visual arts. M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 28 / 31
  29. 29. Introduction Related Work Methodology Results and Discussion References I [1] V. Bairagi, “Symmetry-based biomedical image compression,” Journal of digital imaging, pp. 1–9, 2015. [2] L. Yang, J. Liu, and X. Tang, “Depth from water reflection,” Image Processing, IEEE Transactions on, vol. 24, no. 4, pp. 1235–1243, 2015. [3] C. L. Teo, C. Fermuller, and Y. Aloimonos, “Detection and segmentation of 2d curved reflection symmetric structures,” in Proceedings of the IEEE International Conference on Computer Vision, pp. 1644–1652, 2015. [4] S. Zhao, Y. Gao, X. Jiang, H. Yao, T.-S. Chua, and X. Sun, “Exploring principles-of-art features for image emotion recognition,” in Proceedings of the ACM International Conference on Multimedia, pp. 47–56, ACM, 2014. [5] G. Loy and J.-O. Eklundh, “Detecting symmetry and symmetric constellations of features,” in Computer Vision–ECCV 2006, pp. 508–521, Springer, 2006. [6] Q. Mo and B. Draper, “Detecting bilateral symmetry with feature mirroring,” in CVPR 2011 Workshop on Symmetry Detection from Real World Images, 2011. M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 29 / 31
  30. 30. Introduction Related Work Methodology Results and Discussion References II [7] M. Cicconet, D. Geiger, K. C. Gunsalus, and M. Werman, “Mirror symmetry histograms for capturing geometric properties in images,” in Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pp. 2981–2986, IEEE, 2014. [8] N. Murray, L. Marchesotti, and F. Perronnin, “Ava: A large-scale database for aesthetic visual analysis,” in Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp. 2408–2415, IEEE, 2012. [9] I. Rauschert, K. Brocklehurst, S. Kashyap, J. Liu, and Y. Liu, “First symmetry detection competition: Summary and results,” tech. rep., Technical Report CSE11-012, Department of Computer Science and Engineering, The Pennsylvania State University, 2011. [10] J. Liu, G. Slota, G. Zheng, Z. Wu, M. Park, S. Lee, I. Rauschert, and Y. Liu, “Symmetry detection from realworld images competition 2013: Summary and results,” in Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 IEEE Conference on, pp. 200–205, IEEE, 2013. M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 30 / 31
  31. 31. Introduction Related Work Methodology Results and Discussion Questions? M. ELAWADY, C. BARAT, C. DUCOTTET and P. COLANTONI Image Analysis and Understanding (Hubert Curien Lab, FR) 31 / 31

×