Successfully reported this slideshow.
Your SlideShare is downloading. ×

Multiple Reflection Symmetry Detection via Linear-Directional Kernel Density Estimation

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 33 Ad

Multiple Reflection Symmetry Detection via Linear-Directional Kernel Density Estimation

Download to read offline

Conference: 17th International Conference on Computer Analysis of Images and Patterns (CAIP), Ystad, Sweden

Authors: M. Elawady, O. Alata, C. Ducottet, C. Barat, & P. Colantoni

Affiliation: Universite de Lyon, CNRS, UMR 5516, Laboratoire Hubert Curien, Universite de Saint-Etienne, Jean-Monnet, F-42000 Saint-Etienne, France

Conference: 17th International Conference on Computer Analysis of Images and Patterns (CAIP), Ystad, Sweden

Authors: M. Elawady, O. Alata, C. Ducottet, C. Barat, & P. Colantoni

Affiliation: Universite de Lyon, CNRS, UMR 5516, Laboratoire Hubert Curien, Universite de Saint-Etienne, Jean-Monnet, F-42000 Saint-Etienne, France

Advertisement
Advertisement

More Related Content

Slideshows for you (19)

Similar to Multiple Reflection Symmetry Detection via Linear-Directional Kernel Density Estimation (20)

Advertisement

More from Mohamed Elawady (16)

Advertisement

Multiple Reflection Symmetry Detection via Linear-Directional Kernel Density Estimation

  1. 1. Introduction Related Work Methodology Results and Discussion Multiple Reflection Symmetry Detection via Linear-Directional Kernel Density Estimation M. Elawady1 , O. Alata1 , C. Ducottet1 , C. Barat1 , P. Colantoni2 1 Universit´e de Lyon, CNRS, UMR 5516, Laboratoire Hubert Curien, Universit´e de Saint-´Etienne, Jean-Monnet, F-42000 Saint-´Etienne, France 2 Universit´e Jean Monnet, CIEREC EA n0 3068, Saint-´Etienne, France 17th International Conference on Computer Analysis of Images and Patterns UMR • CNRS • 5516 • SAINT-ETIENNE M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 1 / 33
  2. 2. Introduction Related Work Methodology Results and Discussion Table of Contents 1 Introduction Background Applications Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 2 / 33
  3. 3. Introduction Related Work Methodology Results and Discussion Background Applications Problem Definition Table of Contents 1 Introduction Background Applications Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 3 / 33
  4. 4. Introduction Related Work Methodology Results and Discussion Background Applications Problem Definition Bilateral Symmetry 1Image from book: The Photographer’s Eye by Michael Freeman M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 4 / 33
  5. 5. Introduction Related Work Methodology Results and Discussion Background Applications Problem Definition Bilateral Symmetry in Computer Vision Medial Image Segmentation [1] Aerial-based vehicle detection [2] M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 5 / 33
  6. 6. Introduction Related Work Methodology Results and Discussion Background Applications Problem Definition Detection of Global Symmetries Axis Legend: Strong, Weak M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 6 / 33
  7. 7. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods Table of Contents 1 Introduction Background Applications Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 7 / 33
  8. 8. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods Baseline (Loy 2006) and its Successor (Mo 2011) The general scheme (Loy and Eklundh 2006 [3]) consists of: Disadvantages: Depending mainly on the properties of hand-crafted features (i.e. SIFT). For example: (smooth objects with noisy background) little feature points =⇒ lost symmetry. (Mo and Draper 2011 [4]) proposed refinements in the general scheme. M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 8 / 33
  9. 9. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods First Work (Cic 2014) Instead of SIFT, the general idea (Cicconet et al. 2014 [5]) is extracting a regular set of wavelet segments with local edge amplitude and orientation. Disadvantages: Lacking neighborhood’s information inside the feature representation. Depending on the scale parameter of the edge detector. For example: (high texture objects with noisy background) inferior symmetrical info =⇒ incorrect symmetry. M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 9 / 33
  10. 10. Introduction Related Work Methodology Results and Discussion Intensity-based Methods Edge-based Methods State-of-Art (Ela 2016) (Single Symmetry) Investigating Cicconet’s edge features [5] within Loy’s scheme [3] by adding neighboring-pixel information. (1) Mul�scale Edge Segment Extrac�on (2) Triangula�on based on Local Symmetry Weights: • Geometry Edge Orienta�ons (Cic) • Local Texture Histogram (Loy) (3) Vo�ng Space for Peak Detec�on with Handling Orienta�on Discon�nuity. θ ρ 0 π Legend: Groundtruth, Our2016, Loy2006, Mo2011, Cic2014 M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 10 / 33
  11. 11. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Table of Contents 1 Introduction Background Applications Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 11 / 33
  12. 12. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Proposed Idea M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 12 / 33
  13. 13. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Symmetry Detection Algorithm: Main Steps Input Image d) Symmetry Selection a) Feature Extraction Scale 1 Scale S MagnitudeOrientationHistogram Point 1 Point 2 Point P c) Kernel Density Estimator b) Feature Triangulation Edge Magnitude Symmetry Coefficient Neighborhood Texture M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 13 / 33
  14. 14. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Multiscale Edge Segment Extraction I Input Image Scale 1 Scale 2 Scale S Orientation 1 Orientation O/2 Orientation O Amplitude Map Orientation Map M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 14 / 33
  15. 15. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Multiscale Edge Segment Extraction II A feature point pi and its local edge characteristics (Ji , φi ) are extracted within each cell using a Morlet wavelet ψk,σ of constant scale σ and varying orientation {φo , o = 1 . . . O}. Amplitude Map Orientation Map Max Amplitude Corresponding Orientation M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 15 / 33
  16. 16. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Multiscale Edge Segment Extraction III Neighboring textural histogram hi of size B: hi (b) = r∈D(pi ) Jr δφb−φr , φb ∈ { bπ B , b = 0 . . . B − 1, 8 ≤ B ≤ 32} (1) where hi is l1 normalized and circular shifted respect to the maximum magnitude Ji among the neighborhood window D(pi ). 0 36 72 108 144 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 #106 Magnitude Histogram 108 144 0 36 72 0 0.1 0.2 0.3 0.4 0.5 0.6 Histogram Count (hi) 0 36 72 108 144 0 500 1000 1500 2000 2500 3000 Frequency Histogram M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 16 / 33
  17. 17. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Symmetry Triangulation I (Textural Information) Symmetry degree of the two regions around i and j can be measured by comparing their corresponding local orientation histograms hi and ˜hj (reverse histogram of hj ). Texture-based symmetry measure is given by: d(i, j) = B b=1 min(hi (b), ˜hj (b)) (2) 108 144 0 36 72 0 0.1 0.2 0.3 0.4 0.5 0.6 Histogram Count (hp) 72 36 0 144 108 0 0.1 0.2 0.3 0.4 0.5 0.6 Histogram Count (hq*) 1 2 3 4 5 0 0.1 0.2 0.3 0.4 0.5 0.6 Histogram Intersection M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 17 / 33
  18. 18. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Symmetry Triangulation II (Edge Information) Semi-dense edge magnitude m(i, j) and mirror symmetry coefficient c(i, j) {similar to cosine distance} are defined as [5]: m(i, j) = Ji Jj (3) c(i, j) = |τi S(T⊥ ij )τj | (4) where τi = [cos(φi ), sin(φi )], and S(.) is a reflection matrix. M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 18 / 33
  19. 19. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Symmetry Triangulation III The candidate axis is parametrized by angle θn, and displacement ρn and has pairwise symmetry weight ωn = ωi,j (i = j and σi = σj ) is defined as: ωn = ω(ˆpi , ˆpj ) = m(i, j) c(i, j) d(i, j) (5) M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 19 / 33
  20. 20. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Kernel Density Estimation I (Linear: Displacement) Linear kernel density estimator fl (.) is defined as fl (x; g) = 1 Ng N n=1 G( x − ρn g ) | G(u) = 1 (2π) 1 2 e− 1 2 |u|2 (6) where G(.) is a Gaussian kernel with bandwidth parameter g. (Directional: Angle) Directional kernel density estimator fd (.) is defined as: fd (y; k) = C(k) N N n=1 L(yT µn; k) | L(x; k) = ekx , C(k) = 1 2πS(0, k) (7) where L(.) is a von-Mises Fisher kernel with concentration parameter k, and normalization constant C(k). S(.) is the modified Bessel function of the first kind. M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 20 / 33
  21. 21. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Kernel Density Estimation II Thanks for the linear-directional density estimator fl,d (.) [6]. We define the extended weighted version ˆfl,d (.) as: ˆfl,d (x, y; g, k) = C(k) Ng N n=1 ωnG( x − ρn g )L(yT µn; k) (8) y = [cos(θ), sin(θ)], µn = [cos(θn), sin(θn)] assuming that linear and directional data are independent resulting dot product between accompanying kernels. B A2 A1 A3 B B A2 A1 A3 M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 21 / 33
  22. 22. Introduction Related Work Methodology Results and Discussion Motivation Algorithm Details Kernel Density Estimation III B A2 A1 A3 Input Image A2 A1 A3 Linear KDE fl (.) [A1,A2,A3] B B Directional KDE fd (.) Edge Magnitude mn(.) Symmetry Coefficient cn(.) Neighborhood Texture dn(.) M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 22 / 33
  23. 23. Introduction Related Work Methodology Results and Discussion Table of Contents 1 Introduction Background Applications Problem Definition 2 Related Work Intensity-based Methods Edge-based Methods 3 Methodology Motivation Algorithm Details 4 Results and Discussion M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 23 / 33
  24. 24. Introduction Related Work Methodology Results and Discussion Evaluation Details Datasets: PSUm dataset: Liu’s vision group proposed a symmetry groundtruth [7, 8] for Flickr images (# images = 142, # symmetries = 479) in ECCV2010, CVPR2011 and CVPR2013. NYm dataset: Cicconet et al. [9] presented a new symmetry database (# images = 63, # symmetries = 188) in 2016. Evaluation Metrics: True Positive [8]: ang(SC, GT) < 10◦ (9) dist(CenSC , CenGT ) < 20% × min(LenSC , LenGT ) (10) Precision, Recall, and Maximum F1 Score M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 24 / 33
  25. 25. Introduction Related Work Methodology Results and Discussion Quantitative Results I (Precision-Recall Curves) 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Loy2006 (0.29211) Cicconet2014 (0.15883) Elawady2016 (0.27744) Our2017 (0.32828) PSUm (2010-2013) 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Loy2006 (0.33657) Cicconet2014 (0.2365) Elawady2016 (0.38788) Our2017 (0.43373) NYm (2016) X-axis: Recall, Y-axis: Precision M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 25 / 33
  26. 26. Introduction Related Work Methodology Results and Discussion Quantitative Results II (Statistical Comparison) Max F1 Score and its equivalent Precision and Recall rates M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 26 / 33
  27. 27. Introduction Related Work Methodology Results and Discussion Qualitative Results I - PSUm Columns: (1) GT, (2) Our, (3) Ela2016 [10], and (4) Loy2006 [3] Top 5 detections: red, yellow, green, blue, and magenta. M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 27 / 33
  28. 28. Introduction Related Work Methodology Results and Discussion Qualitative Results II - NYm Columns: (1) GT, (2) Our, (3) Ela2016 [10], and (4) Loy2006 [3] Top 5 detections: red, yellow, green, blue, and magenta. M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 28 / 33
  29. 29. Introduction Related Work Methodology Results and Discussion Conclusion Summary: 1 A weighted joint density estimator is proposed to handle both orientation and displacement information. 2 A reliable detection framework is developed for global multiple symmetries. Future work: 1 The proposed detection can be improved using a continuous maximal-seeking technique to avoid over-extended axes. 2 Entropy-based balance measure can be introduced to describe the existence and degree of global axes inside an image. 3 Possibility of integration within retrieval systems for artistic photographs and paintings in museums M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 29 / 33
  30. 30. Introduction Related Work Methodology Results and Discussion References I [1] F. Abdolali, R. A. Zoroofi, Y. Otake, and Y. Sato, “Automatic segmentation of maxillofacial cysts in cone beam ct images,” Computers in biology and medicine, vol. 72, pp. 108–119, 2016. [2] S. Ram and J. J. Rodriguez, “Vehicle detection in aerial images using multiscale structure enhancement and symmetry,” in Image Processing (ICIP), 2016 IEEE International Conference on, pp. 3817–3821, IEEE, 2016. [3] G. Loy and J.-O. Eklundh, “Detecting symmetry and symmetric constellations of features,” in Computer Vision–ECCV 2006, pp. 508–521, Springer, 2006. [4] Q. Mo and B. Draper, “Detecting bilateral symmetry with feature mirroring,” in CVPR 2011 Workshop on Symmetry Detection from Real World Images, 2011. [5] M. Cicconet, D. Geiger, K. C. Gunsalus, and M. Werman, “Mirror symmetry histograms for capturing geometric properties in images,” in Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on, pp. 2981–2986, IEEE, 2014. M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 30 / 33
  31. 31. Introduction Related Work Methodology Results and Discussion References II [6] E. Garc´ıa-Portugu´es, R. M. Crujeiras, and W. Gonz´alez-Manteiga, “Kernel density estimation for directional–linear data,” Journal of Multivariate Analysis, vol. 121, pp. 152–175, 2013. [7] I. Rauschert, K. Brocklehurst, S. Kashyap, J. Liu, and Y. Liu, “First symmetry detection competition: Summary and results,” tech. rep., Technical Report CSE11-012, Department of Computer Science and Engineering, The Pennsylvania State University, 2011. [8] J. Liu, G. Slota, G. Zheng, Z. Wu, M. Park, S. Lee, I. Rauschert, and Y. Liu, “Symmetry detection from realworld images competition 2013: Summary and results,” in Computer Vision and Pattern Recognition Workshops (CVPRW), 2013 IEEE Conference on, pp. 200–205, IEEE, 2013. [9] M. Cicconet, V. Birodkar, M. Lund, M. Werman, and D. Geiger, “A convolutional approach to reflection symmetry,” Pattern Recognition Letters, 2017. [10] M. Elawady, C. Barat, C. Ducottet, and P. Colantoni, “Global bilateral symmetry detection using multiscale mirror histograms,” in International Conference on Advanced Concepts for Intelligent Vision Systems, pp. 14–24, Springer, 2016. M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 31 / 33
  32. 32. Introduction Related Work Methodology Results and Discussion Questions? M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 32 / 33
  33. 33. Introduction Related Work Methodology Results and Discussion Appendix - Why Feature Normalization? The feature points are normalized with keeping aspect ratio as following: ˆpi = pi − cW ,H max(W , H) (11) where cW ,H represents the original image center (W 2 , H 2 ). Without Normalization With Normalization Voting: unified space and independent parameters M. Elawady, O. Alata, C. Ducottet, C. Barat, P. Colantoni Hubert Curien Laboratory, Jean Monnet University, France 33 / 33

×