A Diffusion Wavelet Approach for 3-D Model Matching<br />Authors: K.P. Zhu, Y.S. Wong, W.F. Lu, J.Y.H. Fuh<br />Presented ...
Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)...
Introduction<br />Currently - A larger than ever number of 3D Models in CAD, computer games, multimedia, molecular biology...
Introduction (2)<br />Tagging are not always available or sufficient to describe the model we require<br />Combine topolog...
Model Reusability (CAD/Animation)<br />Model Matching<br />Video Retrieval (2.5D/Virtual environments)<br />Ecommerce<br /...
Obstacles in Retrieval<br />Partial retrieval - Non-transitive<br />Functional description<br />How to match text tags wit...
7<br />
3D Model Matching – Prior Art<br />Feature vectors using wavelets to mesh vertices – localized in both space & frequency –...
Spherical harmonics (SH)<br />Global method in Euclidean space<br />lacks multi-scale analysis <br />Legendre polynomials ...
Spherical Wavelets (SW)<br />Multi-scale in Euclidean space <br />Lacks connectivity on the manifold<br />Tannenbaum et. a...
Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)...
Diffusion Maps Introduction<br />Originally suggested by Stephan Lafon and  R.R. Coifman from Yale Math, circa 2005<br />M...
Manifold Learning algorithms<br />13<br />MANI - Manifold learning Matlab tool<br />
Diffusion Maps<br />Z<br />Y<br />X<br />Coifman - 2005<br /><ul><li> vi is a feature vector
 Contains descriptive information about the 3D model</li></ul>14<br />
Diffusion Maps<br />Assumptions<br /><ul><li> Points are sampled uniformly on the manifold
 Smooth manifold (no fractals in our case)
 Fixed boundary conditions
 Enough points = feature vectors (N→∞)</li></ul>15<br />
Use RBF Gaussian Kernel to choose ε<br />Normalize W to create a Stochastic Matrix<br />Diffusion Maps Algorithm<br />16<b...
Diffusion Maps algorithm (2)<br />Diffuse by taking higher powers of t <br />	“The diffusion distance is equal to the Eucl...
Diffusion Maps Code Example<br />function checker();<br />close all;<br />tetha=2*pi*rand(1,500);<br />z=[cos(tetha);sin(t...
Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)...
Problems with Mesh Simplification<br />20<br />
Wavelets<br />21<br />
Novelty – Diffusion Wavelets<br />Combination of Diffusion Maps and Wavelets<br />Used for non-linear dimensionality reduc...
Diffusion Wavelets Intuition<br />23<br />
Example of Diffusion Wavelets<br />24<br />Wavelet basis ψ(2,2,3)<br />Scaling basis φ(1,1,1)<br />Wavelet basis ψ(4,2,5)<...
Diffusion Wavelets<br />Use an optimization scheme to construct the scaling functions<br />Each scaling function should de...
Diffusion Wavelets (2)<br />Better than LOD (Level of Detail - simplifies meshes)<br />Involved algorithm – very few imple...
Wavelet decomposition example<br />27<br />
Wavelet coefficients<br />28<br />Scale 1<br />Scale 2<br />Scale 3<br />Scale 4<br />
Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)...
Finding Shape Feature Vectors (X)<br />30<br /><ul><li> Take 1,450 coefficients (wavelet + scaling) at each decomposition ...
Increase wavelet coefficient number from 0 to 450 and decreasescaling coefficient number from 1,450 to 1,000</li></li></ul...
Fisher’s Discriminant Ratio<br />32<br />
IRPR Curve<br />Measure performance – use Princeton University 3D database<br />IRPR – Information Retrieval Precision-Rec...
IRPR Curve<br />m = relevant matches <br />r =  # of retrieved models<br />1) Precision = <br />	2) Recall =<br />34<br />
Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)...
3D Model Retrieval Procedure<br />Compute the diffusionwavelet for each 3D model<br />Obtain the model representing vector...
1) Start with a coarsest scale comparison <br />2)Advance up to the finest scale <br />3) Stop on threshold or when finest...
Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)...
Experimental Results<br />39<br />Differences in scaling levels<br />DW gives better results than SH and SW<br />
Visual Results<br />40<br />
Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)...
Authors’ Conclusions<br />Surfaces with sharp peaks, grooves or holes contain high-frequency information which is not addr...
My Conclusions<br />Paper presents a novel solution<br /><ul><li>Diffusion Wavelets was never used before for 3D Retrieval
Less novel solutions:
IRPR is a common measure in database retrieval
Fischer Discriminant Ratio is a common statistical measure</li></ul>43<br />
My Conclusions (2)<br />Technically sound, feasible<br /><ul><li>Taking available code it seems possible to reconstruct th...
Seems like a reasonable solution to the problem of 3D object retrieval</li></ul>44<br />
My Conclusions (3)<br />The diffusion wavelet part could be explained in more detail<br /><ul><li>Not clear in which way t...
How are the wavelet functions affected when a new model is inserted?</li></ul>45<br />
My Conclusions (4)<br />Not self-containing but reference papers are exceptionally good<br /><ul><li>Missing explanations ...
“Would like to have” (Technical/1)<br /> Non-rigid extensions<br /><ul><li>How would retrieval change if we know the 3D mo...
Can we have an extension of Diffusion Wavelets for non-rigid manifolds?</li></ul>47<br />
“Would like to have” (Technical/2) <br />	How to automatically choose the level of decomposition<br />48<br />
“Would like to have” (Technical/3)<br />	An intuitive explanation - why prefer Diffusion Wavelets over Diffusion Wavelet P...
Upcoming SlideShare
Loading in...5
×

A Diffusion Wavelet Approach For 3 D Model Matching

1,679

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,679
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
41
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Google has their own format –SketchUp (SU) and has been investing a lot of effort in 3D technologies. More examples from google include – google earth, google sketch-up and O3D platform for browser 3D display
  • Other manifold learning techniques – PCA (+variants), Kernel PCA, LLE (locally linear embedding), ISOMAP, Hessian LLE, LaplacianEigenMaps and many more…
  • Reference[9]
  • Reference [2]
  • See [1,6] for diffusion maps equations and [3] for graph
  • Ignore λmax=1 since it is a trivial eigenvalue of the markov transition matrix and corresponds to the stationary distribution of the markov chain at t=∞Cut is promised to conform with min-cut max-flow algorithmSee reference [6] for a thorough description of diffusion maps
  • We see that it is possible to reconstruct the manifold using just a single eigenvalue. Cuts can be made on the eigenvectors that represent the manifold (in this case, the second or the third eigenvectors corresponding to the second or third largest eigenvalue) – these cuts are meaningful since they are taking into account the geometric distribution of the original points. We see that diffusion maps approximate the Fourier series over the circle as the sine and cosine functions are the solution of the differential equation f’’=-f
  • Animated objects can be more sensitive to mesh simplification algorithms than CAD models.
  • Haar wavelet with 3 levels of decomposition to the Stanford Bunny image. By applying a threshold in the wavelet domain we can efficiently find similar images. The threshold is a very efficient way to remove noise. The high value coefficients correspond to edges at various scales. Collecting the high value coefficients to create a feature vector would ensure a good representation of the image. For example, check the lossy compression algorithm JPEG-2000.
  • We take 4 levels of decomposition since there is no real advantage in taking more decomposition levels and the computational burden is heavy. This result is specific to Princeton’s database and can change when dealing with different databases.
  • Take inverse of FDR to avoid numeric problems of overflow. Select model with minimum within cluster scattering and maximum within cluster scattering
  • Can do training on the entire database…See [5] for database
  • Chair – sparse structurePlane – smooth surface with local singularityKangaroo – smooth surfaceFlower – combines smooth surface with local singularity
  • A Diffusion Wavelet Approach For 3 D Model Matching

    1. 1. A Diffusion Wavelet Approach for 3-D Model Matching<br />Authors: K.P. Zhu, Y.S. Wong, W.F. Lu, J.Y.H. Fuh<br />Presented by: Raphael Steinberg<br />
    2. 2. Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)<br />Retrieval Procedure<br />Results<br />Conclusions<br />2<br />
    3. 3. Introduction<br />Currently - A larger than ever number of 3D Models in CAD, computer games, multimedia, molecular biology, computer vision and more<br />There is a need for 3D Retrieval <br />3<br />
    4. 4. Introduction (2)<br />Tagging are not always available or sufficient to describe the model we require<br />Combine topological information with multi-scale properties<br />4<br />
    5. 5. Model Reusability (CAD/Animation)<br />Model Matching<br />Video Retrieval (2.5D/Virtual environments)<br />Ecommerce<br />Correcting defects<br />Efficient Representation<br />Many other uses…<br />Motivation for 3D Retrieval<br />5<br />
    6. 6. Obstacles in Retrieval<br />Partial retrieval - Non-transitive<br />Functional description<br />How to match text tags with vertices and texture?<br />Orthonormal coordinate system<br />6<br />
    7. 7. 7<br />
    8. 8. 3D Model Matching – Prior Art<br />Feature vectors using wavelets to mesh vertices – localized in both space & frequency – <br />Paquet et. al. 2000<br />Random sampling for comparison – Osada et. al. 2001<br />8<br />
    9. 9. Spherical harmonics (SH)<br />Global method in Euclidean space<br />lacks multi-scale analysis <br />Legendre polynomials solve the Laplace equation in Spherical coordinates<br />Vranic et. al. 2001<br />9<br />
    10. 10. Spherical Wavelets (SW)<br />Multi-scale in Euclidean space <br />Lacks connectivity on the manifold<br />Tannenbaum et. al. 2007<br />10<br />
    11. 11. Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)<br />Retrieval Procedure<br />Results<br />Conclusions<br />11<br />
    12. 12. Diffusion Maps Introduction<br />Originally suggested by Stephan Lafon and R.R. Coifman from Yale Math, circa 2005<br />Many other manifold learning techniques exist<br />Data analysis based on geometric properties of the data set<br />12<br />
    13. 13. Manifold Learning algorithms<br />13<br />MANI - Manifold learning Matlab tool<br />
    14. 14. Diffusion Maps<br />Z<br />Y<br />X<br />Coifman - 2005<br /><ul><li> vi is a feature vector
    15. 15. Contains descriptive information about the 3D model</li></ul>14<br />
    16. 16. Diffusion Maps<br />Assumptions<br /><ul><li> Points are sampled uniformly on the manifold
    17. 17. Smooth manifold (no fractals in our case)
    18. 18. Fixed boundary conditions
    19. 19. Enough points = feature vectors (N→∞)</li></ul>15<br />
    20. 20. Use RBF Gaussian Kernel to choose ε<br />Normalize W to create a Stochastic Matrix<br />Diffusion Maps Algorithm<br />16<br />Lu et. al. 2009<br />
    21. 21. Diffusion Maps algorithm (2)<br />Diffuse by taking higher powers of t <br /> “The diffusion distance is equal to the Euclidean distance in the diffusion map space” , Nadler et. al. 2005<br />Cut manifold according to dominant eigenvalues<br />17<br />
    22. 22. Diffusion Maps Code Example<br />function checker();<br />close all;<br />tetha=2*pi*rand(1,500);<br />z=[cos(tetha);sin(tetha)];<br />figure(1);scatter(z(1,:),z(2,:),&apos;b*&apos;);hold on;<br />N=size(z,2);<br />epsilon=linspace(0.01,.3,10);<br />%epsilon=.3;<br />W=nan(N);<br />summer=nan(1,length(epsilon));<br />for k=1:length(epsilon)<br /> for i=1:N<br /> parfor j=1:N<br /> W(i,j)=exp(-sum((z(:,j)-z(:,i)).^2)/2/epsilon(k));<br /> end<br /> end<br /> summer(k)=sum(sum(W));<br />end<br />figure;scatter(log(epsilon),log(summer));title(&apos;Epsilon - linear region&apos;)<br />p=polyfit(log(epsilon),log(summer),1);<br />d=2*p(1);%manifold dimension<br />M=W*diag(1./sum(W,2));<br />[U V]=svds(M);<br />sync=max(U(:,2));<br />figure(1);scatter(U(:,2)./sync,U(:,3)./sync,&apos;rd&apos;)<br />title(&apos;Original manifold as stars and reconstructed manifold as diamonds&apos;)<br />end<br />18<br />
    23. 23. Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)<br />Retrieval Procedure<br />Results<br />Conclusions<br />19<br />
    24. 24. Problems with Mesh Simplification<br />20<br />
    25. 25. Wavelets<br />21<br />
    26. 26. Novelty – Diffusion Wavelets<br />Combination of Diffusion Maps and Wavelets<br />Used for non-linear dimensionality reduction<br />Extension of wavelets to the unit circle (just as diffusion maps extends the Fourier transform)<br />22<br />
    27. 27. Diffusion Wavelets Intuition<br />23<br />
    28. 28. Example of Diffusion Wavelets<br />24<br />Wavelet basis ψ(2,2,3)<br />Scaling basis φ(1,1,1)<br />Wavelet basis ψ(4,2,5)<br />Wavelet<br />basis ψ(3,2,3)<br />
    29. 29. Diffusion Wavelets<br />Use an optimization scheme to construct the scaling functions<br />Each scaling function should deal with a single dimension and be orthogonal to the other scaling functions<br />Extension of wavelets to the sphere (or to any other manifold)<br />25<br />
    30. 30. Diffusion Wavelets (2)<br />Better than LOD (Level of Detail - simplifies meshes)<br />Involved algorithm – very few implementations exist<br />26<br />
    31. 31. Wavelet decomposition example<br />27<br />
    32. 32. Wavelet coefficients<br />28<br />Scale 1<br />Scale 2<br />Scale 3<br />Scale 4<br />
    33. 33. Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)<br />Retrieval Procedure<br />Results<br />Conclusions<br />29<br />
    34. 34. Finding Shape Feature Vectors (X)<br />30<br /><ul><li> Take 1,450 coefficients (wavelet + scaling) at each decomposition level
    35. 35. Increase wavelet coefficient number from 0 to 450 and decreasescaling coefficient number from 1,450 to 1,000</li></li></ul><li>Fisher’s Discriminant Ratio<br />b – between classes<br />w – within class (after wavelet decomposition)<br /> j – scale <br />31<br />
    36. 36. Fisher’s Discriminant Ratio<br />32<br />
    37. 37. IRPR Curve<br />Measure performance – use Princeton University 3D database<br />IRPR – Information Retrieval Precision-Recall<br />33<br />
    38. 38. IRPR Curve<br />m = relevant matches <br />r = # of retrieved models<br />1) Precision = <br /> 2) Recall =<br />34<br />
    39. 39. Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)<br />Retrieval Procedure<br />Results<br />Conclusions<br />35<br />
    40. 40. 3D Model Retrieval Procedure<br />Compute the diffusionwavelet for each 3D model<br />Obtain the model representing vector X<br />Compute the 2nd order statistics of X for each scale<br />36<br />
    41. 41. 1) Start with a coarsest scale comparison <br />2)Advance up to the finest scale <br />3) Stop on threshold or when finest scale reached<br />* Use a threshold to determine if a model is from a certain class<br />Model Matching Procedure<br />37<br />
    42. 42. Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)<br />Retrieval Procedure<br />Results<br />Conclusions<br />38<br />
    43. 43. Experimental Results<br />39<br />Differences in scaling levels<br />DW gives better results than SH and SW<br />
    44. 44. Visual Results<br />40<br />
    45. 45. Schedule<br />Introduction<br />Diffusion Maps<br />Wavelets and Diffusion Wavelets<br />Fisher’s Discriminant Ratio (FDR)<br />Retrieval Procedure<br />Results<br />Conclusions<br />41<br />
    46. 46. Authors’ Conclusions<br />Surfaces with sharp peaks, grooves or holes contain high-frequency information which is not addressed by the wavelet multi-resolution (use diffusion wavelet packets instead?)<br />Possible to extend to partial matching<br />DW presents better results than SH and SW<br />42<br />
    47. 47. My Conclusions<br />Paper presents a novel solution<br /><ul><li>Diffusion Wavelets was never used before for 3D Retrieval
    48. 48. Less novel solutions:
    49. 49. IRPR is a common measure in database retrieval
    50. 50. Fischer Discriminant Ratio is a common statistical measure</li></ul>43<br />
    51. 51. My Conclusions (2)<br />Technically sound, feasible<br /><ul><li>Taking available code it seems possible to reconstruct the results
    52. 52. Seems like a reasonable solution to the problem of 3D object retrieval</li></ul>44<br />
    53. 53. My Conclusions (3)<br />The diffusion wavelet part could be explained in more detail<br /><ul><li>Not clear in which way the wavelets are constructed
    54. 54. How are the wavelet functions affected when a new model is inserted?</li></ul>45<br />
    55. 55. My Conclusions (4)<br />Not self-containing but reference papers are exceptionally good<br /><ul><li>Missing explanations about diffusion wavelets</li></ul>46<br />
    56. 56. “Would like to have” (Technical/1)<br /> Non-rigid extensions<br /><ul><li>How would retrieval change if we know the 3D model is non-rigid?
    57. 57. Can we have an extension of Diffusion Wavelets for non-rigid manifolds?</li></ul>47<br />
    58. 58. “Would like to have” (Technical/2) <br /> How to automatically choose the level of decomposition<br />48<br />
    59. 59. “Would like to have” (Technical/3)<br /> An intuitive explanation - why prefer Diffusion Wavelets over Diffusion Wavelet Packets?<br />Wavelet Packets seem to give more information especially in high frequencies…<br />49<br />
    60. 60. “Would like to have” (Technical/4)<br /> Numerical problems of overflow of the FDR - use logarithm instead of inverse?<br />50<br />
    61. 61. “Would like to have” (Presentation/1)<br /> Block diagram of the algorithm<br />51<br />
    62. 62. “Would like to have” (Presentation/2)<br /> Web-based Graphical User Interface<br />52<br />
    63. 63. “Would like to have” (Presentation/3)<br /> Error analysis<br />53<br />
    64. 64. “Would like to have” (Presentation/4)<br /> More explanations on Diffusion Wavelets<br />54<br />
    65. 65. Conclusions<br />Shape retrieval requires multi-scale analysis<br />3D models, like most real-life objects, are embedded in a low dimension manifold<br />Results are robust to noise and to mesh simplifications<br />55<br />
    66. 66. Conclusions (2)<br />Diffusion Wavelets give good retrieval results for 3D objects<br />Possible to extend the proposed method to include texture, sound, smell, elasticity and any other possibly given attribute of the 3D model<br />56<br />
    67. 67. THE END<br />57<br />
    68. 68. References<br />[1]K.P. Zhu, Y.S. Wong, W.F. Lu, J.Y.H. Fuh. , Department of Mechanical Engineering, National University of Singapore “A diffusion wavelet approach for 3-D model matching” Computer Aided Design, Elsevier, Nov. 2008<br />[2] Presentation by R.R. Coifman et. al.<br />[3] J. Lu et. al. “Dominant Texture and Diffusion Distance Manifolds“, Eurographics, Volume 28 ,<br />Issue 2, Pages 667 - 676, Mar. 2009<br />[4] Diffusion waveletsMatlab code:<br />http://www.math.duke.edu/~mauro/diffusionwavelets.html#Code|outline<br />[5] The Princeton Shape Benchmark:<br />http://shape.cs.princeton.edu/benchmark/<br />[6] Nadler, B., Lafon, S., Coifman, R., Kevrekidis, I. “Diffusion maps, spectral clustering and eigenfunctions of Fokker-Planck operators”. <br />[7] Ulrike von Luxburg, “A tutorial on spectral clustering”. Statistical Journal 2007<br />[8] Personal communications with K.P. Zhu<br />[9] MANI - Manifold learning Matlab tool<br />http://www.math.umn.edu/~wittman/mani/<br />[10] Vranic D, Saupe D, Richter J. Tools for 3D-object retrieval: Karhunen-Loeve transform and spherical harmonics. In: Proc. IEEE workshop on multimedia signal processing; 2001. p. 29398.<br />58<br />
    69. 69. References<br />[11] Osada R, Funkhouser T, Chazelle B, Dobkin D. Matching 3D models with shape distributions, In: Proc. shape modeling international. 2001. p. 15466.<br />[12] Laga H, Nakajima M. Statistical spherical wavelet moments for content-based 3D model Retrieval. In: Computer graphics international 2007, CGI. 2007; 2007. p.1-8.<br />[13] Nain D, Haker S, Bobick A, Tannenbaum A. Multiscale 3-D shape representation and segmentation using spherical wavelets. IEEE Transactions on Medical Imaging 2007;26(4), pages 598-618.<br />59<br />
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×