Introduction   Acquisition   Feature Estimation   F/PFH   Surface Classification   Object Recognition   Registration   Conc...
[Introduction] Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion      ...
[Introduction] Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion      ...
[Introduction] Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion      ...
[Introduction] Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction [Acquisition] Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction [Acquisition] Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction [Acquisition] Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction [Acquisition] Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion      ...
Introduction   Acquisition   Feature Estimation   [F/PFH]   Surface Classification   Object Recognition    Registration   C...
Introduction   Acquisition   Feature Estimation   [F/PFH]   Surface Classification   Object Recognition    Registration   C...
Introduction   Acquisition   Feature Estimation   [F/PFH]   Surface Classification   Object Recognition    Registration   C...
Introduction   Acquisition   Feature Estimation   [F/PFH]   Surface Classification   Object Recognition    Registration   C...
Introduction   Acquisition   Feature Estimation   [F/PFH]   Surface Classification   Object Recognition    Registration   C...
Introduction   Acquisition   Feature Estimation   [F/PFH]   Surface Classification   Object Recognition    Registration   C...
Introduction   Acquisition   Feature Estimation   [F/PFH]   Surface Classification   Object Recognition    Registration   C...
Introduction   Acquisition   Feature Estimation   [F/PFH]   Surface Classification   Object Recognition    Registration   C...
Introduction    Acquisition   Feature Estimation   [F/PFH]   Surface Classification   Object Recognition    Registration   ...
Introduction   Acquisition   Feature Estimation   [F/PFH]   Surface Classification   Object Recognition    Registration   C...
Introduction   Acquisition   Feature Estimation   [F/PFH]   Surface Classification   Object Recognition    Registration   C...
Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH Surface Classification [Object Recognition] Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH Surface Classification [Object Recognition] Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH Surface Classification [Object Recognition] Registration Conclusion      ...
Introduction Acquisition Feature Estimation F/PFH Surface Classification [Object Recognition] Registration Conclusion      ...
Introduction   Acquisition   Feature Estimation   F/PFH   Surface Classification   Object Recognition   [Registration]   Co...
Introduction   Acquisition   Feature Estimation   F/PFH   Surface Classification   Object Recognition   [Registration]   Co...
Introduction   Acquisition   Feature Estimation   F/PFH   Surface Classification   Object Recognition   [Registration]   Co...
Introduction   Acquisition   Feature Estimation   F/PFH   Surface Classification   Object Recognition   [Registration]   Co...
Introduction   Acquisition   Feature Estimation   F/PFH   Surface Classification   Object Recognition   [Registration]   Co...
Introduction   Acquisition   Feature Estimation   F/PFH   Surface Classification   Object Recognition   [Registration]   Co...
Introduction   Acquisition   Feature Estimation   F/PFH   Surface Classification   Object Recognition   [Registration]   Co...
Introduction   Acquisition   Feature Estimation   F/PFH   Surface Classification   Object Recognition   [Registration]   Co...
Introduction   Acquisition   Feature Estimation   F/PFH   Surface Classification   Object Recognition   Registration   [Con...
Introduction   Acquisition   Feature Estimation   F/PFH   Surface Classification   Object Recognition   Registration   [Con...
Upcoming SlideShare
Loading in...5
×

Cvpr2010 open source vision software, intro and training part viii point cloud library - rusu - unknown - 2010

928

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
928
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
13
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Cvpr2010 open source vision software, intro and training part viii point cloud library - rusu - unknown - 2010

  1. 1. Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion Point Cloud Library 3D Features. Registration. Surface Classification Radu Bogdan RUSU June 14, 2010
  2. 2. [Introduction] Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion Outline 1. Introduction 2. Acquisition 3. Feature Estimation 4. (Fast) Point Feature Histograms 5. Surface Classification 6. Object Recognition 7. Registration 8. ConclusionRadu Bogdan RUSU PCL :: 3D Features
  3. 3. [Introduction] Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion Motivation (1/2) Point-based classification Figure out ways to “classify the world”Radu Bogdan RUSU PCL :: 3D Features
  4. 4. [Introduction] Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion Motivation (2/2) Point-based classification Two basic alternatives: 1. create a powerful, discriminative 3D feature descriptor; learn different classes of surface or object types; use a machine learning classifier . (the solution efficiency depends on the method used − let your “neighbor” (the ML guy) solve it) →Radu Bogdan RUSU PCL :: 3D Features
  5. 5. [Introduction] Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion Motivation (2/2) Point-based classification Two basic alternatives: 1. create a powerful, discriminative 3D feature descriptor; learn different classes of surface or object types; use a machine learning classifier . (the solution efficiency depends on the method used − let your “neighbor” (the ML guy) solve it) → 2. use geometric reasoning techniques: segmentation, region growing, robust estimators, octrees, ... fit linear (planes, lines) and non-linear (cylinders, spheres, etc) models to your dataRadu Bogdan RUSU PCL :: 3D Features
  6. 6. Introduction [Acquisition] Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion Outline 1. Introduction 2. Acquisition 3. Feature Estimation 4. (Fast) Point Feature Histograms 5. Surface Classification 6. Object Recognition 7. Registration 8. ConclusionRadu Bogdan RUSU PCL :: 3D Features
  7. 7. Introduction [Acquisition] Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion Acquisition (1/1) How are Point Clouds acquired? Where do they come from? Recap: Point Cloud acquisition sources: laser/lidar sensors (2D/3D) stereo cameras time-of-flight (TOF) cameras etc... *Radu Bogdan RUSU PCL :: 3D Features
  8. 8. Introduction [Acquisition] Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion Filtering (1-2/2) Statistical Gross Outlier Removal :: PCL/Filters/StatisticalOutlierRemoval P = {pi ∈ Praw | dist(pi , pj ) > µ + dthresh · σ} Distance Analysis: Before and after:Radu Bogdan RUSU PCL :: 3D Features
  9. 9. Introduction [Acquisition] Feature Estimation F/PFH Surface Classification Object Recognition Registration Conclusion Filtering (1-2/2) Statistical Gross Outlier Removal :: PCL/Filters/StatisticalOutlierRemoval P = {pi ∈ Praw | dist(pi , pj ) > µ + dthresh · σ} Before and after:Radu Bogdan RUSU PCL :: 3D Features
  10. 10. Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion Outline 1. Introduction 2. Acquisition 3. Feature Estimation 4. (Fast) Point Feature Histograms 5. Surface Classification 6. Object Recognition 7. Registration 8. ConclusionRadu Bogdan RUSU PCL :: 3D Features
  11. 11. Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion Local Features Estimation (1-2/5) Surface Normal Estimation :: PCL/Features/NormalEstimationOMP,TBB Given a point cloud with x,y,z 3D point coordinatesRadu Bogdan RUSU PCL :: 3D Features
  12. 12. Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion Local Features Estimation (1-2/5) Surface Normal Estimation :: PCL/Features/NormalEstimationOMP,TBB Given a point cloud with x,y,z 3D point coordinates Select each point’s k -nearest neighbors, fit a local plane, and compute the plane normalRadu Bogdan RUSU PCL :: 3D Features
  13. 13. Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion Local Features Estimation (3/5) Surface Normal and Curvature Estimation k 1 k Cj = i=1 ξi · (pi − pj )T · (pi − pj ), p = k · i=1 pi  di2 λ0  − σp = e µ2 , pi outlier λ0 +λ1 +λ2 ξi = 1, pi inlier Radu Bogdan RUSU PCL :: 3D Features
  14. 14. Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion Local Features Estimation (4-5/5) Consistent Normal Orientation Before Extended Gaussian Image Orientation consistent for: 1. registration 2. feature estimation 3. surface representation normals on the Gaussian sphere should be in the same half-spaceRadu Bogdan RUSU PCL :: 3D Features
  15. 15. Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion Local Features Estimation (4-5/5) Consistent Normal Orientation Before After (viewpoint − pi ) · npi ≥ 0 or: propagate consistency through an EMSTRadu Bogdan RUSU PCL :: 3D Features
  16. 16. Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion Feature Persistence (1/5) What scale to choose ? bad scale (too small) good scale Selecting the right scale (k -neighborhood) is problematic:Radu Bogdan RUSU PCL :: 3D Features
  17. 17. Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion Feature Persistence (2/5) Solution: Compute the persistence of the feature over multiple scales Scales are independent, can be parallelized Where to threshold ? Computer Graphics: look for jumps in the feature curve Datasets in “robotics” have non-smooth surfaces. Solution:Radu Bogdan RUSU PCL :: 3D Features
  18. 18. Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion Feature Persistence (3/5) Multiple radii analysis L1-Manhattan L2-Euclidean Jeffries-MatusitaRadu Bogdan RUSU PCL :: 3D Features
  19. 19. Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion Feature Persistence (4/5) Multiple radii analysis Bhattacharyya Chi-Square Kullback-LeiblerRadu Bogdan RUSU PCL :: 3D Features
  20. 20. Introduction Acquisition [Feature Estimation] F/PFH Surface Classification Object Recognition Registration Conclusion Feature Persistence (5/5) Example Result RAW/intensity data Persistent Feature Histograms * n−1 Pf = [Pfi ∩ Pfi+1 ] i=1Radu Bogdan RUSU PCL :: 3D Features
  21. 21. Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion Outline 1. Introduction 2. Acquisition 3. Feature Estimation 4. (Fast) Point Feature Histograms 5. Surface Classification 6. Object Recognition 7. Registration 8. ConclusionRadu Bogdan RUSU PCL :: 3D Features
  22. 22. Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion Point Feature Histograms (PFH) (1-4/5) Basic Concepts :: PCL/Features/FPFHEstimation For every point pair (ps , ns ); (pt , nt ) , let u = ns , v = (pt −ps )×u, w = u×v  f0 = v , n j   f1 = u, pj − pi /||pj − pi ||  x≤3 fx ·d i = · dx f2 = ||pj − pi ||  hist  x=0 fxmax −fxmin   f3 = atan( w, nj , u, nj )Radu Bogdan RUSU PCL :: 3D Features
  23. 23. Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion Point Feature Histograms (PFH) (1-4/5) Basic Concepts :: PCL/Features/FPFHEstimation  f0 = v , n j   f1 = u, pj − pi /||pj − pi ||  x≤3 fx ·d i = · dx f2 = ||pj − pi ||  hist  x=0 fxmax −fxmin   f3 = atan( w, nj , u, nj )Radu Bogdan RUSU PCL :: 3D Features
  24. 24. Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion Point Feature Histograms (PFH) (1-4/5) Basic Concepts :: PCL/Features/FPFHEstimation For every point pair (ps , ns ); (pt , nt ) , let u = ns , v = (pt −ps )×u, w = u×vRadu Bogdan RUSU PCL :: 3D Features
  25. 25. Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion Point Feature Histograms (PFH) (1-4/5) Points lying on different geometric primitivesRadu Bogdan RUSU PCL :: 3D Features
  26. 26. Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion Point Feature Histograms (PFH) (5/5) Complexity Analysis Complexity is high: O(k 2 ). Optimizations to the rescue! Unordered OrderedRadu Bogdan RUSU PCL :: 3D Features
  27. 27. Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion Fast Point Feature Histograms (FPFH) (1/5) Basic Concepts 1 k 1 Re-formulate: FPFH(p) = SPF (p) + k i=1 ωk · SPF (pk ) Point Feature Histograms (PFH) Fast Point Feature Histograms (FPFH)Radu Bogdan RUSU PCL :: 3D Features
  28. 28. Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion Fast Point Feature Histograms (FPFH) (2/5) Theoretical formulationRadu Bogdan RUSU PCL :: 3D Features
  29. 29. Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion Fast Point Feature Histograms (FPFH) (3/5) Noise Analysis Synthetically noiseless Synthetically noisifiedRadu Bogdan RUSU PCL :: 3D Features
  30. 30. Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion Fast Point Feature Histograms (FPFH) (4/5) Noise AnalysisRadu Bogdan RUSU PCL :: 3D Features
  31. 31. Introduction Acquisition Feature Estimation [F/PFH] Surface Classification Object Recognition Registration Conclusion Fast Point Feature Histograms (FPFH) (5/5) Persistence SimilarityRadu Bogdan RUSU PCL :: 3D Features
  32. 32. Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion Outline 1. Introduction 2. Acquisition 3. Feature Estimation 4. (Fast) Point Feature Histograms 5. Surface Classification 6. Object Recognition 7. Registration 8. ConclusionRadu Bogdan RUSU PCL :: 3D Features
  33. 33. Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion Surface Classification (1/8) Point-based classification recap Figure out ways to “classify the world”Radu Bogdan RUSU PCL :: 3D Features
  34. 34. Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion Surface Classification (2/8) Learning classes of surfaces Concave vs Convex 13 classes totalRadu Bogdan RUSU PCL :: 3D Features
  35. 35. Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion Surface Classification (3/8) Noise AnalysisRadu Bogdan RUSU PCL :: 3D Features
  36. 36. Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion Surface Classification (4/8) Most Discriminating Features Selection How many training samples to generate ? Need to simulate real scans. Solution: generate a lot, then select the most discriminative onesRadu Bogdan RUSU PCL :: 3D Features
  37. 37. Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion Surface Classification (5/8) Classification results using different methods SVM KNN KMeans Table 1. Classification results Method used Noiseless Noisy SVM RBF Sublinear kernel 95.26% 89.55% KNN Bhattacharyya (µ-dist) 87.11% 83.53% K-Means (81D) Bhattacharyya 73.63% 70.74%Radu Bogdan RUSU PCL :: 3D Features
  38. 38. Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion Surface Classification (6/8) FPFH classification resultsRadu Bogdan RUSU PCL :: 3D Features
  39. 39. Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion Surface Classification (7/8) Classification results using PFH and SVM: 89.95% * *Radu Bogdan RUSU PCL :: 3D Features
  40. 40. Introduction Acquisition Feature Estimation F/PFH [Surface Classification] Object Recognition Registration Conclusion Surface Classification (8/8) Classification results using FPFH and CRF: 97.36%Radu Bogdan RUSU PCL :: 3D Features
  41. 41. Introduction Acquisition Feature Estimation F/PFH Surface Classification [Object Recognition] Registration Conclusion Outline 1. Introduction 2. Acquisition 3. Feature Estimation 4. (Fast) Point Feature Histograms 5. Surface Classification 6. Object Recognition 7. Registration 8. ConclusionRadu Bogdan RUSU PCL :: 3D Features
  42. 42. Introduction Acquisition Feature Estimation F/PFH Surface Classification [Object Recognition] Registration Conclusion GFPFH Concepts From local annotations (FPFH) to global (cluster) annotations (GFPFH)Radu Bogdan RUSU PCL :: 3D Features
  43. 43. Introduction Acquisition Feature Estimation F/PFH Surface Classification [Object Recognition] Registration Conclusion FPFH Classification Classification results using FPFH and CRF: 98.27% *Radu Bogdan RUSU PCL :: 3D Features
  44. 44. Introduction Acquisition Feature Estimation F/PFH Surface Classification [Object Recognition] Registration Conclusion GFPFH Classification Classification results using GFPFH and SVM: 95.13% *Radu Bogdan RUSU PCL :: 3D Features
  45. 45. Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition [Registration] Conclusion Outline 1. Introduction 2. Acquisition 3. Feature Estimation 4. (Fast) Point Feature Histograms 5. Surface Classification 6. Object Recognition 7. Registration 8. ConclusionRadu Bogdan RUSU PCL :: 3D Features
  46. 46. Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition [Registration] Conclusion Registration (1/7) Multiple Scans :: PCL/Registration/IterativeClosestPoint* points on similar surfaces *Radu Bogdan RUSU PCL :: 3D Features
  47. 47. Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition [Registration] Conclusion Registration (2/7) The classics *Radu Bogdan RUSU PCL :: 3D Features
  48. 48. Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition [Registration] Conclusion Registration (3/7) Outdoor Example: Persistence, Initial Alignment *Radu Bogdan RUSU PCL :: 3D Features
  49. 49. Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition [Registration] Conclusion Registration (4/7) Outdoor Example: Non-Linear Optimization *Radu Bogdan RUSU PCL :: 3D Features
  50. 50. Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition [Registration] Conclusion Registration (5/7) Outdoor Example: Non-Linear OptimizationRadu Bogdan RUSU PCL :: 3D Features
  51. 51. Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition [Registration] Conclusion Registration (6/7) Back to Indoor EnvironmentsRadu Bogdan RUSU PCL :: 3D Features
  52. 52. Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition [Registration] Conclusion Registration (7/7) Registered 360◦ scans - 15 million pointsRadu Bogdan RUSU PCL :: 3D Features
  53. 53. Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration [Conclusion] Outline 1. Introduction 2. Acquisition 3. Feature Estimation 4. (Fast) Point Feature Histograms 5. Surface Classification 6. Object Recognition 7. Registration 8. ConclusionRadu Bogdan RUSU PCL :: 3D Features
  54. 54. Introduction Acquisition Feature Estimation F/PFH Surface Classification Object Recognition Registration [Conclusion] Questions? http://www.ros.org/wiki/pclRadu Bogdan RUSU PCL :: 3D Features
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×