CVPR2010: Advanced ITinCVPR in a Nutshell: part 4: Isocontours, Registration

228 views
174 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
228
On SlideShare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
4
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

CVPR2010: Advanced ITinCVPR in a Nutshell: part 4: Isocontours, Registration

  1. 1. A vne da cdIfr t nT e r inomai h oyn oC P “ aN t e” VRi n us l hl CP VR T ti u rl oa J n 1 -82 1 u e 31 0 0 S nFa c c ,A a rn i oC sIsocontours and Image RegistrationAnand Rangarajan
  2. 2. Image RegistrationThe need for information-theoretic measuresWhen there is no clearly established analytic relationship betweentwo or more images, it is often more convenient to minimize aninformation-theoretic distance measure such as the negative of themutual information (MI). Figure: Left: MR-PD slice. Right: Warped, noisy MR-T2 slice. 2/20
  3. 3. The joint space of two images 3/20
  4. 4. Density and Entropy estimationDensity estimation Histogramming Parzen windows Mixture models, wavelet densities (and other parametrizations)Entropy estimation Entropy estimation from the joint density (or distribution) Direct entropy estimation (kNN, MST, Voronoi etc.) Entropy estimation from the cumulative distribution (cdf) 4/20
  5. 5. Moving away from samplesThe underlying commonality in all of the previous approachesAll previous approaches are sample-based. Our new approach doesnot begin with the idea of individual samples. Obtain approx. to Take samples density and entropy Obtain improved Take more samples approximation 5/20
  6. 6. Image-based density estimation Uncountable infinityAssume uniform distribution of samples taken on location Transformation Each point in the Location continuum contributes to intensity Intensity distribution Distribution on intensity Image-Based 6/20
  7. 7. Isocontours 7/20
  8. 8. Isocontour area-based densityIsocontour density estimationArea trapped between level sets α and α + ∆α is proportional to theprobability Pr(α ≤ I ≤ α + ∆α). The density function is ˆ 1 1 p(α) = du A I (x,y )=α | I (x, y )| Level sets at I (x, y ) = α 8/20
  9. 9. Isocontour area-based densityIsocontour density estimationArea trapped between level sets α and α + ∆α is proportional to theprobability Pr(α ≤ I ≤ α + ∆α). The density function is ˆ 1 1 p(α) = du A I (x,y )=α | I (x, y )| Level sets at I (x, y ) = α and I (x, y ) = α + ∆α 8/20
  10. 10. Isocontour area-based densityIsocontour density estimationArea trapped between level sets α and α + ∆α is proportional to theprobability Pr(α ≤ I ≤ α + ∆α). The density function is ˆ 1 1 p(α) = du A I (x,y )=α | I (x, y )| Area in between I (x, y ) = α and I (x, y ) = α + ∆α 8/20
  11. 11. Joint ProbabilityFigure: Two synthetic images 9/20
  12. 12. Joint ProbabilityFigure: Level sets of the two synthetic images 10/20
  13. 13. Joint ProbabilityIsocontour overlay exhibits area overlap Figure: Overlay of the two sets of isocontours 11/20
  14. 14. Joint Probability Level sets at I1 (x, y ) = α1 and I2 (x, y ) = α2The cumulative area of the black regions is proportional toPr(α1 ≤ I1 ≤ α1 + ∆α1 , α2 ≤ I2 ≤ α2 + ∆α2 ). 12/20
  15. 15. Joint Probability Level sets at I1 = α1 , α1 + ∆α1 and I2 = α2 and α2 + ∆α2The cumulative area of the black regions is proportional toPr(α1 ≤ I1 ≤ α1 + ∆α1 , α2 ≤ I2 ≤ α2 + ∆α2 ). 12/20
  16. 16. Joint Probability Areas: α1 ≤ I1 ≤ α1 + ∆α1 and α2 ≤ I2 ≤ α2 + ∆α2The cumulative area of the black regions is proportional toPr(α1 ≤ I1 ≤ α1 + ∆α1 , α2 ≤ I2 ≤ α2 + ∆α2 ). 12/20
  17. 17. Joint Probability ExpressionThe joint density of images I1 (x, y ) and I2 (x, y ) with area ofoverlap A is related to the area of intersection of regionsbetween level curves at α1 and α1 + ∆α1 of I1 and at α2 andα2 + ∆α2 of I2 as ∆α1 → 0, ∆α2 → 0.The joint density ˆ ˆ 1 du1 du2p(α1 , α2 ) = A I1 (x,y )=α1 ,I2 (x,y )=α2 | I1 (x, y ) I2 (x, y ) sin(θ)|where u1 and u2 are the level curve tangent vectors in I1 and I2respectively and θ the angle between the image gradients. 13/20
  18. 18. When there’s no joint densityPathological cases 1Examine | I1 (x,y ) I2 (x,y ) sin(θ)| : Level curves of Image 2 at intensities α2 and α2+∆α Level curves of Image 1 Region in Image 2 at intensities α1 and of constant intensity α1+∆α α2 Region in Image 1 Region in Image 1 with constant intensity of constant intensity α1 α1 Level curves of Image 2 at intensities α2 and Area of intersection Area of intersection α2+∆α Area where level curves of the two regions (contribution to from images 1 and 2 [contribution to P(α1,α2)] P(α1,α2) are parallelFigure: Left: Both images flat. Middle: One image flat. Right: Gradientsrun locally parallel. 14/20
  19. 19. Binning without the binning problemChoose as many bins as desired 15/20
  20. 20. Binning without the binning problemChoose as many bins as desired 15/20
  21. 21. Binning without the binning problemChoose as many bins as desired 15/20
  22. 22. Binning without the binning problemChoose as many bins as desired 15/20
  23. 23. Information-theoretic formulationMutual Information-based registrationGiven two images I1 and I2 , a now standard approach to imageregistration minimizes E (T ) = −MI (I1 , I2 (T )) = H(I1 , I2 (T )) − H(I1 ) − H(I2 (T ))where the mutual information (MI) is unpacked as the sum of themarginal entropies minus the joint entropy. The entropies (Shannon)can be easily estimated from the iscontour density estimators (as wellas other estimators such as histogramming and Parzen windows).The transformation T (usually rigid or affine) is applied to only I2 inthis formulation. 16/20
  24. 24. Comparison with std. histograms 32 binsLeft: Standard histogramming. Right: Isocontours 17/20
  25. 25. Comparison with std. histograms 64 binsLeft: Standard histogramming. Right: Isocontours 17/20
  26. 26. Comparison with std. histograms 128 binsLeft: Standard histogramming. Right: Isocontours 17/20
  27. 27. Comparison with std. histograms 256 binsLeft: Standard histogramming. Right: Isocontours 17/20
  28. 28. Comparison with std. histograms 512 binsLeft: Standard histogramming. Right: Isocontours 17/20
  29. 29. Comparison with std. histograms 1024 binsLeft: Standard histogramming. Right: Isocontours 17/20
  30. 30. Joint density comparisons 16 bins Joint density histograms: 16 bins Joint density isocontours: 16 bins0.05 0.06 0.050.04 0.040.03 0.030.02 0.020.01 0.01 0 0 20 20 15 20 15 20 10 15 10 15 10 10 5 5 5 5 0 0 0 0 Left: Standard histogramming. Right: Isocontours 18/20
  31. 31. Joint density comparisons 32 bins Joint density histograms: 32 bins Joint density isocontours: 32 bins0.012 0.014 0.01 0.012 0.010.008 0.0080.006 0.0060.004 0.0040.002 0.002 0 0 40 40 30 40 30 40 20 30 20 30 20 20 10 10 10 10 0 0 0 0 Left: Standard histogramming. Right: Isocontours 18/20
  32. 32. Joint density comparisons 64 bins Joint density histograms: 64 bins Joint density isocontours: 64 bins −3 −3 x 10 x 10 4 3.5 3 3 2.5 2 2 1.5 1 1 0.5 0 080 80 60 80 60 80 40 60 40 60 40 40 20 20 20 20 0 0 0 0 Left: Standard histogramming. Right: Isocontours 18/20
  33. 33. Joint density comparisons 128 bins Joint density histograms: 128 bins Joint density isocontours: 128 bins −3 −3 x 10 x 10 2 1 0.81.5 0.6 1 0.40.5 0.2 0 0150 150 150 150 100 100 100 100 50 50 50 50 0 0 0 0 Left: Standard histogramming. Right: Isocontours 18/20
  34. 34. Mutual Information comparisonsSingle rotation parameter in 2D Noise standard deviation 0.05 Left: 32 bins, Right: 128 bins ISOCONTOURS ISOCONTOURS HIST BILINEAR HIST BILINEAR PVI PVI HIST CUBIC HIST CUBIC 2DPointProb 2DPointProb 0.4 0.8 0.3 0.6 0.2 0.4 0.1 0.2 0 0 0 10 20 30 40 50 0 10 20 30 40 50 19/20
  35. 35. Mutual Information comparisonsSingle rotation parameter in 2D Noise standard deviation 0.2 Left: 32 bins, Right: 128 bins ISOCONTOURS ISOCONTOURS HIST BILINEAR HIST BILINEAR PVI PVI HIST CUBIC HIST CUBIC 2DPointProb 2DPointProb 0.2 0.8 0.15 0.6 0.1 0.4 0.05 0.2 0 0 0 10 20 30 40 50 0 10 20 30 40 50 19/20
  36. 36. Mutual Information comparisonsSingle rotation parameter in 2D Noise standard deviation 1.0 Left: 32 bins, Right: 128 bins ISOCONTOURS ISOCONTOURS HIST BILINEAR HIST BILINEAR PVI PVI HIST CUBIC HIST CUBIC 2DPointProb 2DPointProb 0.08 0.5 0.4 0.06 0.3 0.04 0.2 0.02 0.1 0 0 0 10 20 30 40 50 0 10 20 30 40 50 19/20
  37. 37. DiscussionWith piecewise linear interpolation, much faster than upsampledhistogrammingExtended to multiple image registration and 3DStatistical significance (Kolmogorov-Smirnov) tests runOther groups (Oxford etc.) involved - analytic studiesApplied to mean shift filtering and unit vector density estimationDrawbacks: Non differentiable, no clean extension to higherdimensions 20/20

×