Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Edge-Based Image Coarsening<br />SIGGRAPH 2010<br />RaananFattal<br />Hebrew University of Jerusalem, Israel<br />Robert C...
Abstract<br />A new dimensionally-reduced linear image space<br />Pixel-by-pixel image  sparse grid kernel<br />High perf...
Outline<br />Introduction<br />Image coarsening<br />Pixel-by-pixel  spanned by spare kernel (BIG)<br />Scale-Adaptive co...
Introduction<br />State-of-art image editing tech.<br />Bilateral filter based<br />Gradient based manipulation <br />
Bilateral filter [Tomasi and Manduchi 1998]<br />A nonlinear filter that locally gathers information from similar pixels<b...
Bilateral filter [Tomasi and Manduchi 1998]<br />Averaging over the 2N+1 neighborhood<br />The weight<br />The neighbor sa...
Bilateral filter [Tomasi and Manduchi 1998]<br />Michael Elad, “Algorithms for Noise Removal and the Bilateral Filter.ppt’...
Bilateral filter [Tomasi and Manduchi 1998]<br />Center Sample<br />Neighborhood<br />It is clear that in weighting this n...
Bilateral filter [Tomasi and Manduchi 1998]<br />Ws<br />WR<br />9/59<br />Michael Elad, “Algorithms for Noise Removal and...
Bilateral filter [Tomasi and Manduchi 1998]<br />It appears that the weight is inversely prop. to the         Total-Distan...
Bilateral filter [Tomasi and Manduchi 1998]<br />A nonlinear filter that locally gathers information from similar pixels<b...
Bilateral filter<br />Decompose images into a pixelwise-smooth layer [Durand and Dorsey 02]<br />Realtime bilateral filter...
Gradient based image editing<br />Propagate locate image editing operation according to the gradient field<br />
The gradient domain provides a natural setting for image manipulation tech-<br />niques, including dynamic range compressi...
Should we need a coarse representation of an image ?<br />Why always pixel-wise kernel of bilateral filtering ?<br />A spa...
A new dimensionally-reduced linear image<br />A coarse image representation consisting of elementary basis functions deriv...
1D kernel construction steps –1D input I(x)<br />
1D kernel construction steps –Grid kernels<br />I(y)<br />y<br />S(x,y)<br />y<br />Kernel S of bilateral filter at grid y...
1D kernel construction steps –Grid kernels<br />I(y)<br />y<br />S(x,y)<br />y<br />Kernel S of bilateral filter at grid y...
1D kernel construction steps –Grid kernels<br />I(y)<br />S(x,y)<br />Kernel S of bilateral filter at grid y<br />
1D kernel construction steps –Grid kernels<br />I(y)<br />y<br />S(x,y)<br />y<br />Kernel S of bilateral filter at grid y...
1D kernel construction steps –Grid kernels<br />I(y)<br />y<br />S(x,y)<br />y<br />Kernel S of bilateral filter at grid y...
1D kernel construction steps.<br />I(y)<br />y<br />S(x,y)<br />y<br />Kernel S of bilateral filter at grid y<br />
1D kernel construction steps-Center adjustment<br />kernel centers are shifted away from edges.  <br />
1D kernel construction steps-add Islands<br />Island kernel are added<br />Island kernel is where surrounding kernel is le...
1D kernel construction steps –normalized grid kernel Ki(x)<br />C(x) : a per-pixel normalization factor<br />xi for i=1..n...
Normalized grid kernel Ki(x) :case C(x)=1<br />
Bilateral Image Coarsening (BIG), J- spanned by Ki(x)<br />
Bilateral Image Coarsening (BIG), J- spanned by Ki(x), Pi(x)<br />Pi(x)<br />
Bilateral Image Coarsening space (BIG), J<br />
Image kernels<br /><ul><li> grid kernel center</li></ul>+ shifted kernel center <br />+ island center<br />non-overlapping...
Processing<br />Down-sample the input pixels by 2k<br />Construct grid kernel center S(xi,x)<br />Fine-tune<br />Local shi...
Pixel normalized factor C<br /><ul><li>set C(x) = I(x)
J is a subspace “surrounding” the input image.
set C(x) = 1
J becomes a space of piecewise-smooth functions</li></li></ul><li>Scale-Adaptive Coarsening<br />Construct kernels varing ...
Scale-Adaptive Coarsening- down sampling grid kernels 1/4<br />grid-based<br />pixel-based<br />
Scale-Adaptive Coarsening- down sampling grid kernels 2/4<br />
Scale-Adaptive Coarsening- down sampling grid kernels 3/4<br />Level l : given m grid kernel<br />Level l+1: Sample m/k2 g...
Scale-Adaptive Coarsening- down sampling grid kernels 4/4<br />
Upcoming SlideShare
Loading in …5
×

stduy Edge-Based Image Coarsening

1,163 views

Published on

Published in: Technology
  • Be the first to comment

stduy Edge-Based Image Coarsening

  1. 1. Edge-Based Image Coarsening<br />SIGGRAPH 2010<br />RaananFattal<br />Hebrew University of Jerusalem, Israel<br />Robert Carroll<br />University of California, Berkeley<br />ManeeshAgrawala<br />University of California, Berkeley<br />
  2. 2. Abstract<br />A new dimensionally-reduced linear image space<br />Pixel-by-pixel image  sparse grid kernel<br />High performance on gradient-based tone mapping techniques<br />Useful for energy-minimization method<br />
  3. 3. Outline<br />Introduction<br />Image coarsening<br />Pixel-by-pixel  spanned by spare kernel (BIG)<br />Scale-Adaptive corasening<br />Projection operations<br />Image/Gradient projection<br />Gradient projection<br />Results – applied on<br />Shadow Removal<br />Sparse-Error Norm<br />AlphaMatting<br />Joint Bilateral upsampling<br />Conclusion <br />
  4. 4. Introduction<br />State-of-art image editing tech.<br />Bilateral filter based<br />Gradient based manipulation <br />
  5. 5. Bilateral filter [Tomasi and Manduchi 1998]<br />A nonlinear filter that locally gathers information from similar pixels<br />Preserve edge<br />Noise removal<br />
  6. 6. Bilateral filter [Tomasi and Manduchi 1998]<br />Averaging over the 2N+1 neighborhood<br />The weight<br />The neighbor sample<br />The result at the kth sample<br />Y[j]<br />Normalization of the weighting<br />j<br />k<br />6/59<br />Michael Elad, “Algorithms for Noise Removal and the Bilateral Filter.ppt’, 2002<br />
  7. 7. Bilateral filter [Tomasi and Manduchi 1998]<br />Michael Elad, “Algorithms for Noise Removal and the Bilateral Filter.ppt’, 2002<br />
  8. 8. Bilateral filter [Tomasi and Manduchi 1998]<br />Center Sample<br />Neighborhood<br />It is clear that in weighting this neighborhood, we would like to preserve the step<br />8/59<br />Michael Elad, “Algorithms for Noise Removal and the Bilateral Filter.ppt’, 2002<br />
  9. 9. Bilateral filter [Tomasi and Manduchi 1998]<br />Ws<br />WR<br />9/59<br />Michael Elad, “Algorithms for Noise Removal and the Bilateral Filter.ppt’, 2002<br />
  10. 10. Bilateral filter [Tomasi and Manduchi 1998]<br />It appears that the weight is inversely prop. to the Total-Distance (both horizontal and vertical) from the center sample.<br />Michael Elad, “Algorithms for Noise Removal and the Bilateral Filter.ppt’, 2002<br />
  11. 11. Bilateral filter [Tomasi and Manduchi 1998]<br />A nonlinear filter that locally gathers information from similar pixels<br />Preserve edge<br />Noise removal<br />
  12. 12. Bilateral filter<br />Decompose images into a pixelwise-smooth layer [Durand and Dorsey 02]<br />Realtime bilateral filter [Chen et al. 07]<br />Bilateral image decomposition [Durand and Dorsey 02]<br />Flash/no-flash enhancement [Eisemann and Durand 2004; Petschnigg et al. 04]<br />Tone management [Bae et al. 06]<br />NPR [Fattal et al. 07] <br />Upsampling[Kopf et al. 07].<br />
  13. 13. Gradient based image editing<br />Propagate locate image editing operation according to the gradient field<br />
  14. 14. The gradient domain provides a natural setting for image manipulation tech-<br />niques, including dynamic range compression [Fattal et al. 2002], seamless image<br />stitching [Levin et al. 2004], image editing [P´erez et al. 2003], alpha matte extrac-<br />tion [Sun et al. 2004], and shadow removal [Finlayson et al. 2006; Xu et al. 2006].<br />Solving the Poisson equation amounts to performing an L2 minimization in which<br />the image gradients are weighted uniformly in space. More recent gradient based<br />methods such as colorization [Levin et al. 2004], interactive tone mapping [Lischin-<br />ski et al. 2006] and alpha matting [Levin et al. 2006], propagate local image editing<br />operations throughout the image according to the underlying gradient field. These<br />approaches require solving a similar optimization problem, but in this case the<br />output image gradients are weighted in a spatially-dependent manner.<br />
  15. 15. Should we need a coarse representation of an image ?<br />Why always pixel-wise kernel of bilateral filtering ?<br />A sparse bilateral kernel ?<br />Coarse but keep edge ?<br />
  16. 16. A new dimensionally-reduced linear image<br />A coarse image representation consisting of elementary basis functions derived from the bilateral filter kernels<br />scale-adaptive coarsened representation<br />detail<br />coarse<br />binding together smooth regions but also shaped by strong edges<br />
  17. 17. 1D kernel construction steps –1D input I(x)<br />
  18. 18. 1D kernel construction steps –Grid kernels<br />I(y)<br />y<br />S(x,y)<br />y<br />Kernel S of bilateral filter at grid y<br />
  19. 19. 1D kernel construction steps –Grid kernels<br />I(y)<br />y<br />S(x,y)<br />y<br />Kernel S of bilateral filter at grid y<br />
  20. 20. 1D kernel construction steps –Grid kernels<br />I(y)<br />S(x,y)<br />Kernel S of bilateral filter at grid y<br />
  21. 21. 1D kernel construction steps –Grid kernels<br />I(y)<br />y<br />S(x,y)<br />y<br />Kernel S of bilateral filter at grid y<br />
  22. 22. 1D kernel construction steps –Grid kernels<br />I(y)<br />y<br />S(x,y)<br />y<br />Kernel S of bilateral filter at grid y<br />
  23. 23. 1D kernel construction steps.<br />I(y)<br />y<br />S(x,y)<br />y<br />Kernel S of bilateral filter at grid y<br />
  24. 24. 1D kernel construction steps-Center adjustment<br />kernel centers are shifted away from edges. <br />
  25. 25. 1D kernel construction steps-add Islands<br />Island kernel are added<br />Island kernel is where surrounding kernel is less than τ<br />
  26. 26. 1D kernel construction steps –normalized grid kernel Ki(x)<br />C(x) : a per-pixel normalization factor<br />xi for i=1..n are the kernel centers <br />
  27. 27. Normalized grid kernel Ki(x) :case C(x)=1<br />
  28. 28. Bilateral Image Coarsening (BIG), J- spanned by Ki(x)<br />
  29. 29. Bilateral Image Coarsening (BIG), J- spanned by Ki(x), Pi(x)<br />Pi(x)<br />
  30. 30. Bilateral Image Coarsening space (BIG), J<br />
  31. 31. Image kernels<br /><ul><li> grid kernel center</li></ul>+ shifted kernel center <br />+ island center<br />non-overlapping kernel<br />
  32. 32. Processing<br />Down-sample the input pixels by 2k<br />Construct grid kernel center S(xi,x)<br />Fine-tune<br />Local shift xi to xi’ where minimal |▽I(x’)| within a window of k-2 by k-2 across xi<br />Add island kernel where ΣS(xi,x) < τ<br />Normalized kernel Ki (x)<br />A set of construction polynomial (CPs)<br />Build the Bilateral Image Coarsening (BIC) space<br />aij : degree of freedom of CPs<br />
  33. 33. Pixel normalized factor C<br /><ul><li>set C(x) = I(x)
  34. 34. J is a subspace “surrounding” the input image.
  35. 35. set C(x) = 1
  36. 36. J becomes a space of piecewise-smooth functions</li></li></ul><li>Scale-Adaptive Coarsening<br />Construct kernels varing in size depending on image content<br />roughly the same size <br />non-regular<br />non-regular<br />grid kernel of roughly the same size<br />grid size depends on image content<br />
  37. 37. Scale-Adaptive Coarsening- down sampling grid kernels 1/4<br />grid-based<br />pixel-based<br />
  38. 38. Scale-Adaptive Coarsening- down sampling grid kernels 2/4<br />
  39. 39. Scale-Adaptive Coarsening- down sampling grid kernels 3/4<br />Level l : given m grid kernel<br />Level l+1: Sample m/k2 grid kernels at level l+1<br />m=8<br />m/k2<br />
  40. 40. Scale-Adaptive Coarsening- down sampling grid kernels 4/4<br />
  41. 41. Scale-Adaptive Coarsening - Island kernels<br />Ω : the list of island kernels<br />ΩC : a coarser subset<br />once the kernels at level l are computed, we run through the current list of island kernels, : , and if we encounter a kernel K that is poorly covered by ΩC, i.e., as before ΣjS(Kj,K) < τ where j ∈ ΩC then we add it to ΩC. Thus, each island kernel that is missing from ΩC is close enough to one (or more) of the kernels in ΩC. Finally, we use this subset of island kernels to define coarser scale-adaptive island kernels KiCfor i ∈ ΩC by<br />
  42. 42. Given an input image I. How to build a corresponding BIG space representation J ?<br />Solutions<br />Computeaij directly (chapter 1,2)<br />Project I or ▽I onto J to obtain the closest image within J (chapter 3)<br />

×