Image Vectorization using Gradient Meshes<br />Jian Sun, Lin Liang, Fang Wen, Heung-Yeung Shum<br />Microsoft Research Asi...
Cutout tool<br />Initial mesh<br />Input image<br />Optimized gradient mesh<br />Reconstruction<br />
Outline<br />Introduction<br />Background<br />Gradient Mesh<br />Optimized Gradient Mesh<br />Result<br />Conclusions<br />
Introduction<br />
Image Vectorization<br />Goal : convert a raster image into a vector graphics <br />Compact<br />Scalable<br />Easy to ani...
Gradient mesh<br />Gradient mesh, adrawing tool of commercial vector graphics editors<br />Tracing photograph<br />Start a...
Image represented by gradient mesh<br />gradient mesh<br />http://www.creativebush.com/tutorials/mesh_tutorial.php<br />
Image vectorization tools<br />Adobe Illustrator, “Live Trace”<br />Corel CoreDraw, “CorelTrace”<br />AutoTrace, “AutoTrac...
Optimized gradient mesh<br />Blend surface colors according to the control points color as constructing surface by the con...
Background<br />
Object-based vectorization<br />Object-based vectorization [Price and Barrett 06]<br />Hierarchically segmentation of obje...
RaveGrid<br />[Swaminarayan and Prasad 06]<br />Constrained Delaunay triangulation of the edge contour set<br />
cont.<br />
Ardeco<br />Automatic Region Detection and Conversion algorithm [Lecot and Levy 06]<br />Cubic splines<br />Each region fi...
cont.<br />
Gradient Mesh<br />
Overview<br />Cutout  tool<br />Input raster image<br />Initial mesh<br />(Coons mesh)<br />E(M) Constrains:<br />Smoothne...
Coherent matting method: sample object boundary color from the estimated foreground colors </li></li></ul><li>Mesh initial...
Ferguson patch<br />TA<br />TB<br />[Ferguson 1964]<br />P(0)=A<br />Basic Curve Segmentation<br />P(1)=B<br />
Ferguson patch<br />Basic Surface Segmentation<br />0<br />
Ferguson patch<br />Basic Surface Segmentation<br />0<br />
Gradient mesh<br />A gradient mesh consists of topologically planar rectangular Ferguson patches with mesh-lines<br />For ...
Ferguson patches are lack of Cv and Cu ! <br />Color interpolation<br />
[Wolberg and Alfy 99]<br />Determine the smoothest possible curve that passes through its control points  and satisfy mono...
Rendering Ferguson patches<br />Sample color of control points<br />Estimate Cu, Cv by Monotoic Cubic Spline algorithm<br ...
Scalability<br />A gradient mesh<br /><ul><li>original resolution (x 1)</li></ul>Scaling result (x8)<br /><ul><li>shape ed...
Minimize E(M)<br />min arg.<br />
Solve NULL problem using LM algorithm<br />Minimizing E(M) is a non-linear least squares (NULL) problem<br />Energy functi...
cont.<br /><ul><li>Gaussian pyramid from the input image and apply coarse-to-fine optimization for LM</li></li></ul><li>Op...
40 iterations</li></li></ul><li>Smooth constraint<br />Opt. gradient mesh without smooth(err. 1.8/pixel)<br />Opt. gradien...
Vector line guided optimized gradient mesh<br />User guided vector, V<br />Initial mesh<br />Opt. gradient mesh with user ...
Vector line guided optimized gradient mesh<br />w = 1/5 L <br />Initial mesh<br />Opt. gradient mesh with V<br />
Boundary constraint<br />The boundary of a gradient – one or more cubic Bezier spline<br />The control points on the bound...
Results<br />
Red pepper<br />Optimized<br /> the highlight and shadow regions are reconstructed <br />Initial<br />gradient meshes<br /...
Sculpture<br />Optimized gradient mesh<br />Input image<br />Reconstruction<br />
Face<br />Optimized gradient mesh<br />Input image<br />Reconstruction<br />
Conclusions<br />Input image<br />Introduce the gradient mesh as an image representation tool first<br />Present optimized...
END<br />
study Image Vectorization using Optimized Gradeint Meshes
study Image Vectorization using Optimized Gradeint Meshes
Upcoming SlideShare
Loading in …5
×

study Image Vectorization using Optimized Gradeint Meshes

2,417 views

Published on

study

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,417
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
36
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

study Image Vectorization using Optimized Gradeint Meshes

  1. 1. Image Vectorization using Gradient Meshes<br />Jian Sun, Lin Liang, Fang Wen, Heung-Yeung Shum<br />Microsoft Research Asia<br />ACM SIGGRAPH 2007<br />
  2. 2. Cutout tool<br />Initial mesh<br />Input image<br />Optimized gradient mesh<br />Reconstruction<br />
  3. 3. Outline<br />Introduction<br />Background<br />Gradient Mesh<br />Optimized Gradient Mesh<br />Result<br />Conclusions<br />
  4. 4. Introduction<br />
  5. 5. Image Vectorization<br />Goal : convert a raster image into a vector graphics <br />Compact<br />Scalable<br />Easy to animate<br />Requirements<br />Vector-based contents (eg. Flash or SVG) on the Internet<br />Vector-based GUIs used in Windows Vista<br />
  6. 6. Gradient mesh<br />Gradient mesh, adrawing tool of commercial vector graphics editors<br />Tracing photograph<br />Start adding mesh points<br />Selecting mid value skin tone<br />Sampling colors from face mesh to hide seam<br />Sampling colors from photo<br />Sampling colors from within the mesh<br />Finished eye/eye socket<br />http://www.creativebush.com/tutorials/mesh_tutorial.php<br />
  7. 7. Image represented by gradient mesh<br />gradient mesh<br />http://www.creativebush.com/tutorials/mesh_tutorial.php<br />
  8. 8. Image vectorization tools<br />Adobe Illustrator, “Live Trace”<br />Corel CoreDraw, “CorelTrace”<br />AutoTrace, “AutoTrace”<br />Input image<br />Adobe, Live Trace<br />
  9. 9. Optimized gradient mesh<br />Blend surface colors according to the control points color as constructing surface by the control points<br />Optimize the gradient mesh as an energy minimization problem<br />Advantages<br />Efficiency of use<br />Easy to edit – modify, animation<br />Scalability<br />Compact representation <br />JPEG, 37.5 KB<br />Optimized, 7.7KB<br />
  10. 10. Background<br />
  11. 11. Object-based vectorization<br />Object-based vectorization [Price and Barrett 06]<br />Hierarchically segmentation of object and sub-objects by a recursive graph cut algorithm<br />Subdivide meshes until the reconstruct error is below a threshold<br />Input image<br />Subdivision mesh<br />
  12. 12. RaveGrid<br />[Swaminarayan and Prasad 06]<br />Constrained Delaunay triangulation of the edge contour set<br />
  13. 13. cont.<br />
  14. 14. Ardeco<br />Automatic Region Detection and Conversion algorithm [Lecot and Levy 06]<br />Cubic splines<br />Each region filled with a constant color, or a linear or circular gradient <br />
  15. 15. cont.<br />
  16. 16. Gradient Mesh<br />
  17. 17. Overview<br />Cutout tool<br />Input raster image<br />Initial mesh<br />(Coons mesh)<br />E(M) Constrains:<br />Smoothness<br />User guidedvector<br />Boundary<br />min argEnergy(Mesh) <br />non-linear least squares (NULL) problem <br />Levenberg-Marquardt (LM) algorithm<br />Optimized gradient mesh<br />(Ferguson patch)<br />Reconstruction image<br /><ul><li>Color fitting: monotonic cubic spline interpolation
  18. 18. Coherent matting method: sample object boundary color from the estimated foreground colors </li></li></ul><li>Mesh initialization (manually)<br />Decompose the input image into sub-objects using Cutout tool [Li et al. 04]<br />Divide each sub-objects with 4 segments manually as Coons patches<br />m(u,v): the position vector of a point (u,v)<br />Q : control points el al.<br />F : basis functions<br />
  19. 19. Ferguson patch<br />TA<br />TB<br />[Ferguson 1964]<br />P(0)=A<br />Basic Curve Segmentation<br />P(1)=B<br />
  20. 20. Ferguson patch<br />Basic Surface Segmentation<br />0<br />
  21. 21. Ferguson patch<br />Basic Surface Segmentation<br />0<br />
  22. 22. Gradient mesh<br />A gradient mesh consists of topologically planar rectangular Ferguson patches with mesh-lines<br />For each point q<br /><ul><li>Position: {xq,yq}</li></ul>Derivatives: {mqu,mqv, αqumqu, αqvmqv}<br />RGB color: cq = {cq(r), cq(g), cq(b)}<br />
  23. 23. Ferguson patches are lack of Cv and Cu ! <br />Color interpolation<br />
  24. 24. [Wolberg and Alfy 99]<br />Determine the smoothest possible curve that passes through its control points and satisfy monotonic constraint<br />The seven data points are monotonically increasing in f(xi) for 0 ≦i ≦ 6, the cubic spline is not monotonic<br />Monotonic cubic spline<br />
  25. 25. Rendering Ferguson patches<br />Sample color of control points<br />Estimate Cu, Cv by Monotoic Cubic Spline algorithm<br />Render Ferguson patches<br />
  26. 26. Scalability<br />A gradient mesh<br /><ul><li>original resolution (x 1)</li></ul>Scaling result (x8)<br /><ul><li>shape edges are well preserved</li></ul>Bi-cubic raster scaling (x8)<br /><ul><li>Blocky artifacts appear </li></li></ul><li>Optimized gradient mesh<br />
  27. 27. Minimize E(M)<br />min arg.<br />
  28. 28. Solve NULL problem using LM algorithm<br />Minimizing E(M) is a non-linear least squares (NULL) problem<br />Energy function<br />z: vector form of unknowns in M<br />Levenberg-Marquardt (LM) algorithm is the most successful solver for NULL<br />[Levenberg 44], [Nocedal and Wright 99]<br />
  29. 29. cont.<br /><ul><li>Gaussian pyramid from the input image and apply coarse-to-fine optimization for LM</li></li></ul><li>Optimization<br />Gradient mesh of Adobe Illustrator<br />Optimized gradient mesh<br /><ul><li>0.7/pixel reconstruction error
  30. 30. 40 iterations</li></li></ul><li>Smooth constraint<br />Opt. gradient mesh without smooth(err. 1.8/pixel)<br />Opt. gradient mesh with smooth (err. 0.9 pixel)<br />Input image<br />Smooth neighboring patches also,<br />mp(- △s, t) = mp-1(1- △s, t) <br />
  31. 31. Vector line guided optimized gradient mesh<br />User guided vector, V<br />Initial mesh<br />Opt. gradient mesh with user guided vector<br />(err. 0.5/pixel)<br />Directly optimized gradient mesh <br />(err. 2.5/pixel)<br />
  32. 32. Vector line guided optimized gradient mesh<br />w = 1/5 L <br />Initial mesh<br />Opt. gradient mesh with V<br />
  33. 33. Boundary constraint<br />The boundary of a gradient – one or more cubic Bezier spline<br />The control points on the boundary only move along the spline<br />Ex: control point q on the spline S in u direction<br />
  34. 34. Results<br />
  35. 35. Red pepper<br />Optimized<br /> the highlight and shadow regions are reconstructed <br />Initial<br />gradient meshes<br />Gradient meshes by an artist (354 patches)<br />
  36. 36. Sculpture<br />Optimized gradient mesh<br />Input image<br />Reconstruction<br />
  37. 37. Face<br />Optimized gradient mesh<br />Input image<br />Reconstruction<br />
  38. 38. Conclusions<br />Input image<br />Introduce the gradient mesh as an image representation tool first<br />Present optimized gradient mesh<br />Limitations<br />A fine image details and highly textured image<br />Boundaries or topologies are too complicated<br />Reconstructed image<br />Optimized gradient meshes<br />
  39. 39. END<br />

×