Morphing Image

1,374 views

Published on

Published in: Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,374
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
55
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Morphing Image

  1. 1. T H E U N I V E R S I T Y of T E X A S HEALTH SCIENCE CENTER AT HOUSTON S C H O O L of H E A L T H I N F O R M A T I O N S C I E N C E S Interpolation and MorphingFor students of HI 5323“Image Processing”Willy Wriggers, Ph.D.School of Health Information Scienceshttp://biomachina.org/courses/processing/05.html
  2. 2. Interpolation
  3. 3. Forward Mapping Let u(x, y) and v(x, y) be a mapping from location (x, y) to (u, v): B[u(x, y), v(x, y)] = A[x, y] x u y v A B© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  4. 4. Forward Mapping: Problems • Doesn’t always map to pixel locations • Solution: spread out effect of each pixel, e.g. by bilinear interpolation x u y v A B© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  5. 5. Forward Mapping: Problems • May produce holes in the output x u ? y v A B© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  6. 6. Forward Mapping: Problems • May produce holes in the output • Solution: sample source image (A) more often – Still can leave holes x u y v A B© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  7. 7. Backward Mapping Let x(u, v) and y(u, v) be an inverse mapping from location (x, y) to (u, v): B[u, v] = A[x(u, v), y(u, v)] x u y v A B© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  8. 8. Backward Mapping: Problems • Doesn’t always map from a pixel • Solution: Interpolate between pixels x u y v A B© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  9. 9. Backward Mapping: Problems • May produce holes in the input • Solution: reduce input image (by averaging pixels) and sample reduced/averaged image MIP-maps x u Where did it go? y v A B© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  10. 10. Interpolation • “Filling In” between the pixels • A function of the neighbors or a larger neighborhood • Methods: – Nearest neighbor – Bilinear – Bicubic or other higher-order© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  11. 11. Interpolation: Nearest-Neighbor • Simplest to implement: the output pixel is assigned the value of the pixel that the point falls within • Round off x and y values to nearest pixel • Result is not continuous (blocky)© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  12. 12. Interpolation: Linear (1D) General idea: original function values interpolated values To calculate the interpolated values f(x2)-f(x1)© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  13. 13. Interpolation: Linear (2D) How a 4x4 image would be interpolated to produce an 8x8 image? f ( x, y ) = µ f ( x, y + 1) + (1 − µ ) f ( x, y ) 4 original pixel values f ( x + 1, y ) = µ f ( x + 1, y + 1) + (1 − µ ) f ( x + 1, y ) one interpolated pixel value Along the y’ column we have f ( x , y ) = λ f ( x + 1, y ) + (1 − λ ) f ( x, y )© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  14. 14. Bilinear Interpolation Substituting with the values just obtained: f ( x , y ) = λ ( µ f ( x + 1, y + 1) + (1 − µ ) f ( x + 1, y ) ) + (1 − λ ) ( µ f ( x, y + 1) + (1 − µ ) f ( x, y ) ) You can do the expansion as an exercise. This is the formulation for bilinear interpolation© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  15. 15. Bilinear Interpolation The output pixel value is a weighted average of pixels in the nearest 2-by-2 neighborhood Linearly interpolate in one direction (e.g., vertically) Linearly interpolate results in the other direction (horizontally) Interpolate vertically Interpolate vertically Interpolate horizontally© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  16. 16. General Interpolation We wish to interpolate a value f(x’) for x1 ≤ x ≤ x2 and suppose x − x1 = λ We define an interpolation kernel R(u) and set f ( x ) = R ( −λ ) f ( x1 ) + R (1 − λ ) f ( x2 ) kernel R(u) is centered at x’ x1 corresponds to u= - λ, and x2 to u= 1- λ λ 1− λ© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  17. 17. General Interpolation: 0th and 1st orders Consider 2 functions R0(u) and R1(u) ⎧0 if u ≤ −0.5 ⎪ ⎧1 + u if u ≤ 0 R0 (u ) = ⎨1 if − 0.5 < u ≤ 0.5 R1 (u ) = ⎨ ⎪0 ⎩1 − u if u ≥ 0 ⎩ if u > 0.5 Substitute R0(u) for R(u) nearest-neighbor interpolation. Substitute R1(u) for R(u) linear interpolation.© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  18. 18. Interpolation Kernel Order Zero -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  19. 19. Interpolation Kernel Order Zero Piece-Wise Y= 1 -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  20. 20. Interpolation Kernel Convolution -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  21. 21. Interpolation Kernel Convolution -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  22. 22. Interpolation Kernel -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  23. 23. Interpolation Kernel Order One -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  24. 24. Interpolation Kernel Order One Piece-Wise Y=(X+1) Y=(1-X) -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  25. 25. Interpolation Kernel Convolution -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  26. 26. Interpolation Kernel Convolution -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  27. 27. Interpolation Kernel Order Two -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  28. 28. Interpolation Kernel Order Two Piece-Wise Y = ( 1 – 2 X2 ) Y = ( X + 3/2 )2 / 2 Y = ( X – 3/2 )2 / 2 -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  29. 29. Interpolation Kernel Convolution -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  30. 30. Interpolation Kernel Convolution -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  31. 31. Interpolation Kernel Order Three -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  32. 32. Interpolation Kernel Order Three Piece-Wise Y = ( - 3X3 - 6X2 + 4 )/6 Y = ( 3X3 - 6X2 + 4 )/6 Y = (2+X)3 / 6 Y = (2-X)3 / 6 -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  33. 33. 1D Interpolation Zero Order Nearest Neighbor -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  34. 34. 1D Interpolation Zero Order Nearest Neighbor -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  35. 35. 1D Interpolation Zero Order Nearest Neighbor -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  36. 36. 1D Interpolation Zero Order Nearest Neighbor -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  37. 37. 1D Interpolation First Order Linear Interpolation -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  38. 38. 1D Interpolation First Order Linear Interpolation -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  39. 39. 1D Interpolation First Order Linear Interpolation -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  40. 40. 1D Interpolation First Order Linear Interpolator -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  41. 41. 1D Interpolation Second Order Quadratic Interpolation -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  42. 42. 1D Interpolation Second Order Quadratic Interpolation -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  43. 43. 1D Interpolation Second Order Quadratic Interpolation -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  44. 44. 1D Interpolation Second Order Quadratic Interpolator -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  45. 45. 1D Interpolation Third Order Cubic Interpolation -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  46. 46. 1D Interpolation Third Order Cubic Interpolation -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  47. 47. 1D Interpolation Third Order Cubic Interpolation -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  48. 48. 1D Interpolation Third Order Cubic Interpolation -2 -1 0 1 2© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  49. 49. Remarks About Higher-Order Interpolation • Higher-degree polynomials: – e.g., cubic • Sometimes other interpolating functions • Requires a larger neighborhood: – e.g., bicubic requires a 4×4 neighborhood • More expensive© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  50. 50. Another 3rd order (Cubic) Example ⎧ 1.5 | u |3 −2.5 | u |2 +1 if | u |≤ 1 R3 (u ) = ⎨ ⎩ −0.5 | u |3 +2.5 | u |2 −4 | u | +2 if 1 < | u |≤ 2 Now have 4 support points: f ( x ) = R3 (−1 − λ ) f ( x1 ) + R3 (−λ ) f ( x2 ) + R3 (1 − λ ) f ( x3 ) + R3 (2 − λ ) f ( x4 )© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  51. 51. 2D Interpolation Kernel Product© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  52. 52. 2D Interpolation Kernel Product© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  53. 53. 2D Interpolation Kernel Product x, y separable variables© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  54. 54. Bicubic (2D) Bicubic interpolation fits a series of cubic polynomials to the brightness values contained in the 4 x 4 array of pixels surrounding the calculated address. Step 1: four cubic polynomials F(i), i = 0, 1, 2, 3 are fit to the control points along the rows. The fractional part of the calculated pixels address in the x-direction is used.© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  55. 55. Bicubic (2D) Step 2: the fractional part of the calculated pixels address in the y-direction is used to fit another cubic polynomial down the column, based on the interpolated brightness values that lie on the curves F(i), i = 0, ..., 3.© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  56. 56. Bicubic (2D) • Substituting the fractional part of the calculated pixels address in the x-direction into the resulting cubic polynomial then yields the interpolated pixels brightness value.© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  57. 57. Three Interpolations Comparison • Trade offs: – Aliasing versus blurring – Computation speed nearest neighbor bilinear bicubic©www.cs.princeton.edu/courses/archive/fall00/cs426/lectures/warp/sld024
  58. 58. General Interpolation: Summary For NN interpolation, the output pixel is assigned the value of the pixel that the point falls within. No other pixels are considered. For bilinear interpolation, the output pixel value is a weighted average of pixels in the nearest 2-by-2 neighborhood. For bicubic interpolation, the output pixel value is a weighted average of pixels in the nearest 4-by-4 neighborhood. Bilinear method takes longer than nearest neighbor interpolation, and the bicubic method takes longer than bilinear. The greater the number of pixels considered, the more accurate the computation is, so there is a trade-off between processing time and quality. Only trade-off of higher order methods is edge-preservation. Sometimes hybrid methods are used.© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  59. 59. 2D Geometric Operations
  60. 60. 2D Geometric Operations: TranslationShifting left-right and/or up-down: x′ = x + x0 y′ = y + y0Matrix form: ⎡ x′ ⎤ ⎡ 1 0 x0⎤ ⎡ x ⎤ ⎡x + x0⎤ ⎢ y′ ⎥ = ⎢ 0 1 y0⎥ ⎢ y ⎥ = ⎢y + y0⎥ ⎣1 ⎦ ⎣0 0 1⎦ ⎣ 1 ⎦ ⎣ 1 ⎦Convenient Notation: Homogeneous Coordinates• Add one dimension, treat transformations as matrix multiplication• Can be generalized to 3D © 2003 http://web.engr.oregonstate.edu/~enm/cs519
  61. 61. 2D Geometric Operations: Reflection Reflection Y ⎡t ′ ⎤ ⎡1 0 0⎤ ⎡t x ⎤ x ⎢t ′ ⎥ = ⎢0 − 1 0⎥ ⋅ ⎢t ⎥ ⎢ y⎥ ⎢ ⎥ ⎢ y⎥ ⎢ 1 ⎥ ⎢0 0 1 ⎥ ⎢ 1 ⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦ Reflection X ⎡t ′ ⎤ ⎡ − 1 0 0 ⎤ ⎡t x ⎤ x ⎢t ′ ⎥ = ⎢ 0 1 0⎥ ⋅ ⎢t ⎥ ⎢ y⎥ ⎢ ⎥ ⎢ y⎥ ⎢ 1 ⎥ ⎢ 0 0 1⎥ ⎢ 1 ⎥ ⎣ ⎦ ⎣ ⎦ ⎣ ⎦©www.cs.rug.nl/~rudy/onderwijs/BB/Lecture_7
  62. 62. 2D Geometric Operations: Scaling Enlarging or reducing horizontally and/or vertically: x′ = Sx x y′ = Sy y Matrix form: ⎡ x′⎤ ⎡Sx 0 0 ⎤ ⎡ x ⎤ ⎡ Sx x ⎤ ⎢ y′⎥ = ⎢ 0 Sy 0 ⎥ ⎢ y ⎥ = ⎢ Sy y ⎥ ⎣ 1⎦ ⎣0 0 1⎦ ⎣ 1 ⎦ ⎣ 1 ⎦© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  63. 63. 2D Geometric Operations: Rotation Result components dependent on both x & y: x′ = cos(θ ) x – sin(θ ) y y′ = sin(θ ) x + cos(θ ) y Matrix form: ⎡ x′ ⎤ ⎡ cos(θ ) –sin(θ ) 0⎤ ⎡ x⎤ ⎡ cos(θ ) x – sin(θ ) y⎤ ⎢ y′ ⎥ = ⎢ sin(θ ) cos(θ ) 0⎥ ⎢ y ⎥ = ⎢ sin(θ ) x + cos(θ ) y⎥ ⎣ 1⎦ ⎣ 0 0 1⎦ ⎣ 1⎦ ⎣ 1 ⎦© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  64. 64. Rotation Operation: Problems • In image space, when rotating a collection of points, what could go wrong? positions after rotation (x’,y’) Original position (x,y)© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  65. 65. Rotation Operation: Problems • Problem1: part of rotated image might fall out of valid image range. • Problem2: how to obtain the intensity values in the rotated image? Consider all integer-valued points (x’,y’) in the dashed rectangle. A point will be in the image if, when rotated back, it lies within the original image limits. 0 ≤ x cos θ + y sin θ ≤ a A rectangle surrounding a rotated image 0 ≤ − x sin θ + y cos θ ≤ b See homework assignment 2!© 2005 Charlene Tsai, http://www.cs.ccu.edu.tw/~tsaic/teaching/spring2005_undgrad/dip_lecture6.ppt
  66. 66. 2D Geometric Operations: Affine Transforms Linear combinations of x, y, and 1: encompasses all translation, scaling, & rotation (also skew and shear): x′ = a x + by + c y′ = d x + ey + f Matrix form: ⎡ x′⎤ ⎡a b c ⎤ ⎡ x ⎤ ⎡a x + by + c⎤ ⎢ y′⎥ = ⎢d e f ⎥ ⎢ y ⎥ = ⎢ d x + ey + f ⎥ ⎣ 1⎦ ⎣0 0 1⎦ ⎣ 1 ⎦ ⎣ 1 ⎦© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  67. 67. Affine Transformations (cont.) • Translations and rotations are rigid body transformations • General affine transformations also include non-rigid transformations (e.g., skew or shear) – Affine means that parallel lines transform to parallel lines©www.cs.rug.nl/~rudy/onderwijs/BB/Lecture_7
  68. 68. Compound Transformations Example: rotation around the point (x0, y0) ⎡ 1 0 x0 ⎤ ⎡ cos(θ ) -sin(θ ) 0 ⎤ ⎡ 1 0 -x0⎤ ⎡ x ⎤ ⎢ 0 1 y0 ⎥ ⎢ sin(θ ) cos(θ ) 0 ⎥ ⎢ 0 1 -y0⎥ ⎢ y ⎥ ⎣0 0 1 ⎦ ⎣ 0 0 1⎦ ⎣ 0 0 1 ⎦ ⎣ 1 ⎦ Matrix multiplication is associative (but not commutative): B(Av) = (BA)v = Cv where C = BA – Can compose multiple transformations into a single matrix – Much faster when applying same transform to many pixels© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  69. 69. Compound Transformations Example: y y x x© 2003 http://web.engr.oregonstate.edu/~enm/cs519
  70. 70. Inverting Matrix TransformationsIf v′ = M vthen v = M-1v′Thus, to invert the transformation, invert the matrixUseful for computing the backward mapping given the forward transformFor more info see e.g. http://home.earthlink.net/~jimlux/radio/math/matinv.htm
  71. 71. Morphing: Deformations in 2D and 3D
  72. 72. Morphing: Deformations in 2D and 3D • Parametric Deformations • Cross-Dissolve • Mesh Warping • Control Points
  73. 73. Parametric Deformations
  74. 74. Parametric Deformations - Taper© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  75. 75. Parametric Deformations - Taper© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  76. 76. Parametric Deformations - Twistx’ = s(z) · xy’ = s(z) · yz’ = z (maxz –z) Where s(z) = (maxz – minz)
  77. 77. Parametric Deformations - Twist© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  78. 78. Parametric Deformations - Bend© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  79. 79. Parametric Deformations - Bend© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  80. 80. Parametric Deformations - Compound© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  81. 81. Image Blending
  82. 82. Image Blending• Goal is smooth transformation between image of one object and another• The idea is to get a sequence of intermediate images which when put together with the original images would represent the change from one image to the other• Realized by – Image warping – Color blending• Image blending has been widely used in creating movies, music videos and television commercials – Terminator 2
  83. 83. Cross-Dissolve (Cross-Fading)• Simplest approach is cross-dissolve: – linear interpolation to fade from one image (or volume) to another• No geometrical alignment between images (or volumes)• Pixel-by-pixel (or voxel by voxel) interpolation• No smooth transitions, intermediate states not realistic from G. Wolberg, CGI ‘96
  84. 84. Problems• Problem with cross-dissolve is that if features don’t line up exactly, we get a double image• Can try shifting/scaling/etc. one entire image to get better alignment, but this doesn’t always fix problem• Can handle more situations by applying different warps to different pieces of image – Manually chosen – Takes care of feature correspondences Image IS with mesh Image IT, mesh MT MS defining pieces from G. Wolberg, CGI ‘96
  85. 85. Mesh Warping
  86. 86. Mesh Warping Application Images IS& meshes MS Morphed images If Images IT& meshes MS from G. Wolberg, CGI ‘96
  87. 87. Mesh Warping y y Transform x x Fixed Image Moving Image© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  88. 88. Mesh Warping y y Transform x x Fixed Image Moving Image© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  89. 89. Mesh Warping• Source and target images are meshed• The meshes for both images are interpolated• The intermediate images are cross-dissolved• Here, we look at 2D example
  90. 90. Mesh Warping Algorithm• Algorithm for each frame f do – interpolate mesh M, between Ms and Mt – warp Image Is to I1, using meshes Ms and M – warp Image It to I2, using meshes Mt and M – interpolate image I1 and I2 end – Is : source image, It : target image – source image has mesh Ms, target image has mesh Mt
  91. 91. Mesh Deformation y x© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  92. 92. Mesh Deformation y x© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  93. 93. Mesh Deformation© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  94. 94. Mesh Deformation For each vertex identify cell, 0.8 fractional u,v coordinate 0.5 in unit cell© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  95. 95. Mesh Deformation P11 E.g. bilinear interpolation Pu1 Pu0 = (1-u)*P00 + u*P10 P01 Pu1 = (1-u)*P01 + u*P11 Puv = (1-v)*P0u + v*P1u P00 Pu0 P01© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  96. 96. Mesh Deformation© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  97. 97. Mesh Deformation© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  98. 98. Free-Form Deformations Define local coordinate system for deformation T U S (not necessarily orthogonal)© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  99. 99. FFD – Register Point in Cell T U S© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  100. 100. FFD – Register Point in Cell T (TxU) . (P-P0) ((TxU) . S) P TxU U S P0 s = (TxU) . (P-P0) / ((TxU) . S) P = P0 + sS + tT + uU© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  101. 101. FFD – Create Control Grid (not necessarily orthogonal)© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  102. 102. FFD – Move and Reposition Move control grid points Usually tri-cubic interpolation is used with FFDs Originally, Bezier interpolation was used. B-spline and Catmull-Romm interpolation have also been used (as well as tri-linear interpolation)© Rick Parent, www.cse.ohio-state.edu/~parent/classes/682/Lectures/L06_Deformations
  103. 103. FFD Example Step1 It is originally a cylinder. Red boundary is FFD block embedded with that cylinder. Step2 Move control points of each end, and you can see cylinder inside also changes.© http://www.comp.nus.edu.sf/~cs3246/CS3246_01/anim4.ppt
  104. 104. FFD Example step3 Move inner control points downwards. step4 Finally, get a shaded version of banana!© http://www.comp.nus.edu.sf/~cs3246/CS3246_01/anim4.ppt
  105. 105. BSplines (Cubic) Interpolation Original Lena© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  106. 106. BSplines (Cubic) Interpolation Deformed with BSpline Transform© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  107. 107. Deformable Registration Framework Fixed Image Metric Interpolator Optimizer Moving Image Transform Parameters Array© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  108. 108. Deformable Registration Deformed with BSpline Transform© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  109. 109. Deformable Registration Registered with BSpline Transform© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  110. 110. Deformable Registration Original Lena© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  111. 111. Deformable Registration Difference Before Difference After Registration Registration© 2004, Chuck Stewart & Luis Ibanez, http://www.cs.rpi.edu/courses/spring04/imagereg/lectureBSplines.ppt
  112. 112. Control Point Warping
  113. 113. Control Point WarpingInstead of a warping mesh, use arbitrary correspondence points: Tip of one person’s nose to the tip of another, eyes to eyes, etc.Interpolate between correspondence points to determine how points moveApply standard warping (forward or backward): In-between image is a weighted average of the source and destination corresponding pixelsHere we look at 3D example…
  114. 114. Finding Control Points in 3D Structures AlternativeActin filament: Reconstruction from EM data at 20Å resolution rmsd: 1.1Å Wriggers et al., J. Molecular Biology, 1998, 284: 1247-1254
  115. 115. Control Point DisplacementsHave 2 conformations,both source and targetcharacterized bycontrol points RNA Polymerase, Wriggers, Structure, 2004, Vol. 12, pp. 1-2.
  116. 116. Piecewise-Linear Inter- / Extrapolation For each probe position find 4 closest control points. w 3 f3 Ansatz: Fx ( x, y, z ) = ax + by + cz + d w 4 f4 F = ( Fx , Fy , Fz ) Fx (w1 ) = f1, x , Fx (w 2 ) = f 2, x , w1 f1 w 2 f2 Fx (w 3 ) = f3, x , Fx (w 4 ) = f 4, x (similar for Fy , Fz ). Cramer’s rule: f1, x w1, y w1, z 1 w1, x f1, y w1, z 1 f 2, x w2, y w2, z 1 w2, x f 2, y w2, z 1 w1, x w1, y w1, z 1 f 3, x w3, y w3, z 1 w3, x f3, y w3, z 1 w2, x w2, y w2, z 1 D= f 4, x w4, y w4, z 1 w4, x f 4, y w4, z 1 w3, x w3, y w3, z 1a= , b= , , w w4, y w4, z 1 D D 4, x See e.g. http://mathworld.wolfram.com/CramersRule.html
  117. 117. Non-Linear Kernel InterpolationConsider all k control points and interpolation kernel function U(r).Ansatz: k Fx ( x, y, z ) = a1 + ax x + a y y + az z + ∑ bi ⋅ U ( w i − ( x, y, z ) ) k =1 Fx (w i ) = fi , x , ∀i (similar for Fy , Fz ).Solve : L−1 ( f1, x , , f k , x , 0, 0, 0, 0) = (b1 , , bk , a1 , ax , a y , az )T , ⎛1 w1, x w1, y w1, z ⎞ ⎛ P Q⎞ ⎜ ⎟ where L = ⎜ T ⎜Q ⎟, Q=⎜ ⎟ , k × 4, ⎝ 0⎟ ⎠ ⎜1 wk , x wk , y wk , z ⎟ ⎝ ⎠ ⎛ 0 U ( w12 ) U ( w1k ) ⎞ ⎜ ⎟ P= ⎜ U ( w21 ) 0 U ( w2 k ) ⎟ , k × k. ⎜ ⎟ ⎜ ⎜U (w ) U (w ) ⎟ ⎝ k1 k2 0 ⎟ ⎠
  118. 118. Bookstein “Thin-Plate” Splines • kernel function U(r) is principal solution of biharmonic equation that arises in elasticity theory of thin plates: ∆ U (r ) = ∇ U (r ) = δ(r ). 2 4 • variational principle: U(r) minimizes the bending energy (not shown). • 1D: U(r) = |r3| (cubic spline) • 2D: U(r) = r2 log r2 • 3D: U(r) = |r| 2D: U(r) F ( x, y )See Bookstein, Morphometric Tools for Landmark Data, Cambridge U Press, 1997
  119. 119. RNAP Example: Source Structure
  120. 120. Piecewise-Linear Inter- / Extrapolation
  121. 121. Thin-Plate Splines, 3D |r| Kernel
  122. 122. Control: Molecular Dynamics
  123. 123. ResourcesTextbooks:Kenneth R. Castleman, Digital Image Processing, Chapter 8John C. Russ, The Image Processing Handbook, Chapter 3Online Graphics Animations:http://nis-lab.is.s.u-tokyo.ac.jp/~nis/animation.html

×