T H E U N I V E R S I T Y of T E X A S HEALTH SCIENCE CENTER AT HOUSTON S C H O O L of H E A L T H I N F O R M A T I O N S C I E N C E S Interpolation and MorphingFor students of HI 5323“Image Processing”Willy Wriggers, Ph.D.School of Health Information Scienceshttp://biomachina.org/courses/processing/05.html
Inverting Matrix TransformationsIf v′ = M vthen v = M-1v′Thus, to invert the transformation, invert the matrixUseful for computing the backward mapping given the forward transformFor more info see e.g. http://home.earthlink.net/~jimlux/radio/math/matinv.htm
Image Blending• Goal is smooth transformation between image of one object and another• The idea is to get a sequence of intermediate images which when put together with the original images would represent the change from one image to the other• Realized by – Image warping – Color blending• Image blending has been widely used in creating movies, music videos and television commercials – Terminator 2
Cross-Dissolve (Cross-Fading)• Simplest approach is cross-dissolve: – linear interpolation to fade from one image (or volume) to another• No geometrical alignment between images (or volumes)• Pixel-by-pixel (or voxel by voxel) interpolation• No smooth transitions, intermediate states not realistic from G. Wolberg, CGI ‘96
Problems• Problem with cross-dissolve is that if features don’t line up exactly, we get a double image• Can try shifting/scaling/etc. one entire image to get better alignment, but this doesn’t always fix problem• Can handle more situations by applying different warps to different pieces of image – Manually chosen – Takes care of feature correspondences Image IS with mesh Image IT, mesh MT MS defining pieces from G. Wolberg, CGI ‘96
Mesh Warping• Source and target images are meshed• The meshes for both images are interpolated• The intermediate images are cross-dissolved• Here, we look at 2D example
Mesh Warping Algorithm• Algorithm for each frame f do – interpolate mesh M, between Ms and Mt – warp Image Is to I1, using meshes Ms and M – warp Image It to I2, using meshes Mt and M – interpolate image I1 and I2 end – Is : source image, It : target image – source image has mesh Ms, target image has mesh Mt
Control Point WarpingInstead of a warping mesh, use arbitrary correspondence points: Tip of one person’s nose to the tip of another, eyes to eyes, etc.Interpolate between correspondence points to determine how points moveApply standard warping (forward or backward): In-between image is a weighted average of the source and destination corresponding pixelsHere we look at 3D example…
Finding Control Points in 3D Structures AlternativeActin filament: Reconstruction from EM data at 20Å resolution rmsd: 1.1Å Wriggers et al., J. Molecular Biology, 1998, 284: 1247-1254
Control Point DisplacementsHave 2 conformations,both source and targetcharacterized bycontrol points RNA Polymerase, Wriggers, Structure, 2004, Vol. 12, pp. 1-2.
Piecewise-Linear Inter- / Extrapolation For each probe position find 4 closest control points. w 3 f3 Ansatz: Fx ( x, y, z ) = ax + by + cz + d w 4 f4 F = ( Fx , Fy , Fz ) Fx (w1 ) = f1, x , Fx (w 2 ) = f 2, x , w1 f1 w 2 f2 Fx (w 3 ) = f3, x , Fx (w 4 ) = f 4, x (similar for Fy , Fz ). Cramer’s rule: f1, x w1, y w1, z 1 w1, x f1, y w1, z 1 f 2, x w2, y w2, z 1 w2, x f 2, y w2, z 1 w1, x w1, y w1, z 1 f 3, x w3, y w3, z 1 w3, x f3, y w3, z 1 w2, x w2, y w2, z 1 D= f 4, x w4, y w4, z 1 w4, x f 4, y w4, z 1 w3, x w3, y w3, z 1a= , b= , , w w4, y w4, z 1 D D 4, x See e.g. http://mathworld.wolfram.com/CramersRule.html
Non-Linear Kernel InterpolationConsider all k control points and interpolation kernel function U(r).Ansatz: k Fx ( x, y, z ) = a1 + ax x + a y y + az z + ∑ bi ⋅ U ( w i − ( x, y, z ) ) k =1 Fx (w i ) = fi , x , ∀i (similar for Fy , Fz ).Solve : L−1 ( f1, x , , f k , x , 0, 0, 0, 0) = (b1 , , bk , a1 , ax , a y , az )T , ⎛1 w1, x w1, y w1, z ⎞ ⎛ P Q⎞ ⎜ ⎟ where L = ⎜ T ⎜Q ⎟, Q=⎜ ⎟ , k × 4, ⎝ 0⎟ ⎠ ⎜1 wk , x wk , y wk , z ⎟ ⎝ ⎠ ⎛ 0 U ( w12 ) U ( w1k ) ⎞ ⎜ ⎟ P= ⎜ U ( w21 ) 0 U ( w2 k ) ⎟ , k × k. ⎜ ⎟ ⎜ ⎜U (w ) U (w ) ⎟ ⎝ k1 k2 0 ⎟ ⎠
Bookstein “Thin-Plate” Splines • kernel function U(r) is principal solution of biharmonic equation that arises in elasticity theory of thin plates: ∆ U (r ) = ∇ U (r ) = δ(r ). 2 4 • variational principle: U(r) minimizes the bending energy (not shown). • 1D: U(r) = |r3| (cubic spline) • 2D: U(r) = r2 log r2 • 3D: U(r) = |r| 2D: U(r) F ( x, y )See Bookstein, Morphometric Tools for Landmark Data, Cambridge U Press, 1997
ResourcesTextbooks:Kenneth R. Castleman, Digital Image Processing, Chapter 8John C. Russ, The Image Processing Handbook, Chapter 3Online Graphics Animations:http://nis-lab.is.s.u-tokyo.ac.jp/~nis/animation.html
A particular slide catching your eye?
Clipping is a handy way to collect important slides you want to go back to later.