Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Galilean Differential Geometry Of Moving Images


Published on

Presented at ECCV2004 (as a poster) and at Institut Mittag-Leffler 2003.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Galilean Differential Geometry Of Moving Images

  1. 1. Galilean Differential Geometry of Moving Images Daniel Fagerström CVAP/NADA/KTH [email_address]
  2. 2. Differential Structure of Movies <ul><li>How can we describe the local structure of an image sequence? </li></ul><ul><li>We will assume that a movie is a smooth function of 2+1 dimensional space-time </li></ul><ul><li>Looking for generic properties </li></ul>
  3. 3. Approaches for Motion Analysis <ul><li>Optical flow </li></ul><ul><li>Spatio-temporal texture </li></ul><ul><li>Spatio-temporal differential invariants </li></ul>
  4. 4. Optical Flow <ul><li>Geometry of the projected motion of particles in the observers field of view </li></ul><ul><li>Binding hypothesis needed to use it on image sequences </li></ul><ul><li>“ Top down” </li></ul><ul><li>Local formulation, but non-local, due to binding hypothesis </li></ul><ul><li>Undefined when particles appears and disappears, e.g. motion boundaries </li></ul><ul><li>Does not use image structure </li></ul>
  5. 5. Spatio-Temporal Differential Invariants <ul><li>Local geometry of spatio-temporal images </li></ul><ul><li>” Bottom Up”, low level </li></ul><ul><li>No binding hypotheses, connection to the environment considered as a higher level problem </li></ul><ul><li>Well defined everywhere </li></ul><ul><li>Does use image structure, extension of low level vision for still images </li></ul>
  6. 6. Overview <ul><li>Galilean geometry </li></ul><ul><li>Moving frames </li></ul><ul><li>Image geometry </li></ul><ul><li>1+1 dimensional Galilean differential invariants </li></ul><ul><li>2+1 dimensional Galilean differential invariants </li></ul><ul><li>What is required for a more realistic movie model </li></ul>
  7. 7. Galilean Geometry <ul><li>Spatial and temporal translation a , spatial rotation R and spatio-temporal shear v </li></ul>Shear x t x t
  8. 8. Galilean Geometry <ul><li>Insensitive to constant relative motion for parallel projection, approximately otherwise </li></ul><ul><li>Simplest meaning full model </li></ul><ul><li>Assumed implicitly when one talk about optical flow invariants: div, rot, dev, i.e. first order flow </li></ul><ul><li>Shape properties from the environment can be derived from relative motion </li></ul><ul><li>(Newton physics describe Galilean invariants) </li></ul>
  9. 9. Galilean Invariants <ul><li>Planes of simultaneity (constant t ) are invariant and has Euclidean geometry: distances and angles are invariants </li></ul><ul><ul><li>i.e. an image sequence </li></ul></ul><ul><li>The temporal distance between planes of simultaneity is an invariant </li></ul>
  10. 10. Galilean ON-System <ul><li>An n+1 dimensional Galilean ON-system (e 1 ,e 2 ,  ,e 0 ) is s.t. (e 1 ,e 2 ,  ,e n ) is an Euclidean ON-system and ||e 0 || T =1 </li></ul>
  11. 11. Moving Frames <ul><li>Galilean geometry has no metric </li></ul><ul><li>We will use Cartan's method of moving frames, that does not require a metric </li></ul><ul><li>Moving frame: e:M ! G ½ GL(n) </li></ul><ul><li>Attach a frame that is adapted to the local structure in each point </li></ul><ul><li>Differential geometry: the local change of the frame: de </li></ul>
  12. 12. Moving Frames C(A) contains the differential geometric invariants expressed in the global frame i
  13. 13. Image Geometry <ul><li>Image space: E 2 ­ I - trivial fiber bundle with Euclidian base space and log intensity as fiber (Koenderink 02) z=f(x,y) </li></ul><ul><li>f is smooth </li></ul><ul><li>Image geometry </li></ul><ul><ul><li>Global gray level transformations </li></ul></ul><ul><ul><li>Lightness gradients </li></ul></ul>
  14. 14. Gradient Gauge <ul><li>For points where r f  0 we can choose an adapted ON-frame {  u ,  v } s.t. f u =0 </li></ul><ul><li>All functions over  i u  j v f, i+j ¸ 1 becomes invariants w.r.t. rotation in space and translation in intensity </li></ul>
  15. 15. Gray Level Invariants
  16. 16. Hessian Gauge <ul><li>For points where r f  0 we can choose an ON-frame {  p ,  q } s.t. f pq =0 and |f pp |>|f qq | </li></ul><ul><li>All functions over  i p  j q f, i+j ¸ 2 becomes invariants w.r.t. rotation in space, translation in intensity and addition of a linear light gradient (Koenderink 02) </li></ul>
  17. 17. Galilean 1+1 D <ul><li>Two cases: </li></ul><ul><ul><li>Isophotes cut the spatial line, motion according to the constant brightness assumption </li></ul></ul><ul><ul><li>Isophotes are tangent to the spatial line (along curves), creation, annihilation </li></ul></ul>
  18. 18. Tangent Gauge <ul><li>Let {  t ,  x } be a global Galilean ON-frame, for points where f x  0 we can define an adapted Galilean ON-frame {  s ,  x } s.t. f s =0. </li></ul>
  19. 19. Isophote Invariants
  20. 20. Hessian Gauge <ul><li>Let {  t ,  x } be a global Galilean ON-frame, for points where f xx  0 we can define an adapted Galilean ON-frame {  r ,  x } s.t. f rx =0 . </li></ul>
  21. 21. Hessian Invariants
  22. 22. Galilean 2+1 D <ul><li>General case </li></ul><ul><li>Also here are two different main cases </li></ul><ul><ul><li>Isophote surfaces transversal to the spatial plane. Motion of isophote curves in the image </li></ul></ul><ul><ul><li>Isophote surfaces tangent to the plane. Creation, annihilation and saddle points </li></ul></ul>
  23. 23. Invariants in the General Case <ul><li>a u , a v - acceleration </li></ul><ul><li> u ,  v - divergence </li></ul><ul><li> u ,  v - skew of the ”flow field” </li></ul><ul><li>- rotation of the plane in the temporal direction </li></ul><ul><li> u ,  v - flow line curvature in the plane </li></ul>
  24. 24. More Descriptive Invariants <ul><li>D - rate of strain tensor for the spatio-temporal part of the frame field </li></ul><ul><li>a, curl D, div D, def D - are flow field invariants </li></ul><ul><li>a  ,  ,  ,  u ,  v - are not flow field invariants </li></ul>
  25. 25. Tangent Gauge <ul><li>Let {  t ,  x ,  y } be a global Galilean ON-frame, for points where ||{ f x ,f y }||  0 we can define an adapted Galilean ON-frame {  s ,  u ,  v } s.t. f s =f u =f su =0 </li></ul><ul><li>Principal acceleration extrema </li></ul><ul><li>Direction of  u constant along  s – used in Guichard (98) </li></ul>
  26. 26. Tangent Gauge
  27. 27. Hessian Gauge <ul><li>Let {  t ,  x ,  y } be a global Galilean ON-frame, we define an adapted Galilean ON-frame {  r ,  p ,  q } s.t. f pq = f rp = f rq =0 . </li></ul><ul><li>Also defined when the spatial tangent disappears, e.g. for creation and disappearance of structure </li></ul><ul><li> r is the same vector field as when the optical flow constraint equation is solved for the spatial image gradient </li></ul>
  28. 28. Hessian Gauge
  29. 29. Real Image Sequences <ul><li>Localized filters are not invariant w.r.t. Galilean shear, velocity adapted (  s – directed) filters are needed </li></ul><ul><li>More generic singular cases for imprecise measurements </li></ul>
  30. 30. Conclusion <ul><li>Theory about differential invariants for smooth Galilean spatio-temporal image sequences </li></ul><ul><li>Local operators </li></ul><ul><li>“ Bottom up” </li></ul><ul><li>Contains more information about the image sequence than optical flow </li></ul><ul><li>Extension of methods for still images </li></ul>