Models and Matching


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Models and Matching

  1. 1. Models and Matching Methods of modeling objects and their environments; Methods of matching models to sensed data for recogniton
  2. 2. Some methods to study <ul><li>Mesh models ( surface ) </li></ul><ul><li>Vertex-edge-face models ( surface ) </li></ul><ul><li>Functional forms: superquadrics ( surface ) </li></ul><ul><li>Generalized cylinders ( volume ) </li></ul><ul><li>Voxel sets and octrees ( volume ) </li></ul><ul><li>View class models ( image-based ) </li></ul><ul><li>Recognition by appearance ( image-based ) </li></ul><ul><li>Functional models and the Theory of affordances ( object-oriented ) </li></ul>
  3. 3. Models are what models do
  4. 4. What do models do?
  5. 5. Vertex-edge-face models Polyhedra and extensions; Model the surface of objects
  6. 6. Vertex-Edge-Face model
  7. 7. Sample object All surfaces are planar or cylindrical
  8. 8. Matching methods <ul><li>Hypothesize point correspondences </li></ul><ul><li>Filter on distances </li></ul><ul><li>Compute 3D alignment of model to data </li></ul><ul><li>Verify positions of other model points, edges, or faces. You can now do this! </li></ul><ul><li>LOTS of work in the literature on this! Can work for many industrial objects (and human faces perhaps!) </li></ul>
  9. 9. Triangular meshes Very general and used by most CAD systems.
  10. 10. Texture-mapped mesh dog Courtesy of Kari Puli With each triangle is a mapping of its vertices into pixels [r, c] of a color image. Thus any point of any triangle can be assigned a color [R, G, B]. There may be several images available to create these mappings. 3D SURFACE MODEL SURFACE PLUS TEXTURE
  11. 11. Meshes are very general They are usually verbose and often are too detailed for many operations, but are often used in CAD. (Volumetric cube models are actually displayed here: made from many views by Kari Pulli.)
  12. 12. Modeling the human body for clothing industry and … Multiple Structured light scanners used: could this be a service industry such as Kinkos? Actually cross sections of a generalized cylinder model.
  13. 13. Mesh characteristics + can be easy to generate from scanned data
  14. 14. Making mesh models
  15. 15. Marching cubes (James Sharman) &quot;Marching Cubes: A High Resolution 3D Surface Construction Algorithm&quot;, William E. Lorensen and Harvey E. Cline, Computer Graphics (Proceedings of SIGGRAPH '87), Vol. 21, No. 4, pp. 163-169. Raster scan through image F(r, c). Look for adjacent pixels, one above threshold and one below threshold. Interpolate real coordinates for f(x, y) = t in between
  16. 16. Marching in 3D space F(s, r, c) Some voxel corners are above threshold t and some are below.
  17. 17. PhD work by Paul Albee 2004 <ul><li>Used Argonne National Labs scanner </li></ul><ul><li>High energy, high resolution planar Xrays penetrate object rotating on a turntable </li></ul><ul><li>Computer aided tomography synthesizes a 3D volume of densities with voxel size of about 5 microns </li></ul>
  18. 18. Segmentation of Scutigera a tiny crablike organism Slice j of material density F( sj, r, c ) “ thresholded” volume
  19. 19. Some common 3D problems <ul><li>analyze blood vessel structure in head </li></ul><ul><li>capture structure and motion of vertebrae </li></ul><ul><li>of spine </li></ul><ul><li>analyze porosity and structure of soil </li></ul><ul><li>analyze structure of materials </li></ul><ul><li>automatic segmentation into regions </li></ul><ul><li>automatic correspondence of 3D points at </li></ul><ul><li>two instants of of time </li></ul><ul><li>3D volume visualization and virtual tours </li></ul>
  20. 20. Scanning technique abstraction CCD camera (row) material sample X-ray planes scintillator Pin head rotate X-rays partly absorbed by sample; excite scintillator producing one row in the camera image; rotate sample a few degrees and produce another row; 3D reconstruction using CT
  21. 21. Scutigera: a tiny crustacean <ul><li>organism is smaller than 1 mm </li></ul><ul><li>scanned at Argonne </li></ul><ul><li>volume segmented and </li></ul><ul><li>meshed by Paul Albee </li></ul><ul><li>roughly ten million triangles </li></ul><ul><li>to represent the surface </li></ul><ul><li>anaglyph created for 3D </li></ul><ul><li>visualization </li></ul><ul><li>(view with stereo glasses) </li></ul>
  22. 22. Presentation of Results to User <ul><li>Can explore the 3D data using rotation/translation </li></ul><ul><li>Can create stereo images from 3D data </li></ul>
  23. 23. Physics-based models Can be used to make meshes; Meshes retain perfect topology; Can span spots of bad or no data
  24. 24. Physics-based modeling
  25. 25. Forces move points on the model; halt at scanned data
  26. 26. Fitting an active contour to image data
  27. 27. Balloon model for closed object surface Courtesy of Chen and Medioni
  28. 28. Balloon evolution <ul><li>balloon stops at data points </li></ul><ul><li>mesh forces constrain neighbors </li></ul><ul><li>large triangles split into 4 triangles </li></ul><ul><li>resulting mesh has correct topology </li></ul><ul><li>hard CS part is detecting when balloon should be stopped by data point </li></ul>
  29. 29. Physics-based models Can also model dynamic behavior of solids (Finite Element Methods)
  30. 30. Tagged MRI: 3D interest points can be written to body! The MRI sensor tags living tissue and can sense its movement. Motion of a 3D tetrahedral finite elements model can then be analyzed. FMA model attempts to model the real physics of the heart. Work by Jinah Park and Dimitry Metaxes.
  31. 31. Algorithms from computer graphics make mesh models from blobs <ul><li>Marching squares applied to some connected image region (blob) </li></ul><ul><li>Marching cubes applied to some connected set of voxels (blob) </li></ul><ul><li>See a CG text for algorithms: see the visualization toolkit for software </li></ul>
  32. 32. The octree for compression
  33. 33. Generalized cylinders
  34. 34. Generalized cylinders <ul><li>component parts have axis </li></ul><ul><li>cross section function describes variation along axis </li></ul><ul><li>good for articulated objects, such as animals, tools </li></ul><ul><li>can be extracted from intensity images with difficulty </li></ul>
  35. 35. Extracting a model from a segmented image region Courtesy of Chen and Medioni
  36. 36. Interpreting frames from video <ul><li>Can we match a frame region to a model? </li></ul><ul><li>What about a sequence of frames? </li></ul><ul><li>Can we determine what actions the body is doing? </li></ul>
  37. 37. Generalized cylinders
  38. 38. View class models Objects modeled by the distinct views that they can produce
  39. 39. “aspect model” of a cube
  40. 40. Recognition using an aspect model
  41. 41. View class model of chair 2D Graph-matching (as in Ch 11) used to evaluate match.
  42. 42. Side view classes of Ford Taurus (Chen and Stockman) These were made in the PRIP Lab from a scale model. Viewpoints in between can be generated from x and y curvature stored on boundary. Viewpoints matched to real image boundaries via optimization.
  43. 43. Matching image edges to model limbs Could recognize car model at stoplight or gate or in car wash.
  44. 44. Appearance-based models Using a basis of sub images; Using PCA to compress bases; Eigenfaces ( see older .pdf slides 14C)
  45. 45. Function-based modeling Object-oriented; What parts does the object have; What behaviors does it have; What can be done with it? (See plastic slides of Louise Starks’s work.)
  46. 46. Louise Stark: chair model <ul><li>Dozens of CAD models of chairs </li></ul><ul><li>Program analyzes model for </li></ul><ul><li>* stable pose </li></ul><ul><li>* seat of right size </li></ul><ul><li>* height off ground right size </li></ul><ul><li>* no obstruction to body on seat </li></ul><ul><li>* program would accept a trash can </li></ul><ul><li>(which could also pass as a container) </li></ul>
  47. 47. Theory of affordances: J.J. Gibson <ul><li>An object can be “sittable”: a large number of chair types, a box of certain size, a trash can turned over, … </li></ul><ul><li>An object can be “walkable”: the floor, ground, thick ice, bridge, ... </li></ul><ul><li>An object can be a “container”: a cup, a hat, a barrel, a box, … </li></ul><ul><li>An object can be “throwable”: a ball, a book, a coin, an apple, a small chair, … </li></ul>
  48. 48. Minski’s theory of frames (Schank’s theory of scripts) <ul><li>Frames are learned expectations – frame for a room, a car, a party, an argument, … </li></ul><ul><li>Frame is evoked by current situation – how? (hard) </li></ul><ul><li>Human “fills in” the details of the current frame (easier) </li></ul>
  49. 49. Make a frame for my house <ul><li>Item 1 </li></ul><ul><li>Item 2 </li></ul><ul><li>Item 3 </li></ul><ul><li>Item 4 </li></ul><ul><li>Item 5 </li></ul><ul><li>Item 6 </li></ul>