Your SlideShare is downloading. ×

Models and Matching


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Models and Matching Methods of modeling objects and their environments; Methods of matching models to sensed data for recogniton
  • 2. Some methods to study
    • Mesh models ( surface )
    • Vertex-edge-face models ( surface )
    • Functional forms: superquadrics ( surface )
    • Generalized cylinders ( volume )
    • Voxel sets and octrees ( volume )
    • View class models ( image-based )
    • Recognition by appearance ( image-based )
    • Functional models and the Theory of affordances ( object-oriented )
  • 3. Models are what models do
  • 4. What do models do?
  • 5. Vertex-edge-face models Polyhedra and extensions; Model the surface of objects
  • 6. Vertex-Edge-Face model
  • 7. Sample object All surfaces are planar or cylindrical
  • 8. Matching methods
    • Hypothesize point correspondences
    • Filter on distances
    • Compute 3D alignment of model to data
    • Verify positions of other model points, edges, or faces. You can now do this!
    • LOTS of work in the literature on this! Can work for many industrial objects (and human faces perhaps!)
  • 9. Triangular meshes Very general and used by most CAD systems.
  • 10. Texture-mapped mesh dog Courtesy of Kari Puli With each triangle is a mapping of its vertices into pixels [r, c] of a color image. Thus any point of any triangle can be assigned a color [R, G, B]. There may be several images available to create these mappings. 3D SURFACE MODEL SURFACE PLUS TEXTURE
  • 11. Meshes are very general They are usually verbose and often are too detailed for many operations, but are often used in CAD. (Volumetric cube models are actually displayed here: made from many views by Kari Pulli.)
  • 12. Modeling the human body for clothing industry and … Multiple Structured light scanners used: could this be a service industry such as Kinkos? Actually cross sections of a generalized cylinder model.
  • 13. Mesh characteristics + can be easy to generate from scanned data
  • 14. Making mesh models
  • 15. Marching cubes (James Sharman) "Marching Cubes: A High Resolution 3D Surface Construction Algorithm", William E. Lorensen and Harvey E. Cline, Computer Graphics (Proceedings of SIGGRAPH '87), Vol. 21, No. 4, pp. 163-169. Raster scan through image F(r, c). Look for adjacent pixels, one above threshold and one below threshold. Interpolate real coordinates for f(x, y) = t in between
  • 16. Marching in 3D space F(s, r, c) Some voxel corners are above threshold t and some are below.
  • 17. PhD work by Paul Albee 2004
    • Used Argonne National Labs scanner
    • High energy, high resolution planar Xrays penetrate object rotating on a turntable
    • Computer aided tomography synthesizes a 3D volume of densities with voxel size of about 5 microns
  • 18. Segmentation of Scutigera a tiny crablike organism Slice j of material density F( sj, r, c ) “ thresholded” volume
  • 19. Some common 3D problems
    • analyze blood vessel structure in head
    • capture structure and motion of vertebrae
    • of spine
    • analyze porosity and structure of soil
    • analyze structure of materials
    • automatic segmentation into regions
    • automatic correspondence of 3D points at
    • two instants of of time
    • 3D volume visualization and virtual tours
  • 20. Scanning technique abstraction CCD camera (row) material sample X-ray planes scintillator Pin head rotate X-rays partly absorbed by sample; excite scintillator producing one row in the camera image; rotate sample a few degrees and produce another row; 3D reconstruction using CT
  • 21. Scutigera: a tiny crustacean
    • organism is smaller than 1 mm
    • scanned at Argonne
    • volume segmented and
    • meshed by Paul Albee
    • roughly ten million triangles
    • to represent the surface
    • anaglyph created for 3D
    • visualization
    • (view with stereo glasses)
  • 22. Presentation of Results to User
    • Can explore the 3D data using rotation/translation
    • Can create stereo images from 3D data
  • 23. Physics-based models Can be used to make meshes; Meshes retain perfect topology; Can span spots of bad or no data
  • 24. Physics-based modeling
  • 25. Forces move points on the model; halt at scanned data
  • 26. Fitting an active contour to image data
  • 27. Balloon model for closed object surface Courtesy of Chen and Medioni
  • 28. Balloon evolution
    • balloon stops at data points
    • mesh forces constrain neighbors
    • large triangles split into 4 triangles
    • resulting mesh has correct topology
    • hard CS part is detecting when balloon should be stopped by data point
  • 29. Physics-based models Can also model dynamic behavior of solids (Finite Element Methods)
  • 30. Tagged MRI: 3D interest points can be written to body! The MRI sensor tags living tissue and can sense its movement. Motion of a 3D tetrahedral finite elements model can then be analyzed. FMA model attempts to model the real physics of the heart. Work by Jinah Park and Dimitry Metaxes.
  • 31. Algorithms from computer graphics make mesh models from blobs
    • Marching squares applied to some connected image region (blob)
    • Marching cubes applied to some connected set of voxels (blob)
    • See a CG text for algorithms: see the visualization toolkit for software
  • 32. The octree for compression
  • 33. Generalized cylinders
  • 34. Generalized cylinders
    • component parts have axis
    • cross section function describes variation along axis
    • good for articulated objects, such as animals, tools
    • can be extracted from intensity images with difficulty
  • 35. Extracting a model from a segmented image region Courtesy of Chen and Medioni
  • 36. Interpreting frames from video
    • Can we match a frame region to a model?
    • What about a sequence of frames?
    • Can we determine what actions the body is doing?
  • 37. Generalized cylinders
  • 38. View class models Objects modeled by the distinct views that they can produce
  • 39. “aspect model” of a cube
  • 40. Recognition using an aspect model
  • 41. View class model of chair 2D Graph-matching (as in Ch 11) used to evaluate match.
  • 42. Side view classes of Ford Taurus (Chen and Stockman) These were made in the PRIP Lab from a scale model. Viewpoints in between can be generated from x and y curvature stored on boundary. Viewpoints matched to real image boundaries via optimization.
  • 43. Matching image edges to model limbs Could recognize car model at stoplight or gate or in car wash.
  • 44. Appearance-based models Using a basis of sub images; Using PCA to compress bases; Eigenfaces ( see older .pdf slides 14C)
  • 45. Function-based modeling Object-oriented; What parts does the object have; What behaviors does it have; What can be done with it? (See plastic slides of Louise Starks’s work.)
  • 46. Louise Stark: chair model
    • Dozens of CAD models of chairs
    • Program analyzes model for
    • * stable pose
    • * seat of right size
    • * height off ground right size
    • * no obstruction to body on seat
    • * program would accept a trash can
    • (which could also pass as a container)
  • 47. Theory of affordances: J.J. Gibson
    • An object can be “sittable”: a large number of chair types, a box of certain size, a trash can turned over, …
    • An object can be “walkable”: the floor, ground, thick ice, bridge, ...
    • An object can be a “container”: a cup, a hat, a barrel, a box, …
    • An object can be “throwable”: a ball, a book, a coin, an apple, a small chair, …
  • 48. Minski’s theory of frames (Schank’s theory of scripts)
    • Frames are learned expectations – frame for a room, a car, a party, an argument, …
    • Frame is evoked by current situation – how? (hard)
    • Human “fills in” the details of the current frame (easier)
  • 49. Make a frame for my house
    • Item 1
    • Item 2
    • Item 3
    • Item 4
    • Item 5
    • Item 6