The powerpoint slides

311 views

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
311
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The powerpoint slides

  1. 1. Facial Animation Wilson Chang Paul Salmon April 9, 1999 Computer Animation University of Wisconsin-Madison
  2. 2. Papers Used <ul><li>Bregler C.,Covell M.,Slaney M., Video Rewrite: Driving Visual Speech with Audio . In SIGGRAPH 97 Conference Proceedings . ACM SIGGRAPH, August 1997 </li></ul><ul><li>Guenter B.,Grimm C.,Wood D., Malvar H., Pighin F, Making Faces . In SIGGRAPH 98 Conference Proceedings . ACM SIGGRAPH, July 1998 </li></ul><ul><li>Pighin F, Hecker J., Lischinski D., Szeliski R., Salesin D., Synthesizing Realistic Facial Expressions from Photographs . In SIGGRAPH 1998 . </li></ul><ul><li>Waters K., A Muscle Model for Animating Three-Dimensional Facial Expression . In SIGGRAPH 1987 . </li></ul>
  3. 3. Motivation <ul><li>Creation of Virtual Characters </li></ul><ul><li>Teleconferencing & Video Compression </li></ul><ul><li>Simulated Movement </li></ul><ul><li>Facial Surgery Planning </li></ul>
  4. 4. Why facial animation is hard. <ul><li>Humans are very good at reading expressions. </li></ul><ul><li>Any slight deviation from a “correct” expression will be immediately noticed. </li></ul><ul><li>Deep-rooted instinct. </li></ul>
  5. 5. Three general catagories <ul><li>2-D Facial Model </li></ul><ul><li>3-D Facial Model </li></ul><ul><li>Muscular Model </li></ul>
  6. 6. 2-D Facial Animation <ul><li>Video Rewrite - modify and sync an actors’ lip motion to a new soundtrack. </li></ul><ul><li>Keyframe approach. </li></ul><ul><li>Uses vision techniques to track mouth movement. </li></ul>
  7. 7. Video Rewrite registration <ul><li>Hand annotation of 26 images with 54 eigenpoints each. </li></ul><ul><li>Morph pairs to 351 images. </li></ul><ul><li>Learn eigenpoint model. </li></ul><ul><li>Warp images to standard reference plane. </li></ul><ul><li>Eigenpoint analysis. </li></ul>
  8. 8. Audio Analysis <ul><li>Video Rewrite uses TIMIT speech database. </li></ul><ul><li>Triphones - emphasize middle. </li></ul><ul><li>“ teapot” = /SIL-T-IY/, /T-IY-P/, /IY-P-AA/, /P-AA-T/, /AA-T-SIL/ </li></ul>
  9. 9. Video Synthesis <ul><li>Triphone Footage selection </li></ul><ul><li>error =  D p + (1-  )D s </li></ul><ul><li>D p phoneme-context distance. </li></ul><ul><li>D s distance between lip shapes. </li></ul><ul><ul><li>Overall Lip Width & Height </li></ul></ul><ul><ul><li>Inner Lip Height </li></ul></ul><ul><ul><li>Height of Visible Teeth </li></ul></ul>
  10. 10. Finish Synthesis <ul><li>Compress and Stretch video. </li></ul><ul><li>Align and blend mouth to face. </li></ul>
  11. 11. Results <ul><li>Good Sync and natural articulation. </li></ul><ul><li>Missing Triphones result in unnatural speech </li></ul>
  12. 12. Making Faces <ul><li>Motion capture. </li></ul><ul><li>3D mesh via Cyberware Laser scanner. </li></ul><ul><li>Deformed by </li></ul><ul><ul><li>Position of 128 Dots </li></ul></ul><ul><ul><ul><li>Manual identification - 1st frame </li></ul></ul></ul><ul><ul><ul><li>Tracked by vision techniques </li></ul></ul></ul><ul><li>Texture Extraction </li></ul><ul><ul><li>Dot removal. </li></ul></ul><ul><ul><li>Cylindrical map. </li></ul></ul>
  13. 13. Synthesizing Realistic Facial Expressions from Photographs <ul><li>3D facial models derived from photographs. </li></ul><ul><li>Smooth transitioning between model expressions. </li></ul><ul><li>Adaptation from one model to another. </li></ul>
  14. 14. Model Fitting <ul><li>Generic 3D mesh model. </li></ul><ul><li>Pose Recovery - using multiple subject views: </li></ul><ul><ul><li>Identify feature points. </li></ul></ul><ul><ul><li>Deduce camera pose. </li></ul></ul><ul><ul><li>Iteratively refine the generic face model. </li></ul></ul>
  15. 15. Model Fitting <ul><li>Scattered Data Interpolation: </li></ul><ul><ul><li>Interpolate mesh between feature points. </li></ul></ul><ul><ul><li>Uses radial basis functions. </li></ul></ul><ul><li>Correspondence based shape refinement: </li></ul><ul><ul><li>Use less accurate correspondences. </li></ul></ul><ul><ul><li>Polylines for eyebrows, eyelids, lips, etc. </li></ul></ul><ul><ul><li>Not used in pose processing due to error. </li></ul></ul>
  16. 16. Texture Extraction <ul><li>View independent vs View dependent. </li></ul><ul><li>Weight maps- bias selection of original photograph: </li></ul><ul><ul><li>Self-occlusion. </li></ul></ul><ul><ul><li>Smoothness. </li></ul></ul><ul><ul><li>Positional certainty. </li></ul></ul><ul><ul><li>View similarity. </li></ul></ul>
  17. 17. View Dependent Texture Extraction <ul><li>Select best photographs. </li></ul><ul><li>Draw model for each photograph. </li></ul><ul><li>Blend rendered image. </li></ul><ul><li>Pros </li></ul><ul><ul><li>adds detail. </li></ul></ul><ul><li>Cons </li></ul><ul><ul><li>sensitive to original photo. </li></ul></ul><ul><ul><li>More memory, slower. </li></ul></ul>
  18. 18. View Independent Texture Extraction <ul><li>Blend photographs to form single texture. </li></ul><ul><ul><li>Map onto virtual cylinder. </li></ul></ul>
  19. 19. View Independent Texture Extraction <ul><li>Blurry </li></ul>Dependent Independent
  20. 20. Special Case Textures <ul><li>Fine Detail - hair. </li></ul><ul><li>Occlusion - eyes, teeth. </li></ul><ul><li>Intricate Projection - ears. </li></ul><ul><li>Shadowing - eyes, teeth </li></ul><ul><li>Solutions </li></ul><ul><ul><li>Use photo with highest visibility. </li></ul></ul><ul><ul><li>Simulate shadowing </li></ul></ul>
  21. 21. Expression Morphing <ul><li>Simplified by common mesh. </li></ul><ul><li>Linearly interpolated vertices. </li></ul><ul><li>Blend result of rendering with each texture. </li></ul><ul><li>Synthesize new expressions via: </li></ul><ul><ul><li>Global blend. </li></ul></ul><ul><ul><li>Regional blend. </li></ul></ul><ul><ul><li>Painterly interface. </li></ul></ul>
  22. 22. Results <ul><li>Smooth transitioned expressions: </li></ul>
  23. 23. Results <ul><li>Applied transitions to different human subject: </li></ul>
  24. 24. Our conclusions <ul><li>Good results between models. </li></ul><ul><li>Relatively inexpensive equipment. </li></ul><ul><li>Notable manual processing. </li></ul>
  25. 25. Muscular Modeling <ul><li>Easy generalized across models. </li></ul><ul><li>22 muscle groups </li></ul><ul><li>Facial Action Coding System (Ekman, Wallace) - Action Unit parameterization </li></ul>
  26. 26. Anatomy
  27. 27. Skin as Mesh <ul><li>Nodal mobility </li></ul><ul><ul><li>Tensile Strength of skin </li></ul></ul><ul><ul><li>Proximity to muscle attachment </li></ul></ul><ul><ul><li>Depth of tissue & proximity to bone </li></ul></ul><ul><ul><li>Elasticity & interaction with other muscles </li></ul></ul><ul><li>Network of springs </li></ul><ul><ul><li>p = F/k </li></ul></ul>
  28. 28. Mesh expression examples
  29. 29. Muscle types modeled <ul><li>Linear/parallel muscles </li></ul><ul><li>Sphincter muscles </li></ul>
  30. 30. Linear/parallel muscles
  31. 31. Sphincter muscles
  32. 32. Animating <ul><li>Not in paper </li></ul><ul><li>Build a library </li></ul><ul><li>Abstract language </li></ul><ul><li>Keyframe </li></ul>

×