FMX 2010 ADN project: Digital Doubles for the cinematograph [email_address] L’Etude et la Supervision des Trucages
Facial modeling and Animation <ul><ul><li>The human face  :  «  a technical and artistic challenge  » </li></ul></ul><ul><...
Fields of Application <ul><ul><li>Virtual characters and avatars are bound to various Computer Graphics markets: </li></ul...
General motivation <ul><ul><li>Constant increasing use over the different medias : </li></ul></ul><ul><ul><li>Anonymous 3D...
Representation typology <ul><li>In the present mode of production,  the indistinguishable CG reproduction of identifiable ...
Standard cinematographic production workflow <ul><li>In the European film industry, these shots are mostly processed durin...
Digital Double for the cinema <ul><li>The movie industry has a long history  of use of « Doubles » (body details, action s...
3D character <ul><li>A common practice for figurative roles and crowd simulation purposes: </li></ul>Large shots << ¼ scre...
Digital Doubles <ul><li>Commonly accepted for stunt actions: </li></ul>Glimpse/Elusive actions Establishing shots Increase...
The ADN Project <ul><li>L’E.S.T: L’Etude et la Supervision des Trucages </li></ul><ul><li>complex visual effects over 100 ...
Our proposal <ul><li>Natural extension of  the traditional principles and framework of « doubles » applied to 3D represent...
Use-cases typology <ul><li>Visual and special effects: - modification of the appearance - make up for an inability - artis...
Process <ul><li>Restructuration of post-production activities </li></ul><ul><li>into 3 stages: </li></ul><ul><li>Preparati...
 
Process Stages Preparation Production Post-production Who ADN ADN VFX studios When Independent of the productions For the ...
Services <ul><li>Preparation:   Bear costs and responsibility   on the Digital Double creation </li></ul><ul><li>Producti...
Productions assets <ul><li>Actor: - image rights governing - conformity of his representation on the screen - control over...
Technical requirements <ul><li>To be implemented, this approach requires the model to be: </li></ul><ul><li>Sustainable: i...
Technical approach <ul><li>Analysis/Synthesis approach founded on: </li></ul><ul><li>Measures acquisition/selection, Data ...
Structured light acquisition <ul><li>Static scan of the neutral face </li></ul>
<ul><li>Dynamic scan of the facial ROM: </li></ul><ul><li>- 90 seconds:   emotional expressions   and locution facial gym ...
<ul><li>Detailed geometry and reflectance fields acquisition: </li></ul><ul><li>- Polarized shots </li></ul><ul><li>- Unpo...
<ul><li>Resulting HD Maps  ( before conformation upon the model’s UV): </li></ul>Polarized gradient light acquisition   Di...
<ul><li>SSS diffusion radius estimation: </li></ul>Polarized gradient light acquisition <ul><li>Detailed geometry: </li></ul>
Neutral Surfacing & Shading Diffuse Specular Displacement + Normal*Bump +  HDRI lighting Hybrid normal shader / Diffuse be...
<ul><li>Stretched/Relaxed extreme expressions: </li></ul>Dynamic Surfacing & Shading uncompressed maps compressed maps bum...
<ul><li>Deformation set-up </li></ul>Deformation & Shading Set-up Shading set-up FAUs influence mask
<ul><li>Inner mouth, teeth, tongue, eyes, eyelids  (geometry) : </li></ul>Complementary elements
<ul><li>Inner mouth, teeth, tongue, eyes, eyelids  (texture) : </li></ul>Complementary elements
Upcoming SlideShare
Loading in …5
×

ADN - Presentation Imagina 2011

2,112 views

Published on

ADN – the Digital Doubles Agency (www.adnda.com) has tackled to structure the methodological, technical and legal stakes drawn by the photo-realistic and accurate CGI representation of prominent persons’ appearance and performance.
Our services introduce a new production process that simultaneously addresses:
- the need for control and regulation of actors' image rights,
- the economic constraints and exploitation guaranties of producers,
- the technical interfaces and requirements bound to this problematic,
for cinema & television vfx and video games cinematics.
This presentation illustrates the implementation of this process on a practical case study.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,112
On SlideShare
0
From Embeds
0
Number of Embeds
43
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

ADN - Presentation Imagina 2011

  1. 1. FMX 2010 ADN project: Digital Doubles for the cinematograph [email_address] L’Etude et la Supervision des Trucages
  2. 2. Facial modeling and Animation <ul><ul><li>The human face : « a technical and artistic challenge » </li></ul></ul><ul><ul><ul><li>First revealing the nature and personality of the considered character, </li></ul></ul></ul><ul><ul><ul><li>Extreme anatomic, morphologic and physiologic complexity , </li></ul></ul></ul><ul><ul><ul><li>Special ability and sensitivity of the human brain to interpret and characterize facial images. </li></ul></ul></ul>
  3. 3. Fields of Application <ul><ul><li>Virtual characters and avatars are bound to various Computer Graphics markets: </li></ul></ul><ul><ul><li>3D Representation : Entertainment : - 3D Animation , SFX - Advertising - Video-Games </li></ul></ul><ul><ul><li>Human-Computer interfaces: </li></ul></ul><ul><ul><li>TV broadcast/ Web3D/ Mobile Terminals: - Conversational agent - Digital presenter </li></ul></ul><ul><ul><li>On-line applications/business services : - Virtual clones </li></ul></ul><ul><ul><li>Virtual reality Training & Simulation: Industry/Architecture/Education/Security applications … </li></ul></ul>
  4. 4. General motivation <ul><ul><li>Constant increasing use over the different medias : </li></ul></ul><ul><ul><li>Anonymous 3D character in Real-time applications: </li></ul></ul><ul><ul><ul><li>- Makes identification and personification easier, </li></ul></ul></ul><ul><ul><ul><li>- Natural and familiar communication scheme, </li></ul></ul></ul><ul><ul><ul><li>- Makes interaction positive and efficient. </li></ul></ul></ul><ul><ul><li>VIPs 3D reproduction in CG productions: </li></ul></ul><ul><ul><ul><li>- Conveys the charisma & fame of a personality for scenaristic, artistic, logistic, economic reasons </li></ul></ul></ul>
  5. 5. Representation typology <ul><li>In the present mode of production, the indistinguishable CG reproduction of identifiable personality is mostly the prerogative of major studios and carries a significant potential </li></ul>Fidelity Realism Virtual Representation Reproduction of the reality Non Photo-Realistic rendering Common N.A. Photo-realistic rendering Emergent Current stake ! Framing Character Large Close-up Anonymous Walk-on player N.A. Identifiable Secondary character Main actor !
  6. 6. Standard cinematographic production workflow <ul><li>In the European film industry, these shots are mostly processed during the movie post-production: - a posteriori, on a case by case basis - significant costs and time constraints over the modeling and set-up - variable quality and usability - dependence towards the technical provider - without the control of the actor on his representation - without legal grants over these data and results </li></ul>
  7. 7. Digital Double for the cinema <ul><li>The movie industry has a long history of use of « Doubles » (body details, action stunt, voice, …) </li></ul><ul><li>The movie industry makes a growing use of visual effects in computer graphics </li></ul><ul><li>The emergence of Digital Doubles : an unescapable evolution already up and running … better be prepared to it </li></ul>
  8. 8. 3D character <ul><li>A common practice for figurative roles and crowd simulation purposes: </li></ul>Large shots << ¼ screen Anonymous persona
  9. 9. Digital Doubles <ul><li>Commonly accepted for stunt actions: </li></ul>Glimpse/Elusive actions Establishing shots Increase in the framing and the notoriety drive new usage evolutions …
  10. 10. The ADN Project <ul><li>L’E.S.T: L’Etude et la Supervision des Trucages </li></ul><ul><li>complex visual effects over 100 feature films in 10 years (Lord of War, Asterix, Le boulet, Faubourg 36, Oceans, …) </li></ul><ul><li>with focus on SFX supervision and executive production </li></ul><ul><li>well positioned between producers and post-production studios in interface with filmmakers and actors </li></ul><ul><li>always conducted a prospective approach </li></ul><ul><li>ADN project: lets try to move the camera closer </li></ul><ul><li>identification/classification of actual requirements and use-cases </li></ul><ul><li>methodological, technical and legal stakes and problematics </li></ul><ul><li>associated to the realistic and accurate CG reproduction of the appearance and performance of an existing actor </li></ul>
  11. 11. Our proposal <ul><li>Natural extension of the traditional principles and framework of « doubles » applied to 3D representation </li></ul><ul><li>Objectives: - supervise and optimise the production </li></ul><ul><li>- share costs related to the modeling/set-up </li></ul><ul><li>- establish a legal framework and jurisprudence </li></ul><ul><li>- garanty control of the actor over his representation </li></ul><ul><li>- ensure the photo-realism </li></ul><ul><li>- certify the results conformity </li></ul>
  12. 12. Use-cases typology <ul><li>Visual and special effects: - modification of the appearance - make up for an inability - artistic choice </li></ul><ul><li>Evolution in the film editing: - retakes / image post-synchronization - dailies loss or damage – insurance - rewriting during editing </li></ul><ul><li>Actor’s unavailability: - planning or geographic constraints - accident - « people » </li></ul>
  13. 13. Process <ul><li>Restructuration of post-production activities </li></ul><ul><li>into 3 stages: </li></ul><ul><li>Preparation:  creation of the Digital Double </li></ul><ul><li>Production :  implementation of the Digital Double </li></ul><ul><li>Post-production :  integration of the Digital Double </li></ul>
  14. 15. Process Stages Preparation Production Post-production Who ADN ADN VFX studios When Independent of the productions For the making of a sequence During the movie post-production What Morphology, Expressions, Reflectance Acting (potential adaptation of the appearance) Rendering & Compositing Legal Agreement with the personality Contract with the executive production No impact on VFX / Producers References Reference image/video of the actor Reference image/video of the character Global illumination conditions of the scene Results Digital Double of the personality ready for animation/lighting Adapted & Animated Digital Double of the character ready for render/compo Final images
  15. 16. Services <ul><li>Preparation:  Bear costs and responsibility on the Digital Double creation </li></ul><ul><li>Production:  Adaptation of the appearance  Performance capture/retargetting </li></ul><ul><li>Post-production: (  Supervision of the integration) </li></ul> Models management and maintenance   Legal and moral framework governing   Administration of portfolio exploitation 
  16. 17. Productions assets <ul><li>Actor: - image rights governing - conformity of his representation on the screen - control over the digital double performance - availability (free-time, additional commitment, notoriety, ….) </li></ul><ul><li>Artist manager: - appropriation of a technological evolution </li></ul><ul><li>Director: - continuation of his role along the process </li></ul><ul><li>Producer: - costs/time sharing on the modeling/set-up - technical and legal garanties on the results exploitation - growing independence regarding technical providers </li></ul>
  17. 18. Technical requirements <ul><li>To be implemented, this approach requires the model to be: </li></ul><ul><li>Sustainable: i.e. not tailored for a single use but built to last </li></ul><ul><li>Versatile in terms of appearance: i.e. the appearance can be adapted upon the production requests ( mock-up, aging, weighting, alteration, … ) </li></ul><ul><li>Generic in terms of implementation: i.e. fit to respond to various animation and lighting conditions </li></ul><ul><li>Exploitable: i.e. results integration follows traditional tools and technics (granularity, nature, format of the interfaces) </li></ul>
  18. 19. Technical approach <ul><li>Analysis/Synthesis approach founded on: </li></ul><ul><li>Measures acquisition/selection, Data analysis/reconstruction </li></ul><ul><li>Separation between Appearance and Performance geometric / illumination / displacements signal processing </li></ul><ul><li>Positioned as technical integrator: Data acquisitions are externalized to benefit from technical progress and market trends </li></ul><ul><li>Technological asset: - Data/Model registration and information transfer/retargeting - Definition of adapted search/parametrization spaces </li></ul><ul><li>Concerns: - Error estimation on data which directly/indirectly impact the results - Interoperability with traditional animation/rendering approaches </li></ul>
  19. 20. Structured light acquisition <ul><li>Static scan of the neutral face </li></ul>
  20. 21. <ul><li>Dynamic scan of the facial ROM: </li></ul><ul><li>- 90 seconds: emotional expressions and locution facial gym </li></ul><ul><li>- Statistical de-correlation close to muscular decomposition </li></ul><ul><li>- Set-up of 103 FAUs </li></ul>Structured light acquisition
  21. 22. <ul><li>Detailed geometry and reflectance fields acquisition: </li></ul><ul><li>- Polarized shots </li></ul><ul><li>- Unpolarized shots under 4 lighting conditions Monocular camera </li></ul><ul><li>- 5 structured light patterns Binocular cameras </li></ul>Polarized gradient light acquisition
  22. 23. <ul><li>Resulting HD Maps ( before conformation upon the model’s UV): </li></ul>Polarized gradient light acquisition Diffuse Specular Displacment Spec. Normal Diff. Normal R Diff. Normal G Diff. Normal B
  23. 24. <ul><li>SSS diffusion radius estimation: </li></ul>Polarized gradient light acquisition <ul><li>Detailed geometry: </li></ul>
  24. 25. Neutral Surfacing & Shading Diffuse Specular Displacement + Normal*Bump + HDRI lighting Hybrid normal shader / Diffuse bent normal + specular / SSS simulation + specular
  25. 26. <ul><li>Stretched/Relaxed extreme expressions: </li></ul>Dynamic Surfacing & Shading uncompressed maps compressed maps bump displace bump displace
  26. 27. <ul><li>Deformation set-up </li></ul>Deformation & Shading Set-up Shading set-up FAUs influence mask
  27. 28. <ul><li>Inner mouth, teeth, tongue, eyes, eyelids (geometry) : </li></ul>Complementary elements
  28. 29. <ul><li>Inner mouth, teeth, tongue, eyes, eyelids (texture) : </li></ul>Complementary elements
  29. 30. Performance capture <ul><li>Characterization of the pose in the expression space drives the animation of the considered appearance </li></ul><ul><li>Motion characterization, editing and retargeting: </li></ul>This principle is also considered to assess the acting’s conformity
  30. 31. Conclusions & Perspectives <ul><li>The treatment of this problematic highly benefits from: - Progress and democratization of data acquisitions systems - Methods and technics issued from computer vision </li></ul><ul><li>Still, data/model registration as well as the legal framework were the most significant issues of this project </li></ul><ul><li>Domain, technical and media convergence lead to extend this approach to: - other celebrities: models, athletes, singers, politicians… - other sectors: advertising, video-games, live broadcast or events… </li></ul><ul><li>Technological evolution and markets trends will rapidly permit to develop flexible approaches to capture and reproduce visual representations of real world objects, persons, scenes, events hardly distinguishable from the image reality </li></ul>… We had better get ready to manage this evolution …
  31. 32. <ul><li>Collaborators and partners implied on this project: </li></ul><ul><li>L’E.S.T </li></ul><ul><li>MIKROS IMAGES </li></ul><ul><li>SOULCLAP STUDIO </li></ul><ul><li>EYETRONICS / ETH </li></ul><ul><li>AGURU IMAGES / ICT </li></ul><ul><li>MOCAP LAB / College de France </li></ul>Thanks !
  32. 33. FMX 2010 Question / Answer Agence de Doublures Numériques Digital Doubles Agency www.adnda.com [email_address] [email_address]

×