ADN
Agence de Doublures Numériques




Creation, Management, Exploitation
         of Digital Doubles

         www.adnda....
Facial modeling and animation
The human face :
« a technical and artistic challenge »
  • First revealing the nature and p...
Fields of application
Constant increasing use in the entertainment industry:
• Anonymous CG character in interactive appli...
CG character in SFX feature films
   A common practice for figurative roles and
    crowd simulation purposes:




    An...
Digital Doubles
     for the cinema and advertising
   Commonly accepted for stunt actions:




   Framing/notoriety dri...
CG characters in video-games
   Sports and IPs based on prominent persons:




   Quality increase on trailers & in-game...
Photo-realism / Fidelity
            Fidelity       Virtual              Reproduction
Realism                 Representati...
Standard production workflow
   These representations are mostly processed during
    the film post-production / video-ga...
Agence de Doublures Numériques
 ADN project: let’s try to move the camera closer
 - identification/classification of actua...
Mission
               Natural extension of
      the traditional principle and framework
   of « doubles » applied to CG ...
Process
Restructuration of production activities into 3 stages:

      Creation:
       creation of the Digital Double


...
Stages         Creation             Implementation                Integration
                                            ...
Services




                                                                               Legal and moral framework gov...
Productions assets
Actor:
‒ image rights governing
‒ conformity of his appearance on the screen
‒ control over the digital...
Technical requirements
To be implemented, this approach requires the model to be:

  Sustainable:
i.e. not tailored for a...
Technical approach
Analysis/Synthesis approach founded on:
  Measures acquisition/selection, Data analysis/reconstruction
...
Creation of the Digital Double
 Production of a model of the actor
 ready to be animated and rendered under any conditions...
Structured light acquisition
   Static scan of the neutral face
Structured light acquisition
   Dynamic scan of the facial ROM:




    - 90 seconds:
      emotional expressions
      a...
Polarized gradient light acquisition
   Detailed geometry and reflectance fields acquisition:




    - Polarized shots
 ...
Polarized gradient light acquisition
   Resulting HD Maps (before conformation upon the model’s UV):




       Diffuse  ...
Polarized gradient light acquisition
       SSS diffusion radius estimation:




        Detailed geometry:
Neutral Surfacing & Shading


                   Diffuse                                Specular




            Hybrid no...
Dynamic Surfacing & Shading
   Stretched/Relaxed extreme expressions:




    uncompressed maps                        co...
Deformation & Shading Set-up




Deformation set-up                         Shading set-up
                     FAUs influ...
Complementary elements
   Eyes, eyelids (modeling/shading):
Complementary elements
   Inner mouth, teeth, tongue (modeling/shading):
Implementation of the Digital Double
Reproduction of the appearance and performance
of the character considered on the dif...
Performance capture
   Motion characterization, editing and retargeting:




      Characterization of the pose in the ex...
Integration of the Digital Double
The integration of the animated Digital Double
for the realization of the final shots
fo...
Conclusions & Perspectives
   The treatment of this problematic highly benefits from:
    - Progress and democratization ...
Conclusions & Perspectives


  Technological evolution and markets trends will rapidly permit
       to develop flexible a...
Question / Answer




       Agence de Doublures Numériques
           Digital Doubles Agency
               www.adnda.com...
Agence de Doublures Numériques : the Digital Doubles Agency
Upcoming SlideShare
Loading in …5
×

Agence de Doublures Numériques : the Digital Doubles Agency

1,909
-1

Published on

http://www.adnda.com
Today, Computer Graphics enables the photo-realistic reproduction of celebrities' appearance and performance. ADN, the Digital Doubles Agency introduces new technical and legal process services:
- to control the creation, implementation and exploitation of Digital Doubles
- to give independent productions and studios access to this innovative technology
for cinema, audiovisual and video-games productions.

Published in: Entertainment & Humor
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,909
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Agence de Doublures Numériques : the Digital Doubles Agency

  1. 1. ADN Agence de Doublures Numériques Creation, Management, Exploitation of Digital Doubles www.adnda.com
  2. 2. Facial modeling and animation The human face : « a technical and artistic challenge » • First revealing the nature and personality of the considered character, • Extreme anatomic, morphologic and physiologic complexity, • Special human ability and sensitivity in the perception and interpretation of facial images
  3. 3. Fields of application Constant increasing use in the entertainment industry: • Anonymous CG character in interactive applications: - Makes identification and personification easier, - Natural and familiar communication scheme, - Makes interaction positive and efficient. • VIPs reproduction in movie productions: - Conveys the charisma & fame of celebrities for narrative, artistic, logistic, economic,… reasons
  4. 4. CG character in SFX feature films  A common practice for figurative roles and crowd simulation purposes: Anonymous Establishing shots character << ¼ screen
  5. 5. Digital Doubles for the cinema and advertising  Commonly accepted for stunt actions:  Framing/notoriety drives new usages evolution:
  6. 6. CG characters in video-games  Sports and IPs based on prominent persons:  Quality increase on trailers & in-games cinematics:
  7. 7. Photo-realism / Fidelity Fidelity Virtual Reproduction Realism Representation of the reality Non Photo-Realistic Common N.A. rendering Photo-realistic Emergent Current stake ! rendering In the present production mode, the indistinguishable CG reproduction of identifiable personality is mostly the prerogative of major studios while it carries the most significant potential Framing Character Large Close-up Anonymous Walk-on player N.A. Secondary Identifiable Main actor ! character
  8. 8. Standard production workflow  These representations are mostly processed during the film post-production / video-game’s development: - on a case by case basis - with significant costs and time constraints over the modeling and set-up - variable quality and usability - dependence towards technical providers - without the control of the actor on his representation - without legal grants over these data and results
  9. 9. Agence de Doublures Numériques ADN project: let’s try to move the camera closer - identification/classification of actual use-cases & requirements - methodological, technical, legal stakes & problematics linked to the photo-realistic and accurate CG reproduction of the appearance and performance of an actor ADN introduces a new process to: - control the creation, management and exploitation of - give independent productions and studios access to Digital Doubles of prominent persons
  10. 10. Mission Natural extension of the traditional principle and framework of « doubles » applied to CG representation Objectives: ‒ supervise and optimize the production ‒ share costs related to the modeling/set-up ‒ establish a legal framework and jurisprudence ‒ ensure the control of the actor over his representation ‒ guaranty the photo-realism ‒ certify the results conformity ‒ insure exploitability of the studios
  11. 11. Process Restructuration of production activities into 3 stages: Creation:  creation of the Digital Double Implementation:  implementation of the Digital Double Integration:  integration of the Digital Double
  12. 12. Stages Creation Implementation Integration VFX / Video-game Who ADN ADN studios During the Independent of For the making When the productions of a sequence cinematics production Performance capture and Morphology, Lighting retargeting of the acting What Reflectance, (potential adaptation to & Expressions Rendering the character appearance) Contract with No impact on Agreement with the executive production Studios - Producers Legal the celebrity deriving from contractual the actor’s commitments relations Reference Reference Global illumination References image/video image/video conditions of the actor of the character of the scene Digital Double Adapted & Animated of the actor Digital Double Final images Results ready for of the character of the sequence animation/lighting ready for render/compo
  13. 13. Services  Legal and moral framework governing   Models management and maintenance   Administration of portfolio exploitation  Creation:  shared investment on on the Digital Double creation Implementation:  Adaptation of the appearance  Performance capture/retargetting Integration: ( Supervision of the integration)
  14. 14. Productions assets Actor: ‒ image rights governing ‒ conformity of his appearance on the screen ‒ control over the digital double performance ‒ availability (free-time, additional commitments, notoriety….) Talent manager: ‒ appropriation of a technological evolution Production studio: ‒ compliancy with standard animation and rendering techniques ‒ technical garanties on the photo-realism and conformity Producer / Editor: ‒ costs/time sharing on the modeling/set-up ‒ legal garanties on the results exploitation
  15. 15. Technical requirements To be implemented, this approach requires the model to be:  Sustainable: i.e. not tailored for a single use but built to last  Versatile in terms of appearance: i.e. the appearance can be adapted upon the production requests (mock-up, aging, weighting, alteration, …)  Generic in terms of implementation: i.e. fit to respond to various animation and lighting conditions  Exploitable: i.e. results integration follows traditional tools and technics (granularity, nature, format of the interfaces)
  16. 16. Technical approach Analysis/Synthesis approach founded on: Measures acquisition/selection, Data analysis/reconstruction Separation between Appearance and Performance geometry / material / deformation signal processing  Positioned as technical integrator: Data acquisitions are externalized to benefit from technical progress and market trends  Technological asset: - Data/Model registration and information transfer/retargeting - Characterization of adapted search and parametrization spaces  Concerns: - Error estimation on data which directly/indirectly impact the results - Interoperability with traditional animation/rendering approaches
  17. 17. Creation of the Digital Double Production of a model of the actor ready to be animated and rendered under any conditions: Appearance and expressions space characterization based on morphologic, reflectance, expressions acquisition/analysis/reconstruction Qualitative & Quantitative validation of the conformity The creation and validation of the Digital Double follows an agreement with the celebrity. This contract can be settled independently of any production since this model is to be shared among/between different titles.
  18. 18. Structured light acquisition  Static scan of the neutral face
  19. 19. Structured light acquisition  Dynamic scan of the facial ROM: - 90 seconds: emotional expressions and locution facial gym - Statistical de-correlation close to muscular decomposition - Set-up of 103 FAUs
  20. 20. Polarized gradient light acquisition  Detailed geometry and reflectance fields acquisition: - Polarized shots - Unpolarized shots under 4 lighting conditions Monocular camera - 5 structured light patterns Binocular cameras
  21. 21. Polarized gradient light acquisition  Resulting HD Maps (before conformation upon the model’s UV): Diffuse Specular Displacment Spec. Normal Diff. Normal R Diff. Normal G Diff. Normal B
  22. 22. Polarized gradient light acquisition  SSS diffusion radius estimation:  Detailed geometry:
  23. 23. Neutral Surfacing & Shading Diffuse Specular Hybrid normal shader / Diffuse bent normal + specular / SSS simulation + specular Displacement + Normal*Bump + HDRI lighting
  24. 24. Dynamic Surfacing & Shading  Stretched/Relaxed extreme expressions: uncompressed maps compressed maps bump displace bump displace
  25. 25. Deformation & Shading Set-up Deformation set-up Shading set-up FAUs influence mask
  26. 26. Complementary elements  Eyes, eyelids (modeling/shading):
  27. 27. Complementary elements  Inner mouth, teeth, tongue (modeling/shading):
  28. 28. Implementation of the Digital Double Reproduction of the appearance and performance of the character considered on the different sequences: Performance capture and retargeting of the acting (potential adaptation to the character appearance) Qualitative & Quantitative validation of the conformity These services are contracted by the executive production in agreement with the actor’s commitment. Each production is only charged for its specific use of the Digital Double.
  29. 29. Performance capture  Motion characterization, editing and retargeting: Characterization of the pose in the expression space drives the animation of the considered appearance This scheme is also considered to assess the acting’s conformity
  30. 30. Integration of the Digital Double The integration of the animated Digital Double for the realization of the final shots follows standard lighting/rendering techniques ADN delivers the animated Digital Double to the production studio contracted by the producer This process has no impact on Studios-Producers contractual relations. In any case, the use of the Digital Double remains limited to the program contracted with the celebrity.
  31. 31. Conclusions & Perspectives  The treatment of this problematic highly benefits from: - Progress and democratization of data acquisitions systems - Methods and technics issued from computer vision Still, Data/Model registration and legal framework definition are the most significant barriers we tackled  Research & development towards in-game derivation: - declension of Digital Doubles for real-time game engine - coupling visual+audio reproduction of actors while maintaining personality’s control over animation possibilities  Domain, technical and media convergence lead to extend this approach to: - other celebrities: athletes, singers, fashion models, peoples… - other sectors: internet applications, live TV broadcast, events…
  32. 32. Conclusions & Perspectives Technological evolution and markets trends will rapidly permit to develop flexible approaches to capture and reproduce visual representations of real world objects, persons, scenes, events hardly distinguishable from the image reality … We had better get ready to manage this evolution …
  33. 33. Question / Answer Agence de Doublures Numériques Digital Doubles Agency www.adnda.com cedric.guiard@adnda.com christian.guillon@adnda.com

×