Digital Doubles agency (ADN) proposes a process to create and manage digital doubles of prominent persons for use in movies and games. The process involves capturing actors using structured light and polarized gradient light to create realistic 3D models. Models are versatile, sustainable, and legally governed by ADN. Productions implement doubles through performance capture and ADN helps integrate doubles. This natural extension of traditional doubles aims to share costs while ensuring control and legal protection for all parties.

1. 3D HUMAN CONFERENCE
Digital Doubles : the prohibited reproduction ?
Cédric Guiard
ADN
Agence de Doublures Numériques
Digital Doubles Agency
www.adnda.com

2. Digital Doubles
in the movie and advertising industry
• Commonly accepted for stunt actions
• Framing & notoriety drive new artistic possibilities

3. CG characters
in video-games cinematics
• Sports and IPs based on prominent persons
• Quality increase on trailers & in-games cinematics

4. Photo-realism / Fidelity
Fidelity Virtual Reproduction
Realism Representation of the reality
Non Photo-Realistic
Common N.A.
rendering
Photo-realistic
Emergent Current stake !
rendering
In the present production mode,
the indistinguishable CG reproduction of identifiable persons
is mostly the prerogative of major studios
while it carries the most significant potential
Framing
Character Large Close-up
Anonymous Walk-on player N.A.
Secondary
Identifiable Main actor !
character

5. Standard production workflow
Such representations are processed
during movies and video-games post-production :
- on a case by case basis for each sequence
- with significant costs and time constraints
over the modeling and set-up
- variable quality and usability
- dependence towards technical providers
- without the control of the actor over his appearance and acting
- without legal grants over these data and results

6. ADN
Agence de Doublures Numériques
let’s manage the big picture
while moving the camera closer :
- classification of actual use-cases & requirements
- methodological, technical, legal stakes & problematics
linked to
the photo-realistic and accurate CG reproduction
of the appearance and performance of an actor
www.youtube.com/user/ADNDAtv
ADN introduces a new process to:
- control the creation, management and exploitation of
- give independent productions and studios access to
Digital Doubles of prominent persons

7. ADN Proposal
Natural extension of
the traditional principle and framework
of « doubles » applied to CG representation
Benefits:
- share costs related to the modeling and set-up
- supervise and optimize the technical production
- establish a legal framework and jurisprudence
- ensure the control of the actor over his representation
- insure the photo-realism
- certify the results conformity
- guaranty the exploitability of studios and producers

8. ADN Process
Restructuration of production activities
into 3 stages:
On the first production:
 creation of the Digital Double
For the production of different sequences:
 implementation of the Digital Double
During the post-production:
 integration of the Digital Double

10. ADN Services
 Digital doubles legal and moral governance 
 Administration of portfolio exploitation 
 Models management and maintenance 
Creation:
 co-investment on
on the Digital Double creation
Implementation:
 Adaptation of the appearance
 Performance capture/retargetting
Integration:
( Supervision of the integration)

11. Technical requirements
To be implemented,
this process requires the model to be:
 Sustainable:
i.e. not tailored for a single use but built to last
 Versatile in terms of appearance:
i.e. the appearance can be adapted upon the production requests
(mock-up, aging, weighting, alteration, …)
 Generic in terms of implementation:
i.e. fit to respond to various animation and lighting conditions
 Exploitable:
i.e. results integration follows traditional tools and techniques
(nature, granularity and format of the interfaces)

12. Technical approach
Analysis/Synthesis approach based on:
Measures acquisition/selection, Data analysis / Model reconstruction
Separation between Appearance and Performance
geometry / reflectance / deformation signal processing
• Positioned as technical integrator:
Data acquisitions are externalized
to benefit from technical progress and market trends
• Technological assets:
- Data/Model registration and information transfer/retargeting
- Characterization of adapted search and parametrization spaces
• Concerns:
- Error estimation on data which directly/indirectly impact the results
- Interoperability with traditional animation/rendering techniques

13. Creation of the Digital Double
• Production of a model of the actor
ready to be animated and rendered under any conditions
Appearance and expressions space
characterization based on
morphologic, material, expressions
acquisition/analysis/reconstruction
Qualitative & Quantitative
validation of the conformity
The creation and validation of the Digital Double
follows an agreement with the celebrity.
This contract can be settled independently of any production
since this model is to be shared among/between productions.

14. Structured light acquisition
• Static scan of the neutral face

15. Structured light acquisition
• Dynamic scan of the facial ROM:
- 90 seconds:
emotional expressions
and locution facial gym
- Statistical de-correlation
close to muscular decomposition
- Set-up of 103 FAUs

16. Polarized gradient light acquisition
• Detailed geometry and reflectance fields acquisition:
- Polarized shots
- Unpolarized shots
under 4 lighting conditions
Monocular camera
- 5 structured light patterns
Binocular cameras

17. Polarized gradient light acquisition
• Resulting HD Maps - before conformation upon UV model
Diffuse Specular Displacment
Spec. Normal Diff. Normal R Diff. Normal G Diff. Normal B

18. Polarized gradient light acquisition
• SSS diffusion radius estimation:
 Detailed geometry:

19. Neutral Surfacing & Shading
Diffuse Specular
Hybrid normal shader / Diffuse bent normal + specular / SSS simulation +w specular
Displacement + Normal*Bump + HDRI lighting

20. Dynamic Surfacing & Shading
• Stretched/Relaxed extreme expressions:
uncompressed maps compressed maps
bump displace bump displace

21. Deformation & Shading Set-up
Deformation set-up Shading set-up
FAUs influence mask

22. Complementary elements
• Eyes, eyelids, inner mouth, teeth, tongue (modeling/shading):

23. Complementary elements
• Eyes, eyelids, inner mouth, teeth, tongue (modeling/shading):

24. Implementation of the Digital Double
• Reproduction of the appearance and performance
of the character considered on the different sequences:
Performance capture/retargeting
of the acting
(potential adaptation to
the character appearance)
Qualitative & Quantitative
validation of the conformity
These services are contracted by the executive production
in agreement with the actor’s commitment.
Each production is only charged
for its specific use of the Digital Double.

25. Performance capture
 Performance characterization, editing and retargeting:
FAU 3
FAU n
FAU 2
FAU 1
Characterization of the pose in the expression space
drives the animation of the considered appearance
This scheme is also considered to assess the acting’s conformity

26. Integration of the Digital Double
• Integration of the animated Digital Double
for the making of the final shots
follows standard lighting/rendering/compositing techniques
ADN delivers
the animated Digital Double
to the production studio contracted by the producer
This process has no impact on
Studios/Producers contractual relations.
In any case, the exploitation of the Digital Double remains limited
to the program contracted with the celebrity.

27. Conclusion & Perspectives
• The treatment of this problematic highly benefits from:
- Progress and democratization of data acquisitions systems
- Methods and techniques stemming from computer vision
Still, the establishment of Data/Model registration and legal control
are the most significant barriers we tackled
• Research & development towards in-game derivation:
- declension of Digital Doubles for real-time applications
- coupling visual+audio reproduction of actors
while maintaining personality’s control over exploitation possibilities
• Domain and media convergence lead to extend this approach to:
- other celebrities: athletes, singers, fashion models, peoples…
- other sectors: internet applications, live TV broadcast, events…

28. Conclusion & Perspectives
Technological evolution and markets trends will
rapidly permit to develop flexible approaches to
capture and reproduce visual representations of
real world objects, persons, scenes, events
hardly distinguishable from the image reality
… We had better get ready
to manage this evolution …

29. Questions / Answers
cedric.guiard@adnda.com - christian.guillon@adnda.com
ADN
Agence de Doublures Numériques
Digital Doubles Agency
www.adnda.com