Talk accompanying the paper: Lihua You and Richard Southern and Jian J. Zhang, Motion in Games (Lecture Notes in Computer Science), 2009/06/01, 5884(1):207-218, doi:10.1007/978-3-642-10347-6_19

1. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Adaptive physics–inspired facial animation
Lihua You, Richard Southern and Jian Jun Zhang
November 6, 2009
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

2. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Introduction
Goals
Mathematical Model
Practicalities
Derivation
Reconstruction
Mesh sampling
Reconstruction
Applications
Force blend shapes
Force transfer
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

3. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Goals
Goals
◮ Facial animation is complex — simple linear blending between
facial models provides insufficient detail.
◮ The skin is a visco–elastic, anisotropic material and is difficult
to simulate.
◮ We want to derive a fast parametric model for skin
deformation of the human face.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

4. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
Practicalities
◮ Skin properties can be well predicted by isotropic elastic
properties.
◮ We derive our model from isotropic elastic plates.
◮ Analytical solution very difficult due to 4th order PDE.
◮ We derive a numerical solution using finite differencing.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

5. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
Forces acting on an element of a skin surface
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

6. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
Derivation
◮ Force equilibrium along x-axis:
∂Qu
∂u
+
∂Qv
∂v
+ Fx = 0 (1)
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

7. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
Derivation
◮ Force equilibrium along x-axis:
∂Qu
∂u
+
∂Qv
∂v
+ Fx = 0 (1)
◮ Equilibrium of moments around the u and v axes, respectively:
Qu =
∂Mu
∂u
+
∂Mvu
∂v
and Qv =
∂Mv
∂v
+
∂Muv
∂u
(2)
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

8. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
Derivation
◮ Force equilibrium along x-axis:
∂Qu
∂u
+
∂Qv
∂v
+ Fx = 0 (1)
◮ Equilibrium of moments around the u and v axes, respectively:
Qu =
∂Mu
∂u
+
∂Mvu
∂v
and Qv =
∂Mv
∂v
+
∂Muv
∂u
(2)
◮ Substituting Eq 2 in Eq 1
∂2Mu
∂u2
+ 2
∂2Muv
∂u∂v
+
∂2Mv
∂v2
+ Fx = 0 (3)
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

9. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
◮ From before:
∂2Mu
∂u2
+ 2
∂2Muv
∂u∂v
+
∂2Mv
∂v2
+ Fx = 0 (4)
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

10. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
◮ From before:
∂2Mu
∂u2
+ 2
∂2Muv
∂u∂v
+
∂2Mv
∂v2
+ Fx = 0 (4)
◮ From theory of bending isotropic elastic plates:
Mu = −D(
∂2x
∂u2
+ µ
∂2x
∂v2
)
Mv = −D(
∂2x
∂v2
+ µ
∂2x
∂u2
)
Muv = −(1 − µ)D
∂2x
∂u∂v
(5)
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

11. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
◮ From before:
∂2Mu
∂u2
+ 2
∂2Muv
∂u∂v
+
∂2Mv
∂v2
+ Fx = 0 (4)
◮ From theory of bending isotropic elastic plates:
Mu = −D(
∂2x
∂u2
+ µ
∂2x
∂v2
)
Mv = −D(
∂2x
∂v2
+ µ
∂2x
∂u2
)
Muv = −(1 − µ)D
∂2x
∂u∂v
(5)
◮ Substituting Eq 5 in Eq 4:
S1x
∂4x
∂u4
+ S2x
∂4x
∂u2∂v2
+ S3x
∂4x
∂v4
= Fx (6)
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

12. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
◮ From before, in vector form, x = [x, y, z]T , f = [fx , fy , fz ]T :
s1
∂4x
∂u4
+ s2
∂4x
∂u2∂v2
+ s3
∂4x
∂v4
= f (7)
◮ Boundary conditions:
u = 0 x = g0(v)
∂x
∂u
= g1(v)
u = 1 x = g2(v)
∂x
∂u
= g3(v)
v = 0 x = g4(u)
∂x
∂v
= g5(u)
v = 1 x = g6(u)
∂x
∂v
= g7(u) (8)
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

13. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
◮ From before, in vector form, x = [x, y, z]T , f = [fx , fy , fz ]T :
s1
∂4x
∂u4
+ s2
∂4x
∂u2∂v2
+ s3
∂4x
∂v4
= f (7)
◮ Boundary conditions:
u = 0 x = g0(v)
∂x
∂u
= g1(v)
u = 1 x = g2(v)
∂x
∂u
= g3(v)
v = 0 x = g4(u)
∂x
∂v
= g5(u)
v = 1 x = g6(u)
∂x
∂v
= g7(u) (8)
◮ Solving this 4th order PDE is very difficult.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

14. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
◮ From before, in vector form, x = [x, y, z]T , f = [fx , fy , fz ]T :
s1
∂4x
∂u4
+ s2
∂4x
∂u2∂v2
+ s3
∂4x
∂v4
= f (7)
◮ Boundary conditions:
u = 0 x = g0(v)
∂x
∂u
= g1(v)
u = 1 x = g2(v)
∂x
∂u
= g3(v)
v = 0 x = g4(u)
∂x
∂v
= g5(u)
v = 1 x = g6(u)
∂x
∂v
= g7(u) (8)
◮ Solving this 4th order PDE is very difficult.
◮ We approximate the solution using Finite Differencing.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

15. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
Finite differencing
◮ We solve this numerically with central differencing:
2(3s1 + 2s2 + 3s3)x0 − 2(2s1 + s2)(x1 + x3)
−2(s2 + 2s3)(x2 + x4) + s2(x5 + x6 + x7 + x8)
+s1(x9 + x10 + x11 + x12) = δ4
f0
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

16. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
Finite differencing
◮ We solve this numerically with central differencing:
2(3s1 + 2s2 + 3s3)x0 − 2(2s1 + s2)(x1 + x3)
−2(s2 + 2s3)(x2 + x4) + s2(x5 + x6 + x7 + x8)
+s1(x9 + x10 + x11 + x12) = δ4
f0
◮ Along with the restated boundary conditions, this can be
posed in matrix form:
KX = F.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

17. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Practicalities
Derivation
Finite differencing
◮ We solve this numerically with central differencing:
2(3s1 + 2s2 + 3s3)x0 − 2(2s1 + s2)(x1 + x3)
−2(s2 + 2s3)(x2 + x4) + s2(x5 + x6 + x7 + x8)
+s1(x9 + x10 + x11 + x12) = δ4
f0
◮ Along with the restated boundary conditions, this can be
posed in matrix form:
KX = F.
◮ K is sparse and square — system can be quickly solved using
conjugate gradient method.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

18. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
Finite difference mesh sampling
◮ For our mathematical model, the surface must be a regular
quadrilateral grid.
◮ Our input is a triangle mesh.
◮ We develop a simple automatic method to find this surface
from an input face mesh.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

19. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
General sampling approach
◮ The input face must have an open
boundary.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

20. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
General sampling approach
◮ The input face must have an open
boundary.
◮ The face is flattened using some
parametrization.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

21. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
General sampling approach
◮ The input face must have an open
boundary.
◮ The face is flattened using some
parametrization.
◮ Nodes are sampled in the parametric
domain.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

22. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
General sampling approach
◮ The input face must have an open
boundary.
◮ The face is flattened using some
parametrization.
◮ Nodes are sampled in the parametric
domain.
◮ Barycentric coordinates of nodes in faces
of parametric domain are determined.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

23. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
General sampling approach
◮ The input face must have an open
boundary.
◮ The face is flattened using some
parametrization.
◮ Nodes are sampled in the parametric
domain.
◮ Barycentric coordinates of nodes in faces
of parametric domain are determined.
◮ 3D node location determined using
barycentric coordinates on original mesh.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

24. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
Manual sampling
An artist deformed the u, v coordinates of the parametrization to
align with a grid texture.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

25. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
Discussion
◮ Regular sampling will probably not capture features in
sufficient detail.
◮ Manual sampling is time consuming. Automatic tools (e.g.
Graphite) cause fold–over.
◮ We tackle the problem using an adaptive sampling approach.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

26. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
Adaptive approach
◮ Sub–regions of the face which have significant deformation
(eyes, mouth) are sub–sampled in the parametric domain.
◮ Boundary conditions of subregions can be deduced by
interpolation of values at original nodes.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

27. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
Reconstruction
◮ The grid mesh is triangulated.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

28. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
Reconstruction
◮ The grid mesh is triangulated.
◮ For each vertex pi in the original mesh
. . .
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

29. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
Reconstruction
◮ The grid mesh is triangulated.
◮ For each vertex pi in the original mesh
. . .
◮ Compute barycentric coordinates bi
and displacement from the face di .
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

30. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
Reconstruction
◮ The grid mesh is triangulated.
◮ For each vertex pi in the original mesh
. . .
◮ Compute barycentric coordinates bi
and displacement from the face di .
◮ Entire mesh can be reconstructed
using the matrix form:
ˆP = BX + D
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

31. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
Reconstruction
◮ The grid mesh is triangulated.
◮ For each vertex pi in the original mesh
. . .
◮ Compute barycentric coordinates bi
and displacement from the face di .
◮ Entire mesh can be reconstructed
using the matrix form:
ˆP = BX + D
◮ di can be rotated based on orientation
of face in grid mesh to improve results.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

32. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Mesh sampling
Reconstruction
Reconstruction results
(a) Ground truth (b) Undersampling (c) No Rotation (d) Rotation
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

33. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Force blend shapes
Force transfer
Force blend shapes
KX =
i
αi Fi
0.0 0.2 0.4 0.6 0.8 1.0
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation

34. Outline
Introduction
Mathematical Model
Reconstruction
Applications
Force blend shapes
Force transfer
Force transfer
◮ Shape deformation can be transfered from one arbitrary input
surface to another.
◮ Both input meshes must have corresponding nodes, especially
around key deforming features.
◮ Each node force vector must be rotated and scaled for the
target model.
Lihua You, Richard Southern and Jian Jun Zhang Adaptive physics–inspired facial animation