Rapid advancement of distributed sensing and imaging technology brings the proliferation of high-dimensional spatiotemporal data, i.e., y(s; t) and x(s; t) in manufacturing and healthcare systems. Traditional regression is not generally applicable for predictive modeling in these complex structured systems. For example, infrared cameras are commonly used to capture dynamic thermal images of 3D parts in additive manufacturing. The temperature distribution within parts enables engineers to investigate how process conditions impact the strength, residual stress and microstructures of fabricated products. The ECG sensor network is placed on the body surface to acquire the distribution of electric potentials y(s; t), also named body surface potential mapping (BSPM). Medical scientists call for the estimation of electric potentials x(s; t) on the heart surface from BSPM y(s; t) so as to investigate cardiac pathological activities (e.g., tissue damages in the heart). However, spatiotemporally varying data and complex geometries (e.g., human heart or mechanical parts) defy traditional regression modeling and regularization methods. This talk will present a novel physics-driven spatiotemporal regularization (STRE) method for high-dimensional predictive modeling in complex manufacturing and healthcare systems. This model not only captures the physics-based interrelationship between time-varying explanatory and response variables that are distributed in the space, but also addresses the spatial and temporal regularizations to improve the prediction performance. In the end, we will introduce our lab at Penn State and future research directions will also be discussed.

1. Physics-driven Spatiotemporal Regularization for
High-dimensional Predictive Modeling
Bing Yao and Hui Yang
杨 徽
Associate Professor
Complex Systems Monitoring, Modeling and Control Lab
The Pennsylvania State University
University Park, PA 16802
November 25, 2017

2. Outline
1 Introduction
2 Physics-driven Spatiotemporal Regularization
3 Experimental Design and Results
4 References

3. Introduction Methodology Experiments References
Research Roadmap
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 3 / 42

4. Introduction Methodology Experiments References
Research Projects
Manufacturing Processes, Precision Machining
Publications: Pattern Recognition, IEEE Transactions, Chaos, IIE/ASME Transactions, J. Mfg
Sys
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 4 / 42

5. Introduction Methodology Experiments References
Research Projects
Electro-mechanical Processes, Biomanufacturing
Publications: IEEE Transactions, Physical Review, Physiological Measurements, Scientific
Reports
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 5 / 42

6. Introduction Methodology Experiments References
Predictive Modeling
Shrinkage methods: ridge regression, LASSO, and elastic net ...
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 6 / 42

7. Introduction Methodology Experiments References
Advanced Sensing
108.5°C
<60.0°C
CInfrared Camera Thermal Image
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 7 / 42

8. Introduction Methodology Experiments References
Advanced Sensing
(*from Yoram Rudy Lab @ WUSTL)
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 8 / 42

9. Introduction Methodology Experiments References
High-dimensional Predictive Modeling
BSPM y(s,t) Heart-surface Potential
Mapping x(s,t)
Inverse
Forward
Y (s, t) = RX(s, t) +
Traditional regression is not generally applicable!
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 9 / 42

10. Introduction Methodology Experiments References
Challenges
Spatially-temporally big data
Dimensionality
Velocity - sampling in milliseconds
Veracity - data uncertainty
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 10 / 42

11. Introduction Methodology Experiments References
Challenges
Complex structured systems
Complex geometries of AM builds
Complex torso-heart geometry
(*from CIMP-3D @ PSU)
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 11 / 42

12. Introduction Methodology Experiments References
Challenges
Y (s, t) = RX(s, t) +
Outer surface profiles y(s, t) ⇒ Inner surface profiles x(s, t)
Transfer matrix R ?
Physical principles
Additive manufacturing - Heat transfer model
Heart - Electrical wave propagation
Ill-conditioned system
Linear systems involving high-dimensional data
Condition number: cond(R) = R R−1
A measure of the relative sensitivity of the solution to changes in y
∆x
x
cond(R)
∆y
y
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 12 / 42

13. Introduction Methodology Experiments References
State of the Art
Tikhonov regularization
min
x(s,t)
{ y(s, t) − Rx(s, t) 2
2 + λ2
Γx(s, t) 2
2}
L1 regularization
min
x(s,t)
{ y(s, t) − Rx(s, t) 2
2 + λ2
Γx(s, t) 1}
Zeroth-order Γ = I
Directly penalize the magnitude of x(s, t)
Sparsity vs. Regularity
Not account for spatial or temporal correlations
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 13 / 42

14. Introduction Methodology Experiments References
State of the Art
First-order Regularization
The first-order derivative: Γx(s, t) = ∂x(s, t)/∂τ
Align x(s, t) in one column as {x(s1|t), x(s2|t), ..., x(sN |t)}T
Apply the bidiagonal gradient matrix
Normal derivative operator: Γx(s, t) = ∂x(s, t)/∂n
Γ =





−1 1
−1 1
...
...
−1 1




 n
τ
Γx(s,t)
Tangent plane
Need to fill the gaps
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 14 / 42

15. Introduction Methodology Experiments References
Physics-driven Spatiotemporal Regularization
Objective function
min
x(s,t)
T
t=1
{ y(s, t)−Rx(s, t) 2
+λ2
s ∆sx(s, t) 2
+λ2
t
t+ w
2
τ=t− w
2
x(s, t)−x(s, τ) 2
}
Parameter Matrix R - physics-based interrelationship
Spatial regularity - handle approximation errors by spatial correlation
Temporal regularity - model robustness to measurement noises
Algorithm - generalized dipole multiplicative update method
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 15 / 42

16. Introduction Methodology Experiments References
Parameter Matrix R
Divergence theorem: if F is a vector
field which is continuously differentiable
and defined on a volume V ⊂ R3 with a
piecewise-smooth boundary S, then
V
( · F )dV =
S
(F · n)dS
Electric Field Body Surface SB
Heart Surface SH
Green’s second identity: If φ and ψ are twice continuously
differentiable on V , and let F = φ ψ − ψ φ, then
S
(φ ψ − ψ φ) · ndS =
V
(φ 2
ψ − ψ 2
φ)dV
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 16 / 42

17. Introduction Methodology Experiments References
Parameter Matrix R
Heart - a bioelectric source
Torso - a homogeneous and isotropic volume conductor
(*from marvel.com)
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 17 / 42

18. Introduction Methodology Experiments References
Parameter Matrix R
Heart - a bioelectric source
Torso - a homogeneous and isotropic volume conductor
Green’s second identity:
S
(φ ψ − ψ φ) · ndS =
V
(φ 2
ψ − ψ 2
φ)dV
ψ = 1/r; φ = electric potentials
No electrical source between SH and SB: 2
φ = 0
Electric field outside SB is negligible: φ = 0 on SB
SH
n
SB
n
∇2
φ = 0
∇y(s,t)=0
y(s,t)
x(s,t)
dΩBB
dSB
dΩBH
dSH
Heart Surface
Body
surface
Volume
conductor
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 18 / 42

19. Introduction Methodology Experiments References
Parameter Matrix R
Boundary element method
Body surface potential on SB
y(s, t) = −
1
4π SH
x(s, t)dΩBH −
1
4π SH
x(s, t) · n
rBH
dSH +
1
4π SB
y(s, t)dΩBB
Heart surface potential on SH
x(s, t) = −
1
4π SH
x(s, t)dΩHH −
1
4π SH
x(s, t) · n
rHH
dSH +
1
4π SB
y(s, t)dΩHB
Numerical integration
ABBy(s, t) + ABHx(s, t) + MBHN(s, t) = 0
AHBy(s, t) + AHHx(s, t) + MHHN(s, t) = 0
Parameter matrix R:
R = (ABB − MBHM−1
HHAHB)−1
(MBHM−1
HHAHH − ABH)
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 19 / 42

20. Introduction Methodology Experiments References
Physics-driven Spatiotemporal Regularization
Objective function
min
x(s,t)
T
t=1
{ y(s, t)−Rx(s, t) 2
+λ2
s ∆sx(s, t) 2
+λ2
t
t+ w
2
τ=t− w
2
x(s, t)−x(s, τ) 2
}
Parameter Matrix R - physics-based interrelationship
Spatial regularity - handle approximation errors by spatial correlation
Temporal regularity - model robustness to measurement noises
Algorithm - generalized dipole multiplicative update method
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 20 / 42

21. Introduction Methodology Experiments References
Spatial Regularity
Surface laplacian ∆s for a square lattice
Surface laplacian at the node p0
x1 = x0 + a
∂x
∂v p0
+
1
2
a2 ∂2x
∂v2 p0
x2 = x0 − a
∂x
∂u p0
+
1
2
a2 ∂2x
∂u2 p0
x3 = x0 − a
∂x
∂v p0
+
1
2
a2 ∂2x
∂v2 p0
x4 = x0 + a
∂x
∂u p0
+
1
2
a2 ∂2x
∂u2 p0
⇒ x1 + x2 + x3 + x4 = 4x0 + a2
(
∂2x
∂u2
+
∂2x
∂v2
)
p0
= 4x0 + a2
∆x0
⇒ ∆x0 =
1
a2
(
4
i=1
xi − 4x0) =
4
a2
(¯x − x0)
Laplacian matrix for the square lattice
∆ij =



− 4
a2 , if i = j
1
a2 , if i = j, pj ∈ neighborhood of pi
0, otherwise
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 21 / 42

22. Introduction Methodology Experiments References
Spatial Regularity
Linear interpolation (imaginary nearest neighbor):
xt(j) = xt(i) +
¯di
dij
(xt(j) − xt(i))
dij is the distance between pi and pj
¯di = 1
ni
ni
j=1 dij
ni is the number of neighbors of pi
Surface laplacian at the node pi
∆sxt(i) =
4
¯di
2
(
1
ni
ni
j=1
xt(j) − xt(i))
=
4
¯di
(
1
ni
ni
j=1
xt(j)
dij
− (
1
di
)xt(i))
Laplacian matrix for 3D triangle mesh
∆ij =



− 4
¯di
( 1
di
), if i = j
4
¯di
1
ni
1
dij
, if i = j, pj ∈ neighborhood of pi
0, otherwise
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 22 / 42

23. Introduction Methodology Experiments References
Physics-driven Spatiotemporal Regularization
Objective function
min
x(s,t)
T
t=1
{ y(s, t)−Rx(s, t) 2
+λ2
s ∆sx(s, t) 2
+λ2
t
t+ w
2
τ=t− w
2
x(s, t)−x(s, τ) 2
}
Parameter Matrix R - physics-based interrelationship
Spatial regularity - handle approximation errors by spatial correlation
Temporal regularity - model robustness to measurement noises
Algorithm - generalized dipole multiplicative update method
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 23 / 42

24. Introduction Methodology Experiments References
Temporal Regularity
Spatiotemporal data x(s, t) and y(s, t) - dynamically evolving over
time and have temporal correlations
T
t=1
t+w
2
τ=t−w
2
x(s, t) − x(s, τ) 2
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 24 / 42

25. Introduction Methodology Experiments References
Physics-driven Spatiotemporal Regularization
Objective function
min
x(s,t)
T
t=1
{ y(s, t)−Rx(s, t) 2
+λ2
s ∆sx(s, t) 2
+λ2
t
t+ w
2
τ=t− w
2
x(s, t)−x(s, τ) 2
}
Parameter Matrix R - physics-based interrelationship
Spatial regularity - handle approximation errors by spatial correlation
Temporal regularity - model robustness to measurement noises
Algorithm - generalized dipole multiplicative update method
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 25 / 42

26. Introduction Methodology Experiments References
DMU Algorithm
Objective function - both spatial and temporal terms, and is difficult
to be solved analytically.
Iterative algorithm - traditional multiplicative update method
requires x(s, t) to be nonnegative
Heart surface - negative and positive electric potentials
A new dipole multiplicative update algorithm for generalized
spatiotemporal regularization
xt = x+
t − x−
t , x+
t = max{0, xt} x−
t = max{0, −xt}
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 26 / 42

27. Introduction Methodology Experiments References
DMU Algorithm
If we define
A = A+
− A−
= RT
R + λ2
s∆T
s ∆s + 2λ2
t wI
B = yT
t R + 2λ2
t
t−1
τ=t− w
2
xT
τ + 2λ2
t
t+ w
2
τ=t+1
xT
τ
The objective function can be rewritten as:
J =
T
t=1
{xT
t Axt − Bxt − xT
t BT
}
= ((xT
t )+
)A+
x+
t − ((xT
t )+
)Ax−
t − ((xT
t )−
)Ax+
t − ((xT
t )+
)A−
x+
t
+((xT
t )−
)A+
x−
t − ((xT
t )−
)A−
x−
t − B(x+
t − x−
t ) − (x+
t − x−
t )T
BT
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 27 / 42

28. Introduction Methodology Experiments References
DMU Algorithm
If we define
a+
i = (2A+
x+
t )i a−
i = (2A+
x−
t )i
b+
i = −(2Ax−
t )i − 2BT
i b−
i = −(2Ax+
t )i + 2BT
i
c+
i = (2A−
x+
t )i c−
i = (2A−
x−
t )i
New update rules
(x+
t )i ←
−b+
i + (b+
i )2 + 4a+
i c+
i
2a+
i
(x+
t )i
(x−
t )i ←
−b−
i + (b−
i )2 + 4a−
i c−
i
2a−
i
(x−
t )i
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 28 / 42

29. Introduction Methodology Experiments References
DMU Algorithm
Table: The Proposed Dipole Multiplicative Update Algorithm for STRE
1: Set constants λs, λt and w. Let
A = A+
− A−
= RT
R + λ2
s∆T
s ∆s + 2λ2
t wI
B = yT
t R + 2λ2
t
t−1
τ=t− w
2
xT
τ + 2λ2
t
t+ w
2
τ=t+1 xT
τ
2: Initialize {x+
t } and {x−
t } as positive random matrices.
3: Repeat
4: for i = 1, . . . , T do
(x+
t )i ←
(Ax−
t )i+Bi+ ((Ax−
t )i+Bi)2+4(A+x+
t )i(A−x+
t )i
(2A+x+
t )i
(x+
t )i
(x−
t )i ←
(Ax+
t )i−Bi+ ((Ax+
t )i−Bi)2+4(A+x−
t )i(A−x−
t )i
(2A+x−
t )i
(x−
t )i
5: end for
6: until convergence
7: Solution: ˆxt = x+
t − x−
t
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 29 / 42

30. Introduction Methodology Experiments References
Experiments - Simulation in a Two-sphere Geometry
Dynamic distributions of electric potentials on the inner surface
x(s, t) and outer surface y(s, t) are calculated analytically
x(s, t) =
1
4πσ
p(t) · rH (s)
r2
BrH
[
2rH
rB
+ (
rB
rH
)2
]
y(s, t) =
3
4πσ
p(t) · rB(s)
r3
B
Gaussian noise ∼ N(0, σ2) is added to y(s, t)
(a) (b)
Figure: (a) Parameters of the two-sphere geometry; (b) Each sphere is triangulated
with 184 nodes and 364 triangles
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 30 / 42

31. Introduction Methodology Experiments References
Results
σε
(a) (b)
Tikh-0thTikh-1st L1-1st STRE
RE
0
0.05
0.1
0.15
0.1 0.2 0.3 0.4 0.5
RE
0.08
0.1
0.12
0.14
0.16
0.18
0.2 Tikh-0th
Tikh-1st
L1-1st
STRE
Figure: (a) The comparisons of relative error (RE) between the proposed STRE model
and other regularization methods (i.e., Tikhonov zero-order, Tikhonov first-order and L1
first-order methods) in the two-sphere geometry when there is no noise on the potential
map y(s, t) of the outer sphere; (b) The comparisons of RE for different noise levels
σ = 0.1; 0.2; 0.3; 0.4; 0.5 on the potential map y(s, t) of the outer sphere.
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 31 / 42

32. Introduction Methodology Experiments References
Results
Dynamic distribution of electric potentials on the inner sphere x(s, t)
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 32 / 42

33. Introduction Methodology Experiments References
Results
Potential mapping on the inner sphere x(s, t), t = 150ms
Reference
Tikh_0th
RE=0.1475
Tikh_1st
RE=0.1026
L1_1st
RE=0.1025
STRE
RE=0.006
Tikh_0th
RE=0.208
Tikh_1st
RE=0.1528
L1_1st
RE=0.1569
STRE
RE=0.0769
(a)
(b)
(c)
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 33 / 42

34. Introduction Methodology Experiments References
Experiments - Realistic Torso-heart Geometry
Heart surface - 257 nodes and 510 triangles
Body surface - 771 nodes and 1538 triangles
y(s, t) - body area sensor network
Data uncertainty - gaussian noise ∼ N(0, σ2).
Five different noise levels: σ = 0.005, 0.01, 0.05, 0.1, 0.2
(a) (b)
Front Back
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 34 / 42

35. Introduction Methodology Experiments References
Results
σε
(a) (b)
Tikh-0th Tikh-1st L1-1st STRE
RE
0
0.05
0.1
0.15
0.2
0.25
0.3
0 0.05 0.1 0.15 0.2
RE
0.5
1
1.5
2
2.5
3
Tikh-0th
Tikh-1st
L1-1st
STRE
Figure: (a) The comparisons of relative error (RE) between the proposed STRE model
and other regularization methods (i.e., Tikhonov zero-order, Tikhonov first-order and L1
first-order methods) in the realistic torso-heart geometry when there is no extra noise on
the potential map y(s, t) of the body surface; (b) The comparisons of RE for different
noise levels σ = 0.005; 0.01; 0.05; 0.1; 0.2 on the potential map y(s, t).
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 35 / 42

36. Introduction Methodology Experiments References
Results
Dynamic distribution of electric potentials on the heart surface x(s, t)
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 36 / 42

37. Introduction Methodology Experiments References
Results
Potential mapping on the heart surface x(s, t), t = 50ms
Reference
Tikh_0th
RE=0.2488
Tikh_1st
RE=0.2839
L1_1st
RE=0.2735
STRE
RE=0.0997
STRE
RE=0.2386
Tikh_0th
RE=0.557
Tikh_1st
RE=0.972
L1_1st
RE=1.248
(a)
(b)
(c)
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 37 / 42

38. Introduction Methodology Experiments References
Summary
Challenges
Spatiotemporal data (predictor and response variables)
Complex-structured system
Ill-conditioned system
Methodology: Physics-driven spatiotemporal regularization
Parameter Matrix R - physics-based interrelationship
Spatial regularity - handle approximation errors by spatial correlation
Temporal regularity - model robustness to measurement noises
Algorithm - generalized dipole multiplicative update method
Significance
A novel approach to solve ECG inverse problem
A new dipole multiplicative update algorithm for generalized
spatiotemporal regularization
Broad applications: thermal effects in additive manufacturing
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 38 / 42

39. Introduction Methodology Experiments References
References
B. Yao, R. Zhu, and H. Yang*, “Characterizing the Location and Extent of Myocardial
Infarctions with Inverse ECG Modeling and Spatiotemporal Regularization,”IEEE Journal
of Biomedical and Health Informatics, page 1-11, 2017, DOI:
10.1109/JBHI.2017.2768534
B. Yao and H. Yang*, “Physics-driven spatiotemporal regularization for high-dimensional
predictive modeling,”Scientific Reports 6, 39012, 2016. DOI:
www.nature.com/articles/srep39012
B, Yao and H. Yang*, “Mesh Resolution Impacts the Accuracy of Inverse and Forward
ECG problems,”Proceedings of 2016 IEEE Engineering in Medicine and Biology Society
Conference (EMBC), August 16-20, 2016, Orlando, FL, United States. DOI:
10.1109/EMBC.2016.7591615
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 39 / 42

40. Introduction Methodology Experiments References
Acknowledgements
NSF CAREER Award
NSF CMMI-1617148
NSF CMMI-1646660
NSF CMMI-1619648
NSF IIP-1447289
NSF IOS-1146882
James A. Haley Veterans’ Hospital
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 40 / 42

41. Introduction Methodology Experiments References
Contact Information
Hui Yang, PhD
Associate Professor
Complex Systems Monitoring Modeling and Control Laboratory
Harold and Inge Marcus Department of Industrial and Manufacturing
Engineering
The Pennsylvania State University
Tel: (814) 865-7397
Fax: (814) 863-4745
Email: huy25@psu.edu
Web: http://www.personal.psu.edu/huy25/
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 41 / 42

42. Introduction Methodology Experiments References
Questions?
Hui Yang (PSU) Spatiotemporal Regularization November 25, 2017 42 / 42