Engineering sets specific demands on data-driven methods. I will discuss how data-driven projects can benefit from technical expertise. In addition, using a real-life case, I will show, how the combined application of computational fluid dynamics and machine learning helps to design aircraft of the future. Website: https://fwdays.com/en/event/data-science-fwdays-2019/review/data-driven-solutions-in-engineering

1. DATA-DRIVEN SOLUTIONS IN
ENGINEERING
Vladislav Rosov
LAMAITA

2. First Industrial Revolution
Mid-18th – mid-19th century
07.09.2019 Vladislav Rosov | Data Science fwdays’19

3. Second Industrial Revolution
End of the 19th century
07.09.2019 Vladislav Rosov | Data Science fwdays’19

4. Third Industrial Revolution
Mid-20th century
07.09.2019 Vladislav Rosov | Data Science fwdays’19

5. Fourth Industrial Revolution or Industry 4.0
• Industrial Internet of Things
– Machines augmented with wireless
connectivity and sensors
– Data-lakes
• Cyber-physical systems
– Computer-aided toolchains
– Digital twins
07.09.2019 Vladislav Rosov | Data Science fwdays’19

6. New demands on data scientists
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Engineering
Science
Computer
Science
Math
Computational Science/
Data-driven Engineering

7. Example: Single Input – Single Output (SISO) System
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Input signal
Output signal

8. System identification: Training
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Model: LSTM
• Layers: 1
• Units: 128
• Delays: 20
• Loss: MSE
• Batch size: 32
• Epochs: 10
Training Data:
• Signal: APRBS
• Samples: 14,000

9. System identification: Test with Sine 2.5 Hz
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Test Data:
• Frequency: 2.5 𝐻𝑧

10. System identification: Test with Sine 2.5 Hz
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Test Data:
• Frequency: 2.5 𝐻𝑧

11. System identification: Test with Sine 5.0 Hz
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Test Data:
• Frequency: 5.0 𝐻𝑧

12. System identification: Test with Sine 5.0 Hz
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Test Data:
• Frequency: 5.0 𝐻𝑧

13. System identification: Test
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Resonance !!!
Test Data:
• Frequency: 5.0 𝐻𝑧

14. Example: Single Input – Single Output (SISO) System
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Input signal
Output signal

15. Example: Single Input – Single Output (SISO) System
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Input signal
Output signal
𝑥
𝐹

16. Mathematical model
07.09.2019 Vladislav Rosov | Data Science fwdays’19
𝑥
𝐹
𝑚
𝑑2 𝑥
𝑑𝑡2
+ 𝑑
𝑑𝑥
𝑑𝑡
+ 𝑘𝑥 = 𝐹
𝑑𝑥
𝑑𝑡
𝑡 ≈
𝑥 𝑡+Δ𝑡 − 𝑥 𝑡−Δ𝑡
2Δ𝑡
𝑑2 𝑥
𝑑𝑡2 𝑡 ≈
𝑥 𝑡+Δ𝑡 − 2𝑥 𝑡 + 𝑥 𝑡−Δ𝑡
Δ𝑡2
𝑥 𝑡+Δ𝑡 = 𝑎1 𝑥 𝑡 + 𝑎2 𝑥 𝑡−Δ𝑡 + 𝑎3 𝐹𝑡 Step-by-step procedure

17. System identification: New approach
07.09.2019 Vladislav Rosov | Data Science fwdays’19
LSTM
𝑥
𝐹
Lessons learned:
• System feedback has to be considered for proper modeling
• Mere 2 previous time steps are sufficient as input
𝑥 𝑡+Δ𝑡 = 𝑎1 𝑥 𝑡 + 𝑎2 𝑥 𝑡−Δ𝑡 + 𝑎3 𝐹𝑡𝑥

18. System identification: Test with Sine 2.5 Hz
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Test Data:
• Frequency: 2.5 𝐻𝑧

19. System identification: Test with Sine 5.0 Hz
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Test Data:
• Frequency: 5.0 𝐻𝑧
Resonance is
predicted

20. Mathematical modeling vs. Machine Learning
07.09.2019 Vladislav Rosov | Data Science fwdays’19
𝑚
𝑑2
𝑥
𝑑𝑡2
+ 𝑑
𝑑𝑥
𝑑𝑡
+ 𝑘𝑥 = 𝐹
𝑥𝑡+Δ𝑡
= 𝑎1 𝑥𝑡 + 𝑎2 𝑥𝑡−Δ𝑡 + 𝑎3 𝐹𝑡
#time integration
for t in range(1,len(x)-1):
x[t+1] = a1*x[t] + a2*x[t-1] +
a3*force[t]
return x
𝒙(𝒕)
Mathematical modeling Numerical modeling SimulationNatural phenomena Code implementation
and validation

21. Example of a mathematical model
07.09.2019 Vladislav Rosov | Data Science fwdays’19
J. Weirather *, V. Rozov* et al. A Smoothed Particle Hydrodynamics Model for Laser Beam
Melting of Ni-based Alloy 718. Computers and Mathematics with Applications (2018)
*equal contribution
• Computation of Navier-Stokes
Equations by Smoothed Particle
Hydrodynamics (SPH)
• C++ code with CUDA acceleration
• Approx. 10 mio. SPH-particles
• Approx. 4 weeks on a NVIDIA
GeForce GTX 1080 Ti
Thermo-Fluid-Dynamic Modeling of Laser Beam Melting

22. Mathematical modeling vs. Machine Learning
07.09.2019 Vladislav Rosov | Data Science fwdays’19
𝑚
𝑑2
𝑥
𝑑𝑡2
+ 𝑑
𝑑𝑥
𝑑𝑡
+ 𝑘𝑥 = 𝐹
𝑥𝑡+Δ𝑡
= 𝑎1 𝑥𝑡 + 𝑎2 𝑥𝑡−Δ𝑡 + 𝑎3 𝐹𝑡
#time integration
for t in range(1,len(x)-1):
x[t+1] = a1*x[t] + a2*x[t-1] +
a3*force[t]
return x
𝒙(𝒕)
Mathematical modeling Numerical modelingNatural phenomena
𝑭(𝒕)
𝒙(𝒕)
PredictionModel implementation
and training
𝒙(𝒕)
SimulationCode implementation
and validation

23. Mathematical modeling & Machine Learning
07.09.2019 Vladislav Rosov | Data Science fwdays’19
𝑚
𝑑2
𝑥
𝑑𝑡2
+ 𝑑
𝑑𝑥
𝑑𝑡
+ 𝑘𝑥 = 𝐹
𝑥𝑡+Δ𝑡
= 𝑎1 𝑥𝑡 + 𝑎2 𝑥𝑡−Δ𝑡 + 𝑎3 𝐹𝑡
#time integration
for t in range(1,len(x)-1):
x[t+1] = a1*x[t] + a2*x[t-1] +
a3*force[t]
return x
𝒙(𝒕)
Mathematical modeling Numerical modelingNatural phenomena
𝑭(𝒕)
𝒙(𝒕)
PredictionModel implementation
and training
𝒙(𝒕)
Physics-aware neural networks
SimulationCode implementation
and validation

24. Machine Learning with Physics-Awareness (PA)
07.09.2019 Vladislav Rosov | Data Science fwdays’19
PA-based loss functions with PA:
𝐿 𝑃𝐴 = 𝑥 𝑡+Δ𝑡 − 𝑎1 𝑥 𝑡 − 𝑎2 𝑥 𝑡−Δ𝑡 − 𝑎3 𝐹𝑡
𝑚
𝑑2 𝑥
𝑑𝑡2
+ 𝑑
𝑑𝑥
𝑑𝑡
+ 𝑘𝑥 = 𝐹
𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙 = 𝑚
𝑑2
𝑥
𝑑𝑡2 + 𝑑
𝑑𝑥
𝑑𝑡
+ 𝑘𝑥 − 𝐹
𝑅𝑒𝑠𝑖𝑑𝑢𝑎𝑙 = 𝑥 𝑡+Δ𝑡 − 𝑎1 𝑥 𝑡 − 𝑎2 𝑥 𝑡−Δ𝑡 − 𝑎3 𝐹𝑡 Finite Differences

25. PA-based choise of input features:
• Numerics: Spring-Mass System
• Mathematical structure: Elliptic boundary value problem
Machine Learning with Physics-Awareness (PA)
Δ𝑇 = 0
𝜕Ω
𝑥 𝑡+Δ𝑡 = 𝑎1 𝑥𝑡 + 𝑎2 𝑥 𝑡−Δ𝑡 + 𝑎3 𝐹𝑡
Inputs: 𝑥𝑡, 𝑥𝑡−Δ𝑡, 𝐹𝑡
Inputs: 𝑇(𝜕Ω)
07.09.2019 Vladislav Rosov | Data Science fwdays’19

26. Machine Learning with Physics-Awareness (PA)
PA-based model architecture:
Causality
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Causal Convolutions:
Space
Time
Past
Future

27. Real-World Example from Aviation
07.09.2019 Vladislav Rosov | Data Science fwdays’19

28. Aircraft Design
07.09.2019 Vladislav Rosov | Data Science fwdays’19

29. Example: data-driven aerodynamic identification
07.09.2019 Vladislav Rosov | Data Science fwdays’19

30. Aerodynamics in a nutshell
07.09.2019 Vladislav Rosov | Data Science fwdays’19
Pitch angle 𝛼
Inflow

31. Aerodynamics in a nutshell
07.09.2019 Vladislav Rosov | Data Science fwdays’19

32. Aerodynamics in a nutshell
07.09.2019 Vladislav Rosov | Data Science fwdays’19

33. Aerodynamics in a nutshell
07.09.2019 Vladislav Rosov | Data Science fwdays’19

34. Thank you for your attention!
Web: www.lamaita.de
07.09.2019 Vladislav Rosov | Data Science fwdays’19