1.
End-to-end Digital Integration based on Modular Simulation of Natural Human Motions
The MOSIM Project:
Combining digital human simulations for industrial applications
Felix Gaisbauer
Daimler AG
Unite, Copenhagen, September 25th, 2019
9/23/2019

2.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Digital Human Simulation for industrial
applications
Automotive assembly planning and the need for
sophisticated simulations
The MOSIM Project
Modularization of Digital Human Simulations
Partners and use cases
Technical realisation
Conclusion & Outlook
Agenda
19-09-23 2

3.
End-to-end Digital Integration based on Modular Simulation of Natural Human Motions
Digital Human Simulation for industrial applications
19-09-23
3

4.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Digital Human Simulations for industrial applications
Digital Human Simulations are utilized throughout
heterogeneous domains
Relevant topic for research, industry and
entertainment
Widely applied since 1980s
Several commercial and non-commercial tools
19-09-23 4

5.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix Gaisbauer
Digital Human Simulations for industrial applications:
Product Design
Comfort
Visibility
Ingress and egress
Reaching and grasping
Multi-person interaction
Strength assessment
19-09-23 5
Hightech ergonomics with RAMSIS, Human Solutions
Human Solutions Group
Source: https://www.youtube.com/watch?v=5K0-HR7wKrI
Categorization based on
Digital Human Modeling for Design and Evaluation of Human-Machine Systems ANJA NAUMANN1 & MATTHIAS ROETTING2

6.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix Gaisbauer
Digital Human Simulations for industrial applications:
Manufacturing
Work cell layout
Workflow simulation
Assembly accessibility
Safety analysis
Lifting
Energy expenditure
Simulation based training
19-09-23 6
Simulation of Seat Assembly using Automatic Computed Motions
Fraunhofer Chalmers Sw
Source: https://www.youtube.com/watch?v=u91UKiJDMFk
Categorization based on
Digital Human Modeling for Design and Evaluation of Human-Machine Systems ANJA NAUMANN1 & MATTHIAS ROETTING2

7.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix Gaisbauer
Digital Human Simulations for industrial applications:
Motion Analysis & Medical Applications
Highly detailed musculoskeletal simulations
Detailed analysis of the underlying processes
Sport, Medical analysis
19-09-23 7
Human simulation in AnyBody: Overview of MoCap features
AnyBody Technology
Source: https://youtu.be/JxKsv9XDP-Y

8.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Digital Human Simulations for industrial applications:
Infotainment & Human Computer Interaction
Intuitive interaction with digital representation
Maximizing user experience
19-09-23 8
Daimler Financial Services: New Customer Experience with Digital Human ”Sarah“
Daimler AG
Source: https://www.youtube.com/watch?v=r2rCbSXZ4jE

9.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Digital Human Simulations for industrial applications:
Entertainment/Gaming
Creation of lifelike characters
Movies
Gaming
Advertisement
19-09-23 9
The Heretic: Unity GDC 2019 reveal
Unity Technologies
https://youtu.be/34K8YJOMDRY

10.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Digital Human Simulations at Daimler
Digital Human Simulation plays an important role for
many use cases and domains at Daimler:
Product design
Manufacturing
Autonomous driving
Customer experience
19-09-23 10
Source: Daimler AG, https://www.daimler-protics.com/projekte/story/ergonomiesimulationen.html

11.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Summary: Benefits of 3D human models
System design: consideration of age groups
Objective, repeatable statements about
comfort
Meaningful simulation results (physics)
Realistic, risk-free simulations
Diagnostic support
Simulation environments with avatars cost
efficient
Increased entertainment value
19-09-23 11

12.
End-to-end Digital Integration based on Modular Simulation of Natural Human Motions
Automotive Assembly Planning and the need for
sophisticated Simulations
19-09-23
12

13.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Challenges of Automotive Assembly Planning
Increasing amount of models and derivatives
New technologies and product concepts
Mixed model production lines
Need for increased efficiency and flexibility
due to growing number of competitors
➢ Current production planning methods
increasingly tend to fail at representing the
reality and induce high efforts
➢ Manual planning methods must be replaced by
computer aided, digital planning tools in order
to be efficient and competitive
19-09-23 13
Source: Daimler AG

14.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Digital Human Simulations as key technology
for assembly planning
▪ Digital human simulations allow to model the
behavior of human beings with regard to specific
metrics (such as ergonomics, buildability)
▪ Benefits
Simulation of hypothetical scenarios
Comprehensive assessment and optimization
based on several metrics
No need for manual testing
No need for physical Hardware
Faster planning iterations & reduced costs
19-09-23 14
Source: Intelium Engineering AB Sweden, https://www.youtube.com/watch?v=Wjf_ilrAfRw

15.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Digital Human Simulations as key technology
for assembly planning
▪ Manual assembly comprises strongly
heterogeneous tasks and motions
▪ Present simulation tools are specialized on a
particular aspect (e.g. buildability, ergonomics)
▪ For a comprehensive assessment multiple
metrics and facets must be considered
▪ Currently no tool is available which accurately
addresses all required aspects
19-09-23 15
Join parts (Physics
based)
Walking (Deep Learning)
Pick up part (Data-
driven)
Source: https://www.youtube.com/
watch?v=Ul0Gilv5wvY
Source: https://www.youtube.com/
watch?v=Wjf_ilrAfRw
Source: https://www.youtube.com/
watch?v=gZceX7fIFYQ

16.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
To allow a realistic simulation of complex human motions different
approaches need to be applied & combined
16
Target Engine
“Assemble Part A and
Part B with three screws”
“Pick up tool” “Walk to part B”
Motion blending
Machine
Learning
Physics based Physics based
“Enter car” “Fasten 3x”
Java C# Python C++
19-09-23

17.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Lacking availability of frameworks/comprehensive simulation tools
“But why is currently no tool/framework available
which covers all needs, if the required technologies
are principally available?”
19-09-23 17

18.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Barriers for third party providers
Uneconomically high porting efforts to
incorporate new technologies
Lack of knowledge (vendors focus on a
particular technology)
Latest technologies are not available as open
source code
➢ By addressing these issues, a software tool
providing the required functionality can be
principally realized, ultimately enhancing digital
planning of manual assembly processes
19-09-23 18
Source: http://ecrivain-alchimiste.fr/bloguer-trouver-ses-lecteurs/
Source: https://letgodbefoundtrue.net/2015/11/13/a-lack-of-knowledge/
Source: https://www.jing.fm/iclipt/u2q8o0o0u2u2a9a9/

19.
End-to-end Digital Integration based on Modular Simulation of Natural Human Motions
The MOSIM Project
Modularization of Digital Human Simulations
19-09-23
19

20.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
MOSIM Idea
Reduce the barriers for the developers
Establish a standard for exchanging digital
human simulation approaches
Provide a framework for the combination of
heterogeneous systems
Increase applicability for manual assembly
planning and further use cases
Combine the different approaches in a
sophisticated target engine such as Unity
19-09-23 20

21.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
But How to combine multiple DHM systems?
Problem can be considered as technical
combination of different simulation
approaches
The FMI standard has already addressed a
similar problem in a different domain
21
Source: https://fmi-standard.org/
19-09-23

22.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
The ITEA projects MODELISAR, AVANTI and ENTOC introduced a
modular co-simulation framework for products and production equipment
22
Functional Mock-Up
Co-Simulation of the FMI Simulation Models
Engine Engine control Gear box Gear box control Vehicle Interior
19-09-23

23.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
MOSIM will transfer this idea to the field of human motion synthesis
23
Motion Model Unit
Co-Simulation of the MMU Simulation Models
„Walk“ „Grasp“ „Drill“ „Pick-up“ „Fasten“
19-09-23

24.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
In contrast to human motion, a modular simulation of products and
production equipment is already possible
Modularization of product simulation
Modul 1
Modul 2 Modul 3
Modul 4
Modul 5
Modul 6
Modularization of process simulation
?
2419-09-23

25.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
In contrast to human motion, a modular simulation of products and
production equipment is already possible
Modularisierung in der Produktsimulation
Modul 1
Modul 2 Modul 3
Modul 4
Modul 5
Modul 6
Modularization of product simulation
Modul 1
Modul 2 Modul 3
Modul 4
Modul 5
Modul 6
Modularization of process simulation
?
2519-09-23

26.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
MODELISAR
(2008-2011)
•Development of
the FMI standard
•Co-simulation of
independent
systems in
heterogeneous
environments
AVANTI
(2013-2016)
•Development of
virtual
commissioning
test methodology
•Using FMI to
combine different
models and tools
into one
application
ENTOC
(since 2016)
•Toward
automated
generation of
virtual production
systems
•FMI is used to
simulate
production
devices via a
standardized
interface
MOSIM
(since 2018)
•Modular
simulation of
natural human
motions
•Developing the
MMI standard
following the FMI
idea of model
encapsulation
The Road To MOSIM
19-09-23 26
History of - projects

27.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Our Project Vision
19-09-23 27
An open modular framework for efficient and
interactive simulation and analysis of realistic human
motions for professional applications.

28.
End-to-end Digital Integration based on Modular Simulation of Natural Human Motions
The MOSIM Project
Partners and Use Cases
19-09-23
28

29.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
21 Partners from 4 different countries
+ Unity as supporting partner
2919-09-23

30.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
End-User
(OEM)
Vendor of simulation /
animation software
Animation and Technology Provider
The MOSIM Project creates a new Market Value Chain
30
CompanySize
Market value chain
Animation provider
community
Distributor MMU
library Provider of tailored
components
Products
19-09-23

31.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
The MOSIM Project addresses different use cases
19-09-23 31

32.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
19-09-23 32
Final Assembly at Mercedes-Benz Cars

33.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
19-09-23 33
Final Assembly at Volvo Group Trucks Operations

34.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
19-09-23 34
Wood processing

35.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Pedestrian Simulation
19-09-23 35

36.
End-to-end Digital Integration based on Modular Simulation of Natural Human Motions
End-to-end Digital Integration based on Modular Simulation of Natural Human
Motions
The MOSIM Project
Technical Realisation
19-09-23
36

37.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Motion Model Units (MMUs)
Represent the basic component encapsulating
the specific simulation approach
Motion Model Unit contains the digital human
simulation approaches
Each MMU is realized in the desired
programming language (e.g., C#, C++, Python)
Encapsulation by a standardized interface,
which allows to access/set the relevant values
Protection of intellectual property by black box
realization
19-09-23 37

38.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Technical Framework: Combining heterogeneous approaches in
Unity
19-09-23 38
Target Engine
Behavior
Modelling
Co-Simulation
C++
Adapter
C++ MMUs
C#
Adapter
C# MMUs
Services
Intermediate
Skeleton
Scene
Middleware
Communication
layer
Communication
layer

39.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Intermediate Skeleton
Defined skeleton (hierarchy and joint
nomenclature) for exchange of posture
information in the framework
Proposal of Intermediate Skeleton created by
expert group in project
Optional retargeting to internal skeleton if
required (e.g. MMU or target engine)
Accuracy is delimited by skeletal representation
19-09-23 39

40.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
How to orchestrate multiple MMUs?
▪ Technical framework incorporates different
approaches in a common platform…
…But how to generate feasible motions from
these approaches?
▪ A Co-Simulation which orchestrates and
merges multiple MMUs is required
19-09-23 40

41.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
How to determine the required sequence of motions?
Co-Simulation orchestrates the modules based
on a given sequence of instruction
Instructions must be generated depending on
the simulation context
Behavior Modeling/Execution is required
19-09-23 41

42.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Overall Vision
Establishing a standard for the exchange and
utilization of digital human simulation
approaches
End-users can generate comprehensive
simulations for their respective use-case
Developers of digital human simulation
approaches can easily deploy their solutions to
a large repertoire of consumers/products
Developers can easily create MMUs using the
desired platform
MMUs can be purchased in an App/MMU Store
42
MMU Store
Walk Pick Up Place
Gaze Reach Point
Touch Fasten Drill
19-09-23

43.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Current Status
Project running since November 2018
Use cases and requirements defined in detail
First draft of framework specification
First specification publicly available in October
(see itea3.org/project/mosim.html,
www.mosim.eu)
19-09-23 43

44.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Current Status - Showcase
44
Walking (AI-based) MOSIM Demonstrator
19-09-23

45.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Current Status - Showcase
4519-09-23

46.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Conclusion & Outlook
The MOSIM Project represents an important
step towards modularization of digital human
simulations
Next steps
Implementation of the Framework & Toolkit
Realization of algorithms (data-driven model based)
contained in the MMUs
Development of demonstrators for the different use
cases
Standardization
19-09-23 46

47.
The MOSIM Project: Combining digital human simulations for industrial applications, Felix
Gaisbauer
Thank you very much!
If you should have any question, please do not hesitate to contact us at the Unite
47
Eva Lampen
Felix Gaisbauer
Jannes Lehwald
We acknowledge the financial support by the Federal Ministry of
Education and Research of Germany (grant number 01IS18060A-H).
19-09-23