MAMOC is a machine learning application that uses motion capture to provide digital process data and optimization for manual production stations. It uses an Intel RealSense camera to record depth video and identify objects, hand poses, and actions. The application was tested by assembling a speaker kit and showed potential to integrate human processes with digital workflows, optimize throughput and quality, and achieve an accuracy level of TRL 3-4. Further development is needed to improve action detection accuracy and integration could provide benefits for industrial engineering optimization projects.

1. MAMOC –
Machine Learning Application for Motion Capture
Speaker: Michael Spijkerman

2. Your partner for software
development and consulting
In our development center and at
your site
• Software conception and development - tailored
to your needs
• Provision of individual specialists and entire
teams for your projects
• Experienced certified project management
• Your extended workbench - taking over task
packages
Tailored to your needs
• Quality, Security, Proof-of-Concepts,
Full agile customer proximity

3. The challenge – what happens on manual
production stations?
Input Output
Within a process step several actions are performed in a
meaningful order
A machine:
- Highly Integrated to digital production processes
- By digital Status Feedback (started, stopped, error, …)
- Optimized for best throughput and quality
A human
- Does things a machine cannot do

4. Input Output
We just need a smart supervision sensor
- provides digital process relevant Data
- Provide data for optimization projects (i.e. industrial engineering)
Challenge accepted!
Object
Detection
+ +
Hand-Pose
Estimation
Action
Recognition
Record
Depth-
Video
X
Z
Y
X
Z
Y
Our Approach:
- Not only RGB Video Material but also:
- Identified Objects
- Identified Handposes
- Depth Image
- Location data of the Hands and Objects

5. The experiment
Setup performed
Converted loupe lamp with Intel Realsense D435i

6. The experiment
Example chosen
Assembling a speaker kit

7. The experiment
Trained Object Detection
YOLOv4 (Cross Stage Partial Network, CSPNet) executed on darknet

8. The experiment
Enabled Hand Pose Estimation
MediaPipe Hands

9. The experiment
Implemented Action Detection Solution
Adjusted 2 Stream Advanced Graph Convolutional Network (2s-AGCN)

10. The experiment
Implement the MAMOC Framework
MIDIH Stack
Using MediaPipe Framework with some additional calculators

11. The experiment
Trained the Action Detection
VGG Annotator

12. The experiment
The feasibility of our approach could be proven!
Test and Validation

13. Conclusion
simple to mount
easy integration through midih compatibility
MIDIH Stack
cheap embedded Hardware
Input Output
A human:
- can be highly Integrated to
digital production processes
- can be optimized for best
throughput and quality
What we reached
- planned accuracy of object detection and hand
pose Estimation
- MIDIH Integration
- Self assessment: TRL 3-4
next Steps
- work on action detection accuracy
- (Applied) funding at federal ministry of
education and research “KI4KMU” program to
reach TRL 7
Presentations
- digital Hannover fair 2020
- 18th (Digital) Trade Congress: "Efficient and
innovative mechanical engineering
- Regional workgroup KI in medium-sized
businesses
- (applied) Embedded Software Engineering
Congress digital

14. THANK
YOU!
dmc-smartsystems GmbH Developement Center
Paderborn
Valentin-Linhof-Str. 8
81829 München
Tel: +49 89 42774-0
Fax: +49 89 42774-199
Ahornallee 9
33106 Paderborn
Tel: +49 5251 6824-441
info@dmc-smartsystems.dmc-group.com
Michael.spijkerman@dmc-group.com
www.dmc-smartsystems.com
MAMOC presentation:
https://youtu.be/Lzp5al4b3IQ
MIDIH marketing:
https://youtu.be/5GAsUu-uNhI