VEDLIoT Use Cases. Micha vor dem Berge. NGIoT session at IoT Week, Dublin, Ireland, June 2022

1. Micha vor dem Berge
Teaching the IoT to learn
or
How to optimise a Smart Home

2. 2
Hardware
Platforms
Applications
Microservers &
Accelerators
Middleware
IoT /
Embedded
Edge Cloud
Jetson AGX
COM-HPC
x86, ARM,
FPGA
SMARC
x86, ARM,
FPGA
Hailo-8
RPi CM4
RISC-V
Smart Home Industrial IoT Automotive AI
Open
Call
AI Optimizer Emulation Benchmarking & Deployment
u.RECS t.RECS RECS|Box
Security & Safety
Requirements
Modelling
&
Verification
Safety
&
Robustness
Monitoring
Trusted
Execution
&
Communication
RISC-V
extensions

3. 3
Industrial IoT Automotive AI
Motor Condition
Classification
Challenges
Battery Powered,
Power Consumption
Achievements
Captured data sets, first
DL model
Arc Detection in DC
Power Distributions
Challenges
Accuracy,
Response Time
WiP
Creating good
learning data
Automatic Pedestrian
Emergency Breaking
Challenges
Response Time,
Robustness,
Distribution of work
Achievements
Captured data sets on airfield,
first DL model
Industrial IoT

4. 4
A “typical” smart home
4
Smart Kitchen
• Gesture, speech recognition
Intelligent Door
• Face, obstacle recognition
Dialog assistance
• Face, speech recognition
Smart Mirror
• Gesture, face, object
and speech recognition
Intelligent Chair and Fitness coach
• Gesture, face and speech recognition

5. 5
 Display coated with a semi-transparent foil
 Own mirror image + digital overlay
 Intuitive control via gesture and voice
 Show personalized information
 Gaelic football scores, weather, news, …
 Data Privacy as highest priority
 Simultaneously computation of many
neuronal networks in the edge
Smart Mirror as Central Demonstrator

6. 6
 Optimization of Neural Networks
 Use EmbeDL toolchain
 Distribute on embedded accelerators
 Generate Virtual Mirror Image
 3D representation
 Frontal view without “real” mirror
 Develop Secured Local Voice Assistant
 Speech to Text, Text to Speech, NLP
 Use VEDLIoT project hardware
 t.RECS and u.RECS
 Increase energy efficiency
Development & Optimisation Goals
Display Messages from/
to other devices
Face Recognition
Object Detection
YOLO
Gesture Detection
Smart Home
DNN
Efficientnet
Trained with WiderFace
Peripheral Input
Combinatorial Logic
Decision Maker
RGB-D Camera
ROS2 Bridge
Pointcloud
Registration/
Alignment
Person
Recognition
MagicMirror²
ROS2
/
t.RECS
Speaker
ROS2
/
u.RECS
Siamese Network
for Identification
Emotion
Recognition
Mask Detection
Tracker
(Sort)
RGB-D Camera
Intent Extraction
Skill System
Text to Speech
Tracker
(Sort)
Pointcloud Fusion for
Virtual Mirror image
Webstream
Microphone Input
Speech to Text

7. 7
Increase of energy efficiency
First Prototype
• Intel i7-7700K
• 2x GTX 1080 TI
• 16 FPS / 600 Watt
-> 37.5 Watt / Frame
Second Prototype
• Intel i9-9900k
• 2x RTX 2070
• 27 FPS / 450 Watt
Third Prototype
• 2x Nvidia Xavier
• 16 FPS / 55 Watt
Actual Prototype
• 2x Nvidia Xavier
in christmann t.RECS
• 28 FPS / 55 Watt
-> 1.9 Watt / Frame
19x
increase

8. 8
Thank you for your
attention
Micha vor dem Berge
christmann IT, Germany
micha.vordemberge@christmann.info