This document summarizes a presentation by Jan Groenefeld on multimodal, collaborative, and autonomous human-machine interaction. It discusses the need for multi-experience strategies that go beyond isolated touchscreen interactions. Key points include the growth of interconnected devices, using multiple human senses like sight, sound, touch, smell and taste to enhance communication between humans and machines, and applying service design methods to address complexity. The presentation advocates becoming a first mover in this area and partnering with experienced design firms to overcome hurdles.

1. Jan Groenefeld • IoT Meetup Remote Session • October 20th, 2020
Multimodal, Collaborative
& Autonomous
Multi-experience strategies as the key to self-
determined interaction with intelligent machines

2. THE SAD ROBOT
AND THE BRAVE GIRL.
FULL VIDEO: https://youtu.be/puDF57e_t_c

3. 75BILLION
interconnected devices
until 2025
TECHNOLOGY,
EVERYWHERE!
60BILLION €
market volume forecast
until 2024

4. STATE OF THE ART
“NICE TOUCH SCREEN”

5. NEXT LEVEL
“NATURAL COMMUNICATION”
https://www.daimler.com/innovation/production/factory-56.html

6. https://www.it-production.com/news/maerkte-und-trends/microsoft-iot-signals-report/
Bildquelle: https://www.bosch-presse.de/pressportal/de/de/wandel-von-bosch-zum-iot-unternehmen-137474.html
SMART THINGS &
INTERACTIVE
ENVIRONMENTS
#3

7. Bildquelle: https://www.powertransmissionworld.com/egp-c-co-act-by-schunk-the-first-certified-industrial-gripper-for-collaborative-operation/
ROBOTS &
COBOTS
#2

8. Bildquelle: https://www.geospatialworld.net/blogs/wearable-rape-sensor/
AUGMENTED HUMAN &
WEARABLE BIONICS
#1

9. KEY IS MULTI-
EXPERIENCE

10. - David Cearley, Gartner Distinguished VP Analyst, Gartner
“
”
…shift from technology-literate to
people-literate technology.
Translating intent will move
from user to computer.
Communication across
many human senses will enhance
nuanced information.

11. - David Cearley, Gartner Distinguished VP Analyst, Gartner
“
”
…shift from technology-literate to
people-literate technology.
Translating intent will move
from user to computer.
Communication across
many human senses will enhance
nuanced information.
TIME OF ISOLATED TOUCHSCREEN
INTERACTION IS OVER!

12. Bildquelle: https://www.express.co.uk/life-style/science-technology/576707/Minority-Report-Gloves-Military-General-BAE-Systems
MR. MULTI-
EXPERIENCE

13. TAKE UP
STARTING POSITION
TOP 3 HURDLES

14. Taking the Hurdles
1. Promising technologies?
‣ Get dirty and gain experience
2. Right steps first?
‣ Apply Service Design methods.
3. How to move forward?
‣ Go get a competent partner.

15. SERVICE
DESIGN
SPOTLIGHT ON

16. SERVICE
DESIGN
UX
DESIGN
Holistic
Multiple Touchpoints
Open
Broad Research
Value-based
Strategic
Visionary
Concrete
Defined Touch Points
Focused Research
Requirement-based
Detail Design
UI Specification
Handover to Developers

17. Service Design
Fact Sheet
‣ Holistic Design Approach
‣ Common strategy and vision.
‣ Integrates multiple touchpoints.
‣ Super Powers
‣ Balances business models and
user acceptance and stakeholder buy-in.
‣ Eases complex design decisions reflecting
human sensing abilities.

18. OUR
SENSES
SEE
LISTEN
FEEL
TASTE
SMELL
Bildquelle: https://www.kurier.de/inhalt.geschaerfte-sinne-sinnesorgane-mit-ueberdurchschnittlichen-faehigkeiten.2b2d38cd-2f71-4ef2-aa14-e183a596e0f8.html

20. CONTEXT
EXPERIENCE
MODALITY
CONDITIONS
INSIGHTS &
OPPOS

21. EXPERIENCE
MODALITY
CONDITIONS
INSIGHTS &
OPPOS
Sensing
Understanding/
Deciding
Acting
Experience Goals
Eyes & ears
blocked
Loud &
hectical
No view
because
Smoke
Need for
stress relief
Dim the
alarm
Extended view
through drone
Guided restart
via voice control
Doing this &
that
Alarm
occurs!

22. From Research to Strategy
With our outside-in perspective, we
collect valuable insights to identify
weaknesses in your customer journey
to improve it most efficentially.
Insights &
Opportunities
Internal & External investigation
Trends
Benchmarking
Insight & Opportunity Report
We create a holistic view of the experience
your customers currently have with you.
With this common view we create together
with you innovative solutions to deliver a
desirable experience which is aligned with
your business
Holistic picture of
current experience
Personas
Service Experience Mapping
Customer Journey (Current)
Eco-system map
Envision future
experience
We visualise your improved service experience
shows how to achieve a service eco-system
which continuously delivers value for your
customers, your business and for your
employees.
Blueprint
Customer Journey (Future)
Value proposition
Roadmap
Design Vision
LEARN MORE ABOUT ERGOSIGNS FULL SERVICE UX DESIGN PORTFOLIO

23. MAKE YOUR IDEAS
COME ALIVE

24. MAKE YOUR IDEAS
COME ALIVE
+

25. Hot & Spicy
Take Aways
‣ Create holistic, innovative and supportive
Interaction Systems using all human senses
instead of limiting to touch interaction
‣ Apply Service Design Methods to cover
complexity
‣ Be a first mover, not a follower and get an
external risk manager like Ergosign
‣ Don’t forget to have fun with
technology!

26. Meet us in our 6 offices in
Germany and Switzerland
for an inspiring chat.
Quelle: Ergosign GmbH

27. TECHNOLOGY
LET’S ENSURE WE
WILL LOOK GOOD!
IS THE MIRROR
OF SOCIETY.

28. THANKS A
LOT!
Jan Groenefeld
Solution Manager Industry
Lead UX Designer
jan.groenefeld@ergosign.de
T +49 681 95 98 20 0
READING RECOMMENDATION ON THE TOPIC
https://www.linkedin.com/in/jan-groenefeld-8833947b/
REACH OUT VIA LINKEDIN

29. JAN GROENEFELD
Jan is Lead User Experience Designer at Ergosign GmbH at
the Saarbrücken site.
As Solution Manager Industry, he designs human-centered
interaction for industrial plants and machines of today and
tomorrow. His project focus is on innovative operating
concepts for machines, control stations, and cross-company
IIoT platforms.
His passion for modern technologies and their use, added
value, and influences on the operator 4.0 of the so-called
Smart Factory has accompanied him throughout the past 14
years as a designer of human-machine interfaces.
His mission: Making modern technologies usable for people.
DIPLOM INFORMATIKER (FH)
LEAD UX DESIGNER
SOLUTION MANAGER INDUSTRY