Conference Information Conference: 17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland Title of the paper: On-line Training and Monitoring of Robot Tasks through Virtual Reality Authors: Leire Amezua Hormaza, Wael M. Mohammed, Borja Ramis Ferrer, Ronal Bejarano and Jose L. Martinez Lastra

1. On-line Training and Monitoring of Robot
Tasks through Virtual Reality
Contact Information
Tampere University
Engineering and Natural Sciences Faculty
Future Automation Systems and Technologies
Laboratory (FAST-Lab.)
P.O. Box 600,
FIN-33014 Tampere
Finland
Email: fast@tuni.fi
research.tuni.fi/fast
Conference Information
Conference: 17th IEEE International Conference on
Industrial Informatics (INDIN’19) 22-25 July 2019,
Helsinki-Espoo, Finland
Title of the paper: On-line Training and Monitoring of
Robot Tasks through Virtual Reality
Authors: Leire Amezua Hormaza, Wael M. Mohammed,
Borja Ramis Ferrer, Ronal Bejarano and Jose L. Martinez
Lastra
if you would like to recieve a reprint of the original paper, please contact us17th IEEE International Conference on Industrial Informatics
(INDIN’19)
22-25 July 2019, Helsinki-Espoo, Finland 1

2. On-line Training and Monitoring of Robot
Tasks through Virtual Reality
17th IEEE International Conference on Industrial Informatics
(INDIN’19)
22-25 July 2019, Helsinki-Espoo, Finland
Leire Amezua Hormaza, Wael M. Mohammed, Borja Ramis Ferrer, Ronal
Bejarano and Jose L. Martinez Lastra

3. Problem Definition
How to train operators to work with collaborative robots without decreasing
productivity during the period of adaptation?
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 3

4. Problem Definition
Distrust in
cobots
Longer time
of
adaptation
Productivity
How to train operators to work with collaborative robots without decreasing
productivity during the period of adaptation?
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 4

5. Research Background
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 5

6. Research Objectives
• Study the human
trustworthiness while
interacting with cobots
• How the interaction can be
improved?
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 6

7. The Approach
Collaborative Task
Cobot
Human
Product
Process
Training
Instruction
Visual explanations
Simulation
Demonstrations
Collaborative
task
Virtual
Reality
Operator do the task
Actual
system Operator do the task
Feedback
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 7

8. The Approach
1) Selection of Collaborative task
2) Design and animation of cobot model
3) Introduction of the model in the code of A-Frame, Blender
4) Validate the interactions between user and virtual cobot
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 8

9. Implemntation
R. Bejarano, B. Ramis, W. Mohammed, J.
Martinez, “Implementing a Human-Robot
Collaborative Assembly Workstation”, 2019
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 9

10. Implementation
WebVR VR directly in the browser
- Compatible with immersive HMD
- Contain a virtual model of the cobot moving as the real one
- User interacting with the virtual environment
- Capable of simulating collaborative tasks
- Generic
A-Frame is a web framework for building virtual reality
experiences based on top of HTML.
https://aframe.io
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 10

11. Design and animation of cobot model Blender
- YuMi model downloaded from ABB and
inserted in Blender
- Blender is a 3D computer graphics software
toolset that allows creating visual effects and
interactive 3D applications
Implementation
https://www.blender.org/
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 11

12. Scenario (Real VS Virtual)
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 12

13. Test
10 users with no experience with cobots performing the training and filling a
questionnaire
Before the training:
How comfortable do you feel relating to working with YuMi?
After the training:
How comfortable did you feel during the simulation?
How realistic did you find the simulation?
How easy was to follow the process?
Would you be able to reproduce the process?
How comfortable do you feel relating to working with YuMi?
Did you learn how the YuMi interacts during the process?
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 13

14. Test
10 users with no experience with cobots performing the training and filling a
questionnaire
Before the training:
How comfortable do you feel relating to working with YuMi?
After the training:
How comfortable did you feel during the simulation?
How realistic did you find the simulation?
How easy was to follow the process?
Would you be able to reproduce the process?
How comfortable do you feel relating to working with YuMi?
Did you learn how the YuMi interacts during the process?
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 14

15. Results
Based on the answers of the test:
- they found the simulation comfortable
- they could focus on the task
- they learnt the interactions with the cobot
- they felt more confident with the cobot after the training
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 15

16. Conclusions
• 10 participants in the test is very low
• “WOOOOW” factor effect
• Do we need cobots and VR?
• Did we learn something?
Future work
• More participants (+100)
• Use control sample
• Automate the process
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 16

17. THANK YOU 
Any questions?
www.youtube.com/user/fastlaboratory
www.facebook.com/fast.laboratory
www.slideshare.net/fastlaboratory
www.twitter.com/FAST_Lab
17th IEEE International Conference on Industrial Informatics (INDIN’19) 22-25 July 2019, Helsinki-Espoo, Finland 17