This document summarizes a conference presentation about developing a mobile application to remotely control the humanoid robot NAO to help people with intellectual disabilities. The goals were to create an app for mobile devices that allows children with special educational needs and their educators to use robotics. The proposed system was tested with educators and children at a school, and it facilitated the integration of robotics in the classroom. User feedback was positive and future work involves expanding activities and translating the app to more languages and robots to continue exploring how robotics can benefit education for those with learning disabilities.

1. Interactive Technologies And Games Conference. 16-17th October 2014
Remote operation of robots via mobile devices to help people with intellectual disabilities
María José Gálvez Trigo
School of Computer Engineering University of Castilla-la Mancha
David J. Brown
Computing and Technology Team
Nottingham Trent University

2. Contents
❖INTRODUCTION
❖Intellectual disabilities and assistive technologies
❖Humanoid robot NAO
❖Problem and justification
❖GOALS
❖THE PROPOSED SYSTEM
❖EVOLUTION OF THE DEVELOPMENT
❖RESULTS AND FUTURE WORK
❖CONCLUSIONS
2

3. INTRODUCTION
3

4. Intellectual disabilities and assistive technologies
❖Intellectual disability Learning disability:
❖Impaired social functioning
❖Reduced ability to understand new information
❖Assistive technology: Product or service used to increase, maintain, or improve the functional capabilities of a person with a disability
4
❖Before adulthood
❖Lasting effect on development º

5. 5
❖Mobile devices:
❖Can be carried everywhere
❖Can be used in multiple contexts
❖Mainstream technology - Social acceptance
❖Robotics:
❖Increases engagement (Standen et. al 2014, Roscoe 2014)
❖Predictable and reliable - Minimally threatening
❖Holds interest more than some traditional methods

6. Humanoid robot NAO
❖Manufactured by Aldebaran Robotics
❖57 cm tall
❖25 Degrees of Freedom (DOF)
❖Multiple tactile sensors
❖Several microphones and speakers
❖Two cameras
❖Fully programmable
❖Toy-like appearance
6

7. Problem and justification
❖Robotics and mobile devices need to be adapted for their use within special education
❖Use of robotics - Requires specific skills and knowledge that educators do not possess
❖Two possible solutions:
1.Specific training for educators
2.Development of tools that allow educators to use robotics without specific training
7

8. Problem and justification
❖Robotics and mobile devices need to be adapted for their use within special education
❖Use of robotics - Requires specific skills and knowledge that educators do not possess
❖Two possible solutions:
1.Specific training for educators
2.Development of tools that allow educators to use robotics without specific training
8

9. GOALS
9

10. Two main goals
❖Develop an application for mobile devices that enables the use of robotics by children with Special Educational Needs (SEN) and their educators
❖Demonstrate how the application works by means of the use of the humanoid robot NAO
❖Development of specific modules and behaviours for the robot
10

11. THE PROPOSED SYSTEM
11

12. The system
12

13. Main functionalities - On the robot
❖Tethering mode
❖Wi-Fi - Touching front sensor on NAO’s head
❖Bluetooth - Touching rear sensor on NAO’s head
❖Stop the robot - Pressing bumper on NAO’s right foot
13

14. 14
Tethering mode and stop the robot

15. ❖Remote control
❖Walk
15
Main functionalities - On the application
❖Talk
❖Change Posture

16. 16
❖Dance and music
❖The robot performs a dance with music
❖The robot teaches movements of a dance

17. 17
❖Simon says - Well-known game in which the robot tells the children to do something
❖Cause and effect - Two or three buttons to check the understanding of the ‘cause and effect’ concept

18. 18
❖Questions and answers
❖The educator sets a question and three possible answers
❖Can be used with speech recognition

19. 19
❖Give orders to NAO
❖React to voice commands
❖React to a sound
❖React to buttons

20. ❖Settings
❖IP address of the robot
❖Volume of the robot
❖Send to rest
❖Children mode
❖Security distance
❖Angle to turn and distance to walk
20

21. ❖Settings
❖Choose reward
❖Speech recognition
❖Sensibility of the sound recognition system
❖Confidence of the speech recognition system
21

22. 22
❖Help - Shows help depending on the screen in which the user is at that moment

23. 23
Examples of screens using ‘children mode’

24. EVOLUTION OF THE DEVELOPMENT
24

25. 25
❖Co-design and testing by the developer (M. J. G. Trigo) and the target group of users (children with SEN and their educators)
❖Meetings with educators from the Oak Field School and Sports College in Nottingham to define the requirements of the system
❖Prototypes tested and improved taking into account the feedback from the users in a process of iterative prototyping
❖Final version tested at the Oak Field School and Sports College in Nottingham

26. RESULTS AND FUTURE WORK
26

27. Results
❖Think-Aloud protocol
❖A questionnaire was given to 7 educators:
❖They had to give a value from 0 to 5 to aspects such as the design, the implemented behaviours, the ease of use, the tolerance to errors, etc.
❖Mean over 4 in all questions - High degree of satisfaction
❖Low standard deviation - High degree of agreement
27

28. 28
❖The main goals have been accomplished
❖Overall the system:
❖Facilitates the integration of robotics in the classroom of special education schools, being useful for children with SEN
❖Is consistent, clear, and easy to use and learn
❖Offers an adequate and of good quality range of behaviours and activities that the robot can perform
❖Is robust and tolerant to errors, and giving appropriate and relevant feedback when they occur

29. Future work
❖RoboMobID will be used within the context of the EDUROB project.
❖EDUROB aims to explore the use of robotics in education for people with learning disabilities and offer a solution for its integration in the education of children with SEN.
29
The EDUROB project (543577-LLP-1-2013-1-UK-KA3-KA3MP) has been funded with support from the European Commission.

30. Future improvements
❖More dances and animations
❖Activities using ‘Makaton‘ symbols for illiterate students
❖User profiles for each child
❖Translation to several languages
❖Robot mimicking daily activities
❖Support for creation of new animations within the application
❖Use along with physical input devices like micro-switches
❖Support for more platforms (OS and robots)
30

31. CONCLUSIONS
31

32. 32
❖The use of mainstream technologies in the classrooms of schools for students with SEN is of great help
❖This is an emerging field with some promising early studies to show that engagement in students is higher using the NAO vs. some traditional methods (Standen et. al 2014, Roscoe 2014)
❖It is very important to help teachers to integrate new technologies in the classroom

33. Thank you for your attention Do you have any questions?
33