1) The document discusses human-robot collaboration research at Cardiff University including situational awareness for task planning and autonomous vehicles, creativity in design, and social robotics. 2) It describes the Shadow Robot project which aims to enable remote help for elderly people through semi-autonomous robots in homes using three different user interfaces. 3) The research is developing semantic mapping and reasoning techniques to improve task planning and navigation by inferring contextual information about objects and locations. Experiments are evaluating gaze prediction algorithms to enable teleoperation of robots.

1. Human- Robot
Collaboration

2. Human-Robot Systems at Cardiff
Situational Awareness
(Task Planning)
Situational Awareness
(Autonomous Vehicles)
Creativity in Design &
Design for Intuitive Use
Artificial CuriositySocial Robotics

3. Shadow Robot
 Concept: Enabling adult’s children or care-givers to help the elderly
remotely and physically through semi-autonomous tele-operation
 Goal: the development and prototyping of remotely-controlled, semi-
autonomous robotic solutions in domestic environments in support
of elderly people
 Challenge: unstructured environment

4. User Interface for Local User (Elderly at Home)

5. User Interface for Informal Caregiver

6. User Interface for Professional Tele-assistants

7. Semantic Map for Navigation
• S-Box (spatial
knowledgebase)
• T-Box (topological
knowledgebase)
How semantic knowledge
improves task planning?
•Extending the capabilities
of the planner by reasoning
with semantic information
• Knowledge inference,
when knowledge is
incomplete or implicit
• To infer the existence of
objects
• To infer possible objects

8. Semantic Knowledge Base
Context-based inference
An example: ‘get a bottle of milk’
milk fridge kitchen
Which room ?Where from?

9. Semantic Reasoning
Human-inspired inference (inferring the existence of instances)
Non-observable objects can be inferred from semantic DB
What ????

10. Evaluations
Stuttgart
Milan

11. Gaze Control
Teleoperation:
IFAC HMS 2016 11

12. Conceptual model
Teleoperation:
IFAC HMS 2016 12
Three cascading algorithms used:
1. A double exponential filter for eye-
tracking data processing
2. A cartographic algorithm to reduce
the number of points while
conserving the topology of the
trajectory
3. A fixation-detection algorithm to
detect fixations and remove the
associated noise

13. Conceptual model
IFAC HMS 2016 13
Principle :
- Several kinds of gaze
trajectories recorded
- The robot assesses which
trajectory to draw from a small
portion of trajectory drawn by
the user
- The entire trajectory is built
using IDCT (inverse discrete
cosine transform)
Teleoperation:

14. Gaze Trajectory Prediction using eye
tracking
Probabilistic approach:
Principle : Record statistics about two discrete state representations and use it for prediction
Training : Use maximum-likelihood probabilities on two discrete state representations (locations
and directions) for every kind of gaze trajectory
Prediction : At each iteration, update a likelihood score for each class of trajectory based on
recorded probabilities, => the maximum likelihood score gives the predicted trajectory
15 position states and 8 direction states used.
IFAC HMS 2016 14
a kinematic state (direction)
a location state

15. Gaze Trajectory Prediction using eye
tracking
Symbolic approach:
Principle : Assign one word per class (similar to word-prediction problems) and learn to
predict words.
Training : Build one word per class of trajectory with one letter per equal-length portion
of trajectory
Prediction : At each iteration, update a likelihood score for each class of trajectory
based on a inter-class distance matrix and recorded words => the maximum likelihood
score gives the predicted trajectory
28 characters used
IFAC HMS 2016 15
3
3
5
2
7
3 7
6
0
A
B
D
C
E
G
F I
H
Word for the corresponding trajectory :
CCCBBBBBAAADDEEEEEEEFFFIIIIIIIHHHHHH

16. Experiments
16IFAC HMS 2016

17. Experiments
IFAC HMS 2016 17
Goal :
Evaluate and compare the
performance of the two algorithms
in terms of prediction accuracy and
storage load
Principle :
-12 classes of trajectory are
recorded
-For each class and each
approach, the prediction is checked
three times
=> For example, for class A, we
draw A three times and see if
predictions are correct and fast

18. Experiments
IFAC HMS 2016 18
• Experimental values (measuring
how quickly the right prediction is
made):
• The percentage of the total length of
the desired trajectory from which the
desired class is among the three
best-ranked classes (measure M1)
• The percentage of the total length of
the desired trajectory from which the
desired class is the best-ranked
class (measure M2)
• They measure how fast and
accurate the prediction is.