Presentation by Andreas Stainer-Hochgatterer (ATT) during the workshop Affective interaction with avatars by Carsten Stocklöw - AAL Forum 2015

1. Affective Interaction with Avatars
23 Sep 2015, AAL Forum 2015, Ghent, Belgium
Andreas Stainer-Hochgatterer (AIT)

2. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)
 Design, develop and evaluate an innovative user-centric
technological solution, the Virtual Support Partner (VSP),
attending to the elder (65+) daily activity and safety needs,
while the elder goes about his normal daily life.
Key Data:
Start Date: 1st Dec 2013
Duration: 36 months
Budget: EUR 4.195.276
Funding: EUR 3.320.457

3. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)

4. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)
 Virtual Characters with properties as humans in face-to-face
conversations (CASSELL, 2000)
◦ Ability to recognize and respond to verbal and non-verbal input
◦ Ability to generate verbal and non-verbal output such as mouth
movements, eye movements, head movements, hand gestures,
facial expressions, and body posture
◦ Ability to deal with conversational functions such as turn taking,
feedback, and repair mechanisms
◦ The ability to give signals that indicate the state of conversation,
as well as to contribute new propositions
 Up to now avatars monotonous and soon “boring”
 ECAs can improve interaction in AAL environments

5. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)
 Older adults are capable of recognizing emotions in facial
expressions of an agent and follow instructions better
 ECAs allow development of affinitive relationships /
companionship
 Might contribute to reduce the feeling of loneliness
 Motivational aspect 
◦ Increased interactions
◦ Increase of activities / services usage
 Move beyond “computer as a tool”
◦ Easy to use, engaging, trustful for older adults

6. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)
 Stimulate and motivate the elder to remain longer active at
home through a virtual partner support
 Enhance the engagement of the elder in carrying out daily
activities at home through emotional understanding
 Increase the elder’s satisfaction in using the system via a
natural and intuitive way to interact with the system
 Improve quality of life and prolong autonomy of the elder
 Provide benefits on the social level of the elder and also
improve the integrated care processes for elderly care at
home
 Achieve high usefulness of the system for the user through
pilots and related evaluation and assessment

7. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)

8. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)
 Embodied Conversational Agents
 Dialogue Management
 Emotional Speech Synthesis
 Multimodal Affect and Context Analysis
 Behaviour and Gesture recognition
 Safety ICT-based Services in AAL
 ICT based service solutions for the elderly at home in other
project initiatives

9. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)
 Miraculous-Life VSP  emotionally-enriched HCI
 Miraculous-Life VSP fuses together
◦ Facial expressions (face reader)
◦ Voice intonation (emotion from speech)
◦ Gestures
◦ + contextual information of the users environment
 3 modes  listening, waiting, talking
 4 emotional states recognised
◦ Anger/disgust, fearful/stressful, sadness, happiness/positive
 7 emotional states/expressions of avatar
◦ Happy, sad, worried, relieved, compassionate (expresses sympathy), directive
behaviour, and neutral
 Mapping is key and based on scientific & domain knowledge
 Computational model of VSP emotions based on Roseman’s model

10. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)
 Categories: Care & Wellness, Guidance,
Education/Leisure, Safety

11. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)

12. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)
 Avatar as interface appreciated by elderly
◦ Older adults voted for human character (and animals)
◦ Caregivers voted for female human character
 Service offers are key (avatar is mediator to the services)
 Speech/audio recognition still tricky in home environments
 Limitations in emotion recognition
◦ Gestures (from older adults) give less information on emotional state
◦ Activity recognition is more beneficial
◦ Training data needed (for target group)
 Dialogue management is key component (high efforts in implementation)
 Balanced interaction between speech & touch interaction needed
 Recognition (objects, etc.) capabilities based on Kinect limited
 Privacy  non-intrusive solutions/systems needed

13. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)
 Final pre-trials in preparation
 Validation of the system will be realized in two well selected
use cases in two different countries
◦ Zuyderland (formerly ORBIS), NL
◦ MRPS, CH
 Up to 50 elderly and their caregivers will use the system over
a six month period
 System will be delivered as a stand-alone consumer product,
operating on a scalable distributed network of interconnected
PCs, tablets and Kinect devices
 Success indicators will be evaluated (pre/post measurements)

14. Miraculous-Life
Miraculous Life for elderly Independent Living (STREP No. 611421)
The project Miraculous-Life is co-funded by the 7th Framework
Programme for Research and Technological Development of the European
Commission, within the Call FP7-ICT-2013-10.
Further information: www.miraculous-life.eu
Or visit us on Booth 14 (AIT)…