This document discusses social robotics for assisted living. It describes motivations for social robotics including an aging world population, changes in the workplace and healthcare, and a demographic gap. Examples of social robotics applications are provided, including smart cabs that can recognize a driver's state of mind, affective robotics that can physically express emotions, and real-time emotion recognition. The document promotes a vision for social robotics including semi-autonomous robots that can interact with people and augment healthcare providers. It provides information on Tampere University of Applied Sciences research and development programs in technology, wellbeing and culture.

1. Social Robotics
(for Assisted Living)
Rod Walsh
Project Manager, Social Robotics
Tampere University of Applied Sciences
The First DMU workshop on Assisted Living Technologies
(ALT2012)
21.November.2012

2. http://www.tamk.fi/en
http://www.tamk.fi/tamktoday
http://tamk-en.blogspot.com/
http://www.facebook.com/Tam
pereUAS
2012

3. TAMK R&D & Innovation (RDI) programmes
Technology
• development of energy efficiency in buildings and facilities,
and reduction of environmental impacts
• efficiency of mobile machinery
• assessment of human activity in genuine operational
environments
• development of new business opportunities
Wellbeing
• development of services for seniors, including promotion of
wellbeing entrepreneurship and development of wellbeing
technology
• promotion of wellbeing at work
• promotion of children’s and adolescents’ health
Culture
• integration of various fields to create internationally
competitive cultural products with a strong regional flavour
• promotion of networking opportunities for companies
• helping the development of culture products for the export
market

4. Social Robotics
A social robot is an
autonomous robot that
interacts and communicates
with humans or other
autonomous physical agents
by following social behaviors
and rules attached to its role.
http://en.wikipedia.org/wiki/Social_robot

5. Motivations: Aging World
• Demographics changes
• Workplace changes
• Health care changes

6. Motivations: Demographic Gap
• Decreased health budget per person
• Insufficient (over-stretched) professionals

7. Motivations: Longer Working Lives
• Assisted self-care
• Augmented healthcare
• Assisted working

8. Social Robotics Vision
• Semi-autonomous robots and avatars
• Interacting with people in their own space
• Augmenting healthcare givers
• Providing rehabilitation and lifestyle support
• Reliable and inexpensive (step-by-step)

9. A Few Examples

10. Plutchik, R. "The Nature of Emotions". American Scientist.
Darwin, Charles (1872), The expression of the emotions in man and animals, London: John Murray.
Physical Emotion
Emotion, mood &
sentiment affect…
• Wellbeing
• Behaviour
• Interaction
Detailed models
exist (e.g. Plutchik)
Ekman, Paul (1992). "An Argument for Basic Emotions". Cognition and

11. Plutchik, R. "The Nature of Emotions". American Scientist.
Darwin, Charles (1872), The expression of the emotions in man and animals, London: John Murray.
Physical Emotion
Emotion, mood &
sentiment affect…
• Wellbeing
• Behaviour
• Interaction
Detailed models
exist (e.g. Plutchik)
Most research uses
Ekman’s (original) six
emotions
Machine emotion is
potentially useful Ekman, Paul (1992). "An Argument for Basic Emotions". Cognition and

12. Real-time Emotion Recognition

13. Smart cabs:
Machines That know their Drivers
Database: non-contact sensing:
state of mind video, image, audio
Pattern log
recognition sensor logs
logged
logged offline
real-time
7/10
capability
state of mind
estimation
~7/10
capability
Match
with
task
8/10 Simulate simple changes
minimum
• Music, lighting,
airflow, …

14. Affective Robotics:
Physical expression from social robots
face in
unity
robot
virtual
User selected behaviors that affect emotional
state for 1to1 dialogue and performance cases.
Things like, “be happy”,
“welcome”, “reject”,
“complain”, “cry”,
“superman”, …
actual robot
virtual robot
body in the
body in unity
real world
body
face
Full
Model-specific compute: Output-specific compute:
Translate behavior into spatial “Render” spatial model
model (behavior)

15. Ekman’s six
anger neutral
disgust
fear
happiness
sadness
surprise
neutral
Physical Emotion

16. anger
Physical Emotion

17. disgust
Physical Emotion

18. fear
Physical Emotion

19. happiness
Physical Emotion

20. sadness
Physical Emotion

21. surprise
Physical Emotion

22. Come and play!
Social http://socialrobotics.tamk.fi/apps/
Robotics
Applications and Demos
Demo App:
ILO apps&demos home
Emotional Face Avatar (alpha)
ILO Interactive Demo (flash) - includes David and Karl's Stories Demo App: Welcome to the Social Robotics Human-like face test demo from TAMK 2012. This
avatar face can simulate emotional, and other, facial expressions using movements
Virtual Vaino Emotional Body along 10 dimensions of facial movement:
(alpha) eyebrows inside up
App More info Link Demo App: m yBut t on
eyebrows outside and up
eyebrows down
eyes opening
Welcome to the Social Robotics Bioloid-like robot body test demo from TAMK 2012. closingeyes
Virtual Vaino Body Positioner This avatar robot can simulate emotional, and other, body gestures and poses with it's (corners) up (and out)
lip
18 degrees of freedom (i.e. just like a real bioloid robot it has 18 actuators that can lip (corners) down
Mac OSX (alpha) rotate this way or that to a precise position). lips pressed (together)
jaw (and mouth) open
Welcome to the Social Robotics Bioloid-like robot body slider test demo from TAMK nose (pinch)
(download) 2012. This version of the avatar robot can be moved around in all 18 degrees of
freedom and, unlike a real robot, it is not restricted by the laws of physics. It won't fall
Windows over, won't mind having it's arm swing in and out of it's head, and much more. This
demo is essentially a play-thing to experiment with different positions and simple
Virtual Vaino movements of the bioloid robot, and especially to work out how to replicate or newly
See emobody page or bundled (32bit) create poses and gestures with human meanings, such as attributable to emotions.
Emotional
readme file (in the download) Takes a (download) to go from ILO Interaction to Video Stories (and "back" to return to ILO Interaction).
few seconds to load!
Body (alpha) Use "skip" to skip forward, "goodbye"
Windows is not currently functional (requires a localhost server we will put online later).
Note: The ILO Interaction part
ILO: (64bit) Karl's Video Stories
David and
(download) Application Display: Emotional Body Gesture Avatar.
David's Story David and ILO Play Minesweeper with ILO There are two modes of operation: standalone and networked. See the bundled readme
Application Display: Emotional Face Avatar.
file for more details.
There are two modes of operation: standalone and networked. See the bundled
Mac OSX Some Instructions readme.txt for more details.
Some Instructions
Hide/unhide controls
(download) Application Display: Bioloid Robot Avatar.
The "list" button in the top right hides and unhides action listHide/unhide controls
on the
Some Instructions left. The Quit button - although self explanatory - doesn't always
Windows work :) The logo button hides and unhides the logo. The "buttons" button in the top right hides and unhides the
Emotional Hide/unhide controls
Body gesture and action controls
other buttons in the top right. Most of the other buttons hide
and unhide the interesting looking things on the left (so you
See emoface page or bundled (32bit) The "buttons" button in the top right hides and unhides can tidy up the UI and just see what you want - and when
Face Avatar the other buttons in the top right. Most of the other The action list UI widget is on the left. Click on an you use a low resolution avatar setting you'll find the things
on the left overlap (yep, the UI needs perfecting) so now you
readme file (in the download) (download) buttons hide and unhide the other interesting looking
UI widgets (so you can tidy up the UI and just see what
action and see it performed. Scroll up and down so
access all the actions. Because the actual physical can hide the stuff that's annoying. The other button in the top
(alpha) you want). The other button in the top right is Quit - bioloid robot understands sets of exactly 256 right is Quit - although self explanatory, it doesn't always
Windows - -
although self explanatory, it doesn't always work :) actions/commands/presets, and because we use
exactly the same action definition file format for
our avatar as for the real robot, the action list
work :)
Remote and debug messages
00:00 00:00 Mouse control
(64bit) 01:20 01:45 shows all 256 slots available. Not all of these are
When this avatar is full screen or its window is in focus, moving your mouse around(you'll see blanks). Not all of the remainder
used The remote UI widget is the "incoming messaged"
Karl's Story Karl and ILO will change the view angle. If you find an angle you want to hold (e.g. while movingare useful (some are leftovers from previous box in the bottom left. It reports some internal program info, and in networking mode
(download) experiments and factory settings). Not all of the
the sliders), press the ctrl key while moving the mouse to freeze the view angle. You
can also zoom in and out with the mouse scroll button/gesture. remainder of that work well. The complete long list
says what has been sent (useful in determining that the magical controller person can't
spell and so isn't making the avatar move).
of commands is given in the demo app UI and also
Slider controls in the bundled readme file. Emotion
selectors
However, a small set of 9 basic expressions do
Mac OSX The sliders on the
left represent each
work well and simply (see the table below).
Beyond these 9 expressions, play with the long list
The local UI
widget is the 3
of the 18 actuators
(download) on the robot. Move
them from left (0.0
so see what commands work, and how they work. by 3 grid of
buttons in the
lower-middle left. The button labels are a one-to-one match to the instructions that the
- one extreme) to Basic 9 Equivalent long list
Virtual Vaino Windows center (0.5 -
middle position) to
Expression command
avatar can understand from our remote magical controller in networked mode. In
standalone mode you can spoil the surprise and make the avatar do what only magical
remote controllers can do otherwise.
Happy
right (1.0 - the other extreme) to pose the avatar as H_3_2_1
Body See mankin page or bundled (32bit) you wish - one actuator at a time. For convenience,
Sad
the sliders are clustered into left arm, right arm, left S_2_1 Facial expression controls
leg and right leg groups. To simplify the UI,
Positioner readme file (in the download) (download)
00:00
-
00:00
-
individual sliders are not labeled - but they areFear
order: from top to bottom relates to from closest to
in F_2_2 The sliders UI widget is the long one from the top
left. It has 10 sliders - one for each of the 10
01:00 01:48 the torso to furthest from the torso respectively. (The astute among you will have Surprised SU_1 dimensions of facial movement (or degrees of
(alpha) Windows
ILO's Emotional Faces
that there are only 16 sliders. Hip movement is generated by two actuators, and the
seen
closest of these to the torso, for each leg, is given a slider at the end (not the start,Disgusted
as D_2_1_1
freedom of our model - if you prefer that
terminology). Move them from a neutral left (0.0) to
full on activation right (1.0) to see each in action.
(64bit) would be logical) of the list just to keep you on your t oes.
Downloads
Angry A_1_1 When more than one of these is not neutral (i.e. none-
zero) the avatar magically combines the "positions"
Waving Waving
(download) Bow Bow
from each of the dimensions to produce an combined
expressions. The emotion presets are simply
combinations of these 10 dimension/slider values. If
Mac OSX (download) you want to see (a strong of) the values press "show
Windows (32bit) (download) parameters" and 10 numbers will appear in the
In networked mode, the remote magical controller has exactly the same menu of action
Windows (64bit) (download) commands at their disposal as you see in the action list and in the bundled readme file. box below. You can copy and paste
parameter
from/into this parameter box, and manually change
Downloads numbers. When you are ready to set the face and
ILO sliders to your (string of) numbers, press "make face"
Includes child (David) and and yet more magic occurs.
Interactive Mac OSX (download) Downloads
senior's (Karl) stories (child n/a Windows (32bit) (download)
Windows (64bit) (download)
Social Robot
and senior). See ILO's page 00:00
- Mac OSX (download)
Windows (32bit) (download)
Demo 00:16 Windows (64bit) (download)

23. Mindreader
(Emotion Hero)
Train One,
(work-in-progress) Train All

24. Our 2013 Focus
Teleportation Physio’
(webconf) buddy Emotion play
Rich info, video & Kinetic robot
ambiance in a simple UI “programming” Emotion & activity
Instant media spaces Flexible humanoid detection & recognition
(smart active cam/mic)
Physical/Mobile/PC Playful engagement
Affordable locomotion Avatar “mobility”
(raised tablet on wheels)
Probing
Remote nursing Behavioural agent
Stimulation
Companionship Augmented care
Intervention

25. Our 2013 Focus
Teleportation Physio’
(webconf) buddy Emotion play
Rich info, video & Kinetic robot
ambiance in a simple UI “programming” Emotion & activity
Instant media spaces Flexible humanoid detection & recognition
(smart active cam/mic)
Physical/Mobile/PC Playful engagement
Affordable locomotion Avatar “mobility”
(raised tablet on wheels)
Probing
Remote nursing Behavioural agent
Stimulation
Companionship Augmented care
Intervention

26. Our 2013 Focus
Teleportation Physio’
(webconf) buddy Emotion play
Rich info, video & Kinetic robot
ambiance in a simple UI “programming” Emotion & activity
Instant media spaces Flexible humanoid detection & recognition
(smart active cam/mic)
Physical/Mobile/PC Playful engagement
Affordable locomotion Avatar “mobility”
(raised tablet on wheels)
Probing
Remote nursing Behavioural agent
Stimulation
Companionship Augmented care
Intervention

27. Our 2013 Focus
Teleportation Physio’
(webconf) buddy Emotion play
Rich info, video & Kinetic robot
ambiance in a simple UI “programming” Emotion & activity
Instant media spaces Flexible humanoid detection & recognition
(smart active cam/mic)
Physical/Mobile/PC Playful engagement
Affordable locomotion Avatar “mobility”
(raised tablet on wheels)
Probing
Remote nursing Behavioural agent
Stimulation
Companionship Augmented care
Intervention

28. New Beginnings in 2013
Teleportation Physio’
(webconf) buddy Emotion play
Rich info, video & Kinetic robot
ambiance in a simple UI
Instant media spaces
Other ideas?
“programming” Emotion & activity
Flexible humanoid detection & recognition
(smart active cam/mic)
Physical/Mobile/PC Playful engagement
Affordable locomotion Avatar “mobility”
(raised tablet on wheels)
Probing
Remote nursing Behavioural agent
Stimulation
Companionship Augmented care
Intervention

29. Thank you!
Questions, discussion…