2. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Why did I chose Social Robotics?
You’re Number 2
3. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Understanding Humans
Sometimes it’s difficult to
understand and live with others
You’re Number 3
4. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Understanding Robots
So what about living with robots?
You’re Number 4
5. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics You’re Number 5
6. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
What is a social robot?
Robots that people apply a social model to in
order to interact with and to understand
A question:
“Does the robot only appear to be socially
intelligent or is it genuinely so?”
You’re Number 6
7. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Social Models
Constrained Full
You’re Number 7
8. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Robotic Social Development
Leverage from playful, infant like interaction to foster its social Development
Kismet
You’re Number 8
9. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
What it should look like?
Dogs are socially Responsive and socially
intelligent in a genuine sense
Robot doesn’t have to look like a human
To be genuinely social intelligent
You’re Number 9
10. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Social Robot Success
What matters is how the
robot interacts face
to face
with people, and how
people
interact with it in
a human
environment
You’re Number 10
11. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Known Types
Encourage people to anthropomorphize the technology in order
To interact with it (but goes no further)
Uses human-like social cues and communication modalities
To facilitate interactions with humans, the social model
Tends to be shallow and its social behavior is pre-canned.
Benefit from interaction with people (training model). These
Interactions their internal structure in a deeper level. People
Shape the robot’s behavior. They are socially passive.
Socially participative with their own internal goals and
Motivations. Engage people in a social manner to benefit
Both the person and the robot. The robot’s social behavior
Stems from its computational social psychology.
You’re Number 11
12. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Meet Kismet
You’re Number 12
13. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Kismet Technicalities
You’re Number 13
14. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Speaking Turns
The ability to exchange turns during face-to-face
interactions is the cornerstone of human style
communication and instruction
You’re Number 14
15. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Speaking turns Limitations
Respond After
0.1 sec
Respond After
0.25 sec
Respond After
0.50 sec
Respond After
0.25 sec
You’re Number 15
16. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Envelope Displays
Regulating
Speaking
turns
Regulating
Speaking
turns
Facial DisplaysFacial Displays
gesturesgestures
Shifts in gazeShifts in gaze
Eye blinksEye blinks
Eye contactEye contact
Raise of eyebrowsRaise of eyebrows
Paralinguistic
Social Cues
Paralinguistic
Social Cues
You’re Number 16
17. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Experiments
You’re Number 17
18. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Kismet’s Envelope Displays
You’re Number 18
No. Action Cue
1 Acquire the Floor Break eye contact
Lean back
2 Start a speaking turn Vocalize a Kismet-esque
babble
3 Stop a speaking turn Stop vocalization and re-
establish eye contact
4 Hold the floor Look to the side
5 Relinquish the floor Raise brows
Lean forward a bit
Blinking occurs at most at the end of vocalization.
19. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Social Robot Evaluation
• Do People intuitively read and naturally respond to
Kismet’s social cues?
• Can Kismet perceive and appropriately respond to
theses naturally offered cues?
• Does the human adapt to the robot and the robot
adapt to the human, in a way that benefits the
interaction
• Specifically, is the resulting interaction natural,
intuitive and enjoyable for the human
• Can Kismet perform well despite its perceptual,
mechanical, behavioral and computational limitations?
You’re Number 19
20. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Kismet’s Turn-Taking Performance
You’re Number 20
21. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Kismet’s Turn-Taking Performance
You’re Number 21
22. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Kismet’s Turn-Taking Performance
You’re Number 22
23. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics
Future?
You’re Number 24
24. Illai J. Gescheit – Mechanical Engineering Dept. – Tel Aviv University
Social Robotics – Semester Project – Introduction to Robotics You’re Number 25
Editor's Notes
It is important to recognize that humans are a profoundly social species. Our social-emotional intelligence is a useful and powerful means for understanding the behavior of’ and for interacting with some of the most complex entities in our world – people and other living creatures . Faced with non-living things of sufficient complexity (i.e. when the observable behavior is not easily understood in terms of its underlying mechanisms)’ we oftern apply a social model to explain ‘ understand and predict their behabior as well. For instance’ we are all familiar that people anthoropomotrphize all sorts of technologies( e.g. cars’ computers ect.) . The studies of Premack and Premack show that people attribute mental state (i.e. intents’ beliefs ‘ feelings desires ect( to describe the bejavior of his ingenious vehicles. Right or wrong’ people rely on social modeels (* or fluidly seitch between using a social model with other mental models( to make complex behavior more familiar and understandable and more intuitive with which to interact. We do this because it is enjoyable forr us ‘ and it is often surprisingly wuite useful.
We take the position that dogs are another socially responsive species- socially intelligent in a genuine sense, althought their social sophistication is less than that of a human. Hence this criterion for success should not be confused with trying to build a robot that is indistinguishable from a human inside and out. – the appearance of the robot and its internal design details can be quite different from the human couterpart, what mattes is how it interacts face to face with people and how people interact with it in a human environment.
3 DOF – directs the robot’s gaze
3DOF – Controls the head orientation
15 DOF more – move the facial features (e.g eyelids, eyebrows, lips and ears)
4 CCD Cameras
A lavalier mic. Is worn on the person interacting with kismet to process the vocalization
Given that a robotic implementation is limited by perceptual, motor and computational resources, we have found that theses social cues are usful in regulating the turn –taking of humans and robots.
These limitations force the robot to exchange speaking turns at a slower rate than a regular human. The times specified are the interval time of response after pause after the speech has ended.
Humans seem to intuitively read Kismet’s cues and use them to regulate the rate of exchange at a pace where both partners perform well.
The robot uses a Kismet-esque babble for its speaking turn. Hence, the envelope displays are used to regulate the dynamics of interaction during the exchange of speaking turns rather than focus on the content of what is said.
The same holds true for proto dialogs between human care-givers and their prelinguistic infants.
Well studied by discourse theorists, human employ avariety of para-linguistic social cues, called envelope displays’ to regulate the exchange of speaking turns. They involve facial displays, gestures, shifts in gaze and eye blinks – such as raising one’s brows and establishing eye contact to relinquish one’s speaking turn’ or looking aside to hold one’s speaking turn even when speech is paused.
Given that a robotic implementation is limited by perceptual ‘ motor and computational resources’ we have found that these social cues are useful in regulating the turn-taking of humans and robots. This has proven particularly important for Kismet because its speech processing limitations force the robot to exchange speaking turn s at a slower rate thatn is typical for human adults.
Invitation of 4 naïve subjects ranged in age from 12 to 28 years old.
2 male and 2 female
They were asked to carry on a “play” conversation with the robot.
All was video recorded for later analysis. They were told that the robot does not speak nor understands English, but babbles in a characteristic manner.
Here I should explain what does it mean to hold the floor ect…
This table shows that the robot is engaged in a smooth proto-dialog with the human partner the majority of the time (approx. 82.5% of the time) .
At first they use only the vocals of the robot to gauge their speaking turn
They expect the robot to start talking immediately after they finish talking.
And the robot is most likely to interrupt them (common at the first phrases the people perform)
They call it “Hiccups” in the flow – they appear in short clusters of mutual interruptions and pauses (over 2-4 utterances of the speaker) before the turn phases become coordinated and the flow of the exchange of speaking turns smoothes out.
As the subject seem to adjust their behavior according to Kismet’s envelope displays these “hiccups” within speaking turns become less frequent.
As we can see in this table, for each subject there are progressively onlger runs of cleanly exchanged turns as time progresses. The flow of communication becomes smoother (e.g. fewer interuuptions, pauses, and significant flow disturbances), as people read and entrain to Kismet’s envelope displays.
Then the rate of vocal exchange is well matched to the robot’s perceptual limitations.
This table shows that the robot is engaged in a smooth proto-dialog with the human partner the majority of the time (approx. 82.5% of the time) .