CPDP 2020
Emotional AI and Empathic
Technologies: Rights, Children,
Domestication
PANELLISTS
Ben Bland (@ben_bland), Sensum, UK/Northern Ireland
Frederike Kaltheuner (@F_Kaltheuner), Mozilla Fellow, UK/Germany
Giovanna Mascheroni (@giovannamas), Università Cattolica del Sacro Cuore, Italy
Gilad Rosner (@GiladRosner), IOT Privacy Forum, Eleos, Spain
MODERATOR
Andrew McStay (@digi_ad), UK, Bangor University, Eleos, UK
Empathic Technology
Thermal Imaging
Ultra-wideband
Data Types for Understanding Human States
Human-State Modelling
Empathic Digital Ecosystem
IoToys as
01 Toys
02 Social robots
03 Media
IoToys as social robots
 Robots as media with which humans interact
(Zhao, 2006; Guzman, 2018)
 Robots and humans engage in the co-
construction of meaning (=communication)
(Guzman, 2018)
 The interactions between humans and robots as
‘entity- making practices’ in which boundaries
between subject/object and sameness/difference
are relationally negotiated (Suchman, 2011)
1 Liveliness
1 Liveliness
Samuele (father, Family 1): she enjoys it a
lot when Cozmo gets angry, otherwise
she makes it do the basic functions, I would
say, emotions, it seems to me, now from
the little I hear, otherwise emotions and
songs, in short, Cozmo singing.
Luca (8yo, Family 3): with Cozmo you can
interact with, with the PlayStation and Wii even
but… in differ…Differently…because with the
PlayStation and the Wii you interact with a remote
control and you make an avatar, a thing… While
with Cozmo you can drive it, but Cozmo also does
things on its own…Uhm… Like, you look at
Cozmo, it looks back at you and it says your name
Cozmo as a lively entity
Cozmo as an hybrid entity – children
testing Cozmo’s liveliness
Martina (8 yo, Family 1): If you can believe, I even tried
to close his eyes but he knew it.
Interviewer: What do you mean to close his eyes?
Martina (8 yo, Family 1): So, to put your hands on it, but
he knew it so he won even if I closed its eyes.
Interviewer: Do you mean is he able to understand if the
colours are correct even if you cover his eyes?
Martina (8 yo, Family 1): Yes, since it is practically
connected to the phone and to the cubes so it
Cozmo as an hybrid entity
Interviewer: And why doesn’t Cozmo understand you
in your opinion?
Sofia (8 yo, Family 2): because it is not Italian
Margaux (5 yo, Family 2): it seems French
Interviewer: and how does Cozmo speak?
Sofia (8 yo, Family 2): in its own language
Interviewer: that is not even English?
Sofia (8 yo, Family 2): no, it is Cozmese
2 Affective stickiness
Luca (8 yo, Family 3): I feel affection for Cozmo.
Because it plays with you…Uhm…it makes you
happy.
Olivia (Mother, Family 2): One thing that made me
laugh is that for our youngest, it is a puppy. So, she
would pet it, gave it kisses, I don’t know if in some
photos you can she is tickling it. What really
impressed me was that even though it is small and
“Federal agencies must avoid
regulatory or non-regulatory actions
that needlessly hamper AI innovation
and growth… Agencies must avoid a
precautionary approach that holds AI
systems to such an impossibly high
standard that society cannot enjoy
their benefits.”
Permissionless Innovation
• “the notion that experimentation with new technologies and business models should
generally be permitted by default.
• the general freedom to experiment and learn through ongoing trial-and-error
experimentation.
- Thierer, A. (2016). Permissionless Innovation: The Continuing Case for Comprehensive Technological Freedom
Permissionless Innovation
• Self-regulation
• Cost-Benefit Analysis
• The need to show evidence of a harm before regulating
• “Regulatory humility”
- Thierer, A. (2016). Permissionless Innovation: The Continuing Case for Comprehensive Technological Freedom
“It is … vital that government officials … approach new technologies with a dose of regulatory humility, by working
hard to educate ourselves and others about the innovation, understand its effects on consumers and the marketplace,
identify benefits and likely harms, and, if harms do arise, consider whether existing laws and regulations are sufficient
to address them, before assuming that new rules are required.”
– FTC Commissioner Maureen Ohlhausen
“the belief that new innovations should be curtailed or disallowed until their
developers can prove that they will not cause any harm to individuals, groups,
specific entities, cultural norms, or various existing laws, norms, or traditions.
“many critics adopt a mindset that views the future as something that is to be
feared, avoided, or at least carefully planned. This is known as the ‘stasis mentality'
and it is, at its root, what motivates precautionary principle-based thinking and
policymaking.’”
- Ibid.
(One way of) Defining the Precautionary Principle
“PP1. Scientific uncertainty should not automatically preclude regulation of activities that pose a
potential risk of significant harm (Non-Preclusion PP).
PP2. Regulatory controls should incorporate a margin of safety; activities should be limited below
the level at which no adverse effect has been observed or predicted (Margin of Safety PP).
PP3. Activities that present an uncertain potential for significant harm should be subject to best
technology available requirements to minimize the risk of harm unless the proponent of the activity
shows that they present no appreciable risk of harm (BAT pp).
PP4. Activities that present an uncertain potential for significant harm should be prohibited unless
the proponent of the activity shows that it presents no appreciable risk of harm (Prohibitory PP).”
- Stewart, R. (2002). “Environmental regulatory decision making under uncertainty,”
(More fully) Defining the Precautionary Principle
“Where there are threats of serious or irreversible damage, lack of full
scientific certainty shall not be used as a reason for postponing cost-
effective measures to prevent environmental degradation.”
- United Nations Rio Declaration on Environment and Development, 1992
Precautionary Principle and Environmental Regulation
• addresses issues of information and power asymmetry
• spurs public debate
• recognizes that monetary damages for future harms may be inadequate
• takes account of the uncertainty of scientific evaluation
- Adapted from Luiz Costa, L. (2012) ”Privacy and the Precautionary Principle”
Precautionary Principle and Privacy & Data Protection
Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional Expressions Reconsidered: Challenges to Inferring Emotion From
Human Facial Movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930
Some concerns about affect recognition tech
(1) Context beyond the face and body matters
(2) Detecting facial movements ≠ detecting emotional states
(3) Variance in how people communicate emotional states
in practice: a spectrum - flawed, incomplete, rough approximation
Some concerns about affect recognition tech
(1) Context beyond the face and body matters
(2) Detecting facial movements ≠ detecting emotional states
(3) Variance in how people communicate emotional states
in practice: a spectrum - flawed, incomplete, rough approximation
Lessons for trustworthy AI
Harms
Harms that
result from
technology
working as
intended
Harms that result
from technology
NOT working as
intended
privacy
flawed
Software
doesn’t work
Tools are biased
Harvesting of
emotions
manipulation
false positives
/ false
negatives
We’re surrounded by dubious claims about technology’s capabilities…..
From public procurement to
How to recognise AI snake oil
by Arvind Narayanan
https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf
https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf
Inherently problematic:
- Assessing identity (ethnicity, gender,
sexual orientation)
- Judging intention (deceit, lieing)
- Inner life
Assessing & judging
people
Predicting the
future
How to recognise AI snake oil
adapted
CPDP 2020
Emotional AI and Empathic
Technologies: Rights, Children,
Domestication
PANELLISTS
Ben Bland (@ben_bland), Sensum, UK/Northern Ireland
Frederike Kaltheuner (@F_Kaltheuner), Mozilla Fellow, UK/Germany
Giovanna Mascheroni (@giovannamas), Università Cattolica del Sacro Cuore, Italy
Gilad Rosner (@GiladRosner), IOT Privacy Forum, Eleos, Spain
MODERATOR
Andrew McStay (@digi_ad), UK, Bangor University, Eleos, UK

Cpdp slides

  • 1.
    CPDP 2020 Emotional AIand Empathic Technologies: Rights, Children, Domestication PANELLISTS Ben Bland (@ben_bland), Sensum, UK/Northern Ireland Frederike Kaltheuner (@F_Kaltheuner), Mozilla Fellow, UK/Germany Giovanna Mascheroni (@giovannamas), Università Cattolica del Sacro Cuore, Italy Gilad Rosner (@GiladRosner), IOT Privacy Forum, Eleos, Spain MODERATOR Andrew McStay (@digi_ad), UK, Bangor University, Eleos, UK
  • 2.
  • 3.
    Thermal Imaging Ultra-wideband Data Typesfor Understanding Human States
  • 4.
  • 5.
  • 7.
    IoToys as 01 Toys 02Social robots 03 Media
  • 8.
    IoToys as socialrobots  Robots as media with which humans interact (Zhao, 2006; Guzman, 2018)  Robots and humans engage in the co- construction of meaning (=communication) (Guzman, 2018)  The interactions between humans and robots as ‘entity- making practices’ in which boundaries between subject/object and sameness/difference are relationally negotiated (Suchman, 2011)
  • 9.
  • 10.
    1 Liveliness Samuele (father,Family 1): she enjoys it a lot when Cozmo gets angry, otherwise she makes it do the basic functions, I would say, emotions, it seems to me, now from the little I hear, otherwise emotions and songs, in short, Cozmo singing.
  • 11.
    Luca (8yo, Family3): with Cozmo you can interact with, with the PlayStation and Wii even but… in differ…Differently…because with the PlayStation and the Wii you interact with a remote control and you make an avatar, a thing… While with Cozmo you can drive it, but Cozmo also does things on its own…Uhm… Like, you look at Cozmo, it looks back at you and it says your name Cozmo as a lively entity
  • 12.
    Cozmo as anhybrid entity – children testing Cozmo’s liveliness Martina (8 yo, Family 1): If you can believe, I even tried to close his eyes but he knew it. Interviewer: What do you mean to close his eyes? Martina (8 yo, Family 1): So, to put your hands on it, but he knew it so he won even if I closed its eyes. Interviewer: Do you mean is he able to understand if the colours are correct even if you cover his eyes? Martina (8 yo, Family 1): Yes, since it is practically connected to the phone and to the cubes so it
  • 13.
    Cozmo as anhybrid entity Interviewer: And why doesn’t Cozmo understand you in your opinion? Sofia (8 yo, Family 2): because it is not Italian Margaux (5 yo, Family 2): it seems French Interviewer: and how does Cozmo speak? Sofia (8 yo, Family 2): in its own language Interviewer: that is not even English? Sofia (8 yo, Family 2): no, it is Cozmese
  • 14.
    2 Affective stickiness Luca(8 yo, Family 3): I feel affection for Cozmo. Because it plays with you…Uhm…it makes you happy. Olivia (Mother, Family 2): One thing that made me laugh is that for our youngest, it is a puppy. So, she would pet it, gave it kisses, I don’t know if in some photos you can she is tickling it. What really impressed me was that even though it is small and
  • 17.
    “Federal agencies mustavoid regulatory or non-regulatory actions that needlessly hamper AI innovation and growth… Agencies must avoid a precautionary approach that holds AI systems to such an impossibly high standard that society cannot enjoy their benefits.”
  • 18.
    Permissionless Innovation • “thenotion that experimentation with new technologies and business models should generally be permitted by default. • the general freedom to experiment and learn through ongoing trial-and-error experimentation. - Thierer, A. (2016). Permissionless Innovation: The Continuing Case for Comprehensive Technological Freedom
  • 19.
    Permissionless Innovation • Self-regulation •Cost-Benefit Analysis • The need to show evidence of a harm before regulating • “Regulatory humility” - Thierer, A. (2016). Permissionless Innovation: The Continuing Case for Comprehensive Technological Freedom “It is … vital that government officials … approach new technologies with a dose of regulatory humility, by working hard to educate ourselves and others about the innovation, understand its effects on consumers and the marketplace, identify benefits and likely harms, and, if harms do arise, consider whether existing laws and regulations are sufficient to address them, before assuming that new rules are required.” – FTC Commissioner Maureen Ohlhausen
  • 20.
    “the belief thatnew innovations should be curtailed or disallowed until their developers can prove that they will not cause any harm to individuals, groups, specific entities, cultural norms, or various existing laws, norms, or traditions. “many critics adopt a mindset that views the future as something that is to be feared, avoided, or at least carefully planned. This is known as the ‘stasis mentality' and it is, at its root, what motivates precautionary principle-based thinking and policymaking.’” - Ibid. (One way of) Defining the Precautionary Principle
  • 21.
    “PP1. Scientific uncertaintyshould not automatically preclude regulation of activities that pose a potential risk of significant harm (Non-Preclusion PP). PP2. Regulatory controls should incorporate a margin of safety; activities should be limited below the level at which no adverse effect has been observed or predicted (Margin of Safety PP). PP3. Activities that present an uncertain potential for significant harm should be subject to best technology available requirements to minimize the risk of harm unless the proponent of the activity shows that they present no appreciable risk of harm (BAT pp). PP4. Activities that present an uncertain potential for significant harm should be prohibited unless the proponent of the activity shows that it presents no appreciable risk of harm (Prohibitory PP).” - Stewart, R. (2002). “Environmental regulatory decision making under uncertainty,” (More fully) Defining the Precautionary Principle
  • 22.
    “Where there arethreats of serious or irreversible damage, lack of full scientific certainty shall not be used as a reason for postponing cost- effective measures to prevent environmental degradation.” - United Nations Rio Declaration on Environment and Development, 1992 Precautionary Principle and Environmental Regulation
  • 23.
    • addresses issuesof information and power asymmetry • spurs public debate • recognizes that monetary damages for future harms may be inadequate • takes account of the uncertainty of scientific evaluation - Adapted from Luiz Costa, L. (2012) ”Privacy and the Precautionary Principle” Precautionary Principle and Privacy & Data Protection
  • 26.
    Barrett, L. F.,Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930
  • 28.
    Some concerns aboutaffect recognition tech (1) Context beyond the face and body matters (2) Detecting facial movements ≠ detecting emotional states (3) Variance in how people communicate emotional states in practice: a spectrum - flawed, incomplete, rough approximation
  • 29.
    Some concerns aboutaffect recognition tech (1) Context beyond the face and body matters (2) Detecting facial movements ≠ detecting emotional states (3) Variance in how people communicate emotional states in practice: a spectrum - flawed, incomplete, rough approximation
  • 30.
  • 31.
    Harms Harms that result from technology workingas intended Harms that result from technology NOT working as intended privacy flawed Software doesn’t work Tools are biased Harvesting of emotions manipulation false positives / false negatives
  • 32.
    We’re surrounded bydubious claims about technology’s capabilities…..
  • 35.
  • 36.
    How to recogniseAI snake oil by Arvind Narayanan https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf
  • 37.
    https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf Inherently problematic: - Assessingidentity (ethnicity, gender, sexual orientation) - Judging intention (deceit, lieing) - Inner life Assessing & judging people Predicting the future How to recognise AI snake oil adapted
  • 40.
    CPDP 2020 Emotional AIand Empathic Technologies: Rights, Children, Domestication PANELLISTS Ben Bland (@ben_bland), Sensum, UK/Northern Ireland Frederike Kaltheuner (@F_Kaltheuner), Mozilla Fellow, UK/Germany Giovanna Mascheroni (@giovannamas), Università Cattolica del Sacro Cuore, Italy Gilad Rosner (@GiladRosner), IOT Privacy Forum, Eleos, Spain MODERATOR Andrew McStay (@digi_ad), UK, Bangor University, Eleos, UK