This document discusses responsible artificial intelligence and human-agent-robot interaction. It addresses the importance of designing AI systems to be autonomous while also being responsible, predictable, and able to explain their decisions and actions to humans. The document advocates for an approach called "ethics by design" where values are aligned and ethical behavior is implemented from the beginning of an AI system's development. This involves determining which human values and ethical theories should guide a system's behavior, and how to translate these into functional requirements and choices around implementation, users, and institutions. The goal is to develop AI that is ethically acceptable, socially accepted, and legally allowed.

1. © Virginia Dignum, 2016Responsible Artificial Intelligence
Responsible interaction
in human-agent-robot teamwork
Virginia Dignum
SAIL - Social Artificial Intelligence Lab
Delft University of Technology
email: m.v.dignum@tudelft.nl
twitter: @vdignum
url: staff.tudelft.nl/en/M.V.Dignum

2. © Virginia Dignum, 2016Responsible Artificial Intelligence
PUTTING PEOPLE FIRST
IN THE DIGITAL ERA
• Human-agent-robot interaction
• Ethics by Design

3. © Virginia Dignum, 2016Responsible Artificial Intelligence
Human-like AI
Partners
Deep learning
Big data
Autonomy
intelligent
system
Autonomous
vehicles
Artificial Intelligence

4. © Virginia Dignum, 2016Responsible Artificial Intelligence
Autonomy
• Autonomous agents
• Proactive (goal-directed)
• Reactive (response-stimulus)
• Social (communication)
• Act on an environment
• Action / plan autonomy
• Goal autonomy
• Motive autonomy

5. © Virginia Dignum, 2016Responsible Artificial Intelligence
Autonomy
• Autonomous agents
• Proactive (goal-directed)
• Reactive (response-stimulus)
• Social (communication)
• Act on an environment
• Action / plan autonomy
• Goal autonomy
• Motive autonomy

6. © Virginia Dignum, 2016Responsible Artificial Intelligence
Adaptability
• Machine Learning
• Focus is on performance, optimization
• Bias from data
• Leads to black box systems
• We must build for transparency
• Needs new focus in ML
• Ethical feedback
• Educate AI
• Like children, adjusted expectations
• Training wheels / L-plates
• Less is more
• ML is heavily relying on data correlation
• Abstraction / causality
• https://www.linkedin.com/pulse/small-data-next-big-thing-ai-make-smarter-
virginia-dignum/

7. © Virginia Dignum, 2016Responsible Artificial Intelligence
Interaction
Together, AI and people, we can do much more
http://ieeexplore.ieee.org/document/6249609/

8. © Virginia Dignum, 2016Responsible Artificial Intelligence
Interaction
• Teamplayers
• Commitment
• Mutually predictable in their
actions
• Mutually directable
• Maintain common ground
• Challenges
• Model the ‘other’ (Theory of
Mind)
• Coordination (take and give
control)
• Negotiation

9. © Virginia Dignum, 2016Responsible Artificial Intelligence
Designing for interaction
• Interaction is dependability
• Interaction is capacity
• Interaction is relationship
• Principles
• observability
• predictability
• directability

10. © Virginia Dignum, 2016Responsible Artificial Intelligence
Johnson, M, Bradshaw, J, Feltovich, P, Jonker, C, Van Riemsdijk, B, Sierhuis, M:
https://www.researchgate.net/publication/260479210_Coactive_Design_Designing_Supp
ort_for_Interdependence_in_Joint_Activity
Coactive Design: Designing Support for
Interdependence in Joint Activity

11. © Virginia Dignum, 2016Responsible Artificial Intelligence
Interdependence analysis

12. © Virginia Dignum, 2016Responsible Artificial Intelligence
Interdependence analysis - example

13. © Virginia Dignum, 2016Responsible Artificial Intelligence
From AI to Responsible Artificial Intelligence
Autonomy
intelligent
system
Responsibility
decide
explain
inspect

14. © Virginia Dignum, 2016Responsible Artificial Intelligence
Ethics in AI design
• AI systems will take decisions that have ethical grounds and
consequences
• Need for design methods that ensure
ART• Accountability
• Explanation and justification
• Responsibility
• Chain of responsible actors
• AI is artefact!
• Transparency
• Data and processes
• Algorithms
See more: https://ai.xprize.org/news/bringing-art-ai

15. © Virginia Dignum, 2016Responsible Artificial Intelligence
Ethics by Design - requirements
1. Value alignment
• Identify relevant human values
• Are there universal human values?
• Who gets a say? Why these?
2. Ethical Behaviour
• Ethical theories: How to behave according to these values?
• How to prioritize those values?
3. Implementation
• Role of user
• Role of society
• Role of AI system

16. © Virginia Dignum, 2016Responsible Artificial Intelligence
Ethically
acceptable
Socially
accepted
Legally
allowed
1. Which values?
Sources: practices, law, ethics

17. © Virginia Dignum, 2016Responsible Artificial Intelligence
2. Which Ethics theories ?
• Teleology / Utilitarianism
• Results matter
• It is rational
• reasons can be given to explain why actions are good or bad
• But it ignores the unjust distribution of good consequences
• Deontology / Kant
• Actions matter; people matter
• It is rational, i.e. logic can be used to determine if actions are ethical, but
• If several rules apply gives no way to resolve a conflict between rules
• It allows no exceptions to moral rules
• Virtues
• Motives matter
• Relational rather than rational
• Follow virtuous examples Does not provide ways to resolve conflicting
rights
• Deontology and Virtue Ethics focus on the individual decision
makers while Teleology considers on all affected parties.

18. © Virginia Dignum, 2016Responsible Artificial Intelligence
2. Which ethics theories?
Ethical Autonomous Vehicle
• Utilitarian car
• The best for most; results matter
• maximize lives
• Kantian car
• Do no harmful action; people and actions matter
• do not take a decision to swerve if that action causes others harm
• Aristotelian car
• Pure motives; motives matter
• Harm the least;
• Rawls car
• Fairness matters
• spare the least advantaged (pedestrians?)
Can you personalise yours?

19. © Virginia Dignum, 2016Responsible Artificial Intelligence
3. Implementation: From values to
functionalities
values
social norms
functionalities
interpretation
concretization
safety
speed < 100 crash-worth
…
…
How ethical are our norms?
How social are our actions?

20. © Virginia Dignum, 2016Responsible Artificial Intelligence
user&machine
machine
3. Implementation choices
algorithmic
regulation
random
collaboration
infrastructures
& institutions

21. © Virginia Dignum, 2016Responsible Artificial Intelligence
Ethical decisions by Autonomous Systems
• Responsible Artificial Intelligence
• Society shapes and is shaped by design
• The AI systems we develop
• The processes we follow
• The institutions we establish
• Knowing ethics is not being ethical
• Not for us and not for machines
• Responsible AI concerns the systems AND concerns us
• We design, we use, we are responsible
• Ethics is the new green

22. Responsible Artificial Intelligence
WE ARE RESPONSIBLE