The document discusses issues around liability and automation in complex socio-technical systems. It examines who is responsible for accidents or problems in highly automated systems, and how liability can be assigned among various participants. It also looks at how different liability regulations between levels (international, national, local) can be harmonized. The ALIAS project specifically focuses on these issues in air traffic management but also other domains involving automation like healthcare and transport.

1. Liability and automation: issues and
challenges for socio-technical systems
Patrizia Marti | Deep Blue
Giovanni Sartor | European University Institute
Giuseppe Contissa | European University Institute

2. • Who is responsible for accidents or troublesome events in highly
automated systems?
• How do we apportion liability among the various participants in
complex socio-technical organisations?
• How can different liability regulations at different levels
(supranational, national, local) be harmonized?

3. The "Ping Body: An Internet
Actuated Performance“
By Stelarc

4. A series of muscle stimulating electrodes placed on various parts of Stelarc's body that
responded to remote users loggin on to the performance's web interface. Ping values
were gathered from the users' collective activity and caused areas of stelarc’s body to be
stimulated. Users also watched the resulting effects upon stelarc over a live webcast.

5. ALIAS (Addressing Liability in Automated Systems) co-financed
by EUROCONTROL on behalf of the SESAR Joint Undertaking -
Work Package E.
The project focuses on the legal implications of automation
exploring the wide spectrum of relations between automation
and liability.
Main focus on Air Traffic Management, but also on various
domains that face similar issues, such as HealthCare, ICT, Train
Transport, Navy, the automotive industry.

6. • In the time horizon of SESAR, that is over the next 30 years, a
new generation of air traffic management systems will be
developed.
• Such systems will be highly automated. They will make choices
and engage in actions with some level of human supervision, or
even without any such supervision.

7. • How different degrees of autonomy of agents and machines
shape the responsibilities of the different actors.
• How forthcoming operational concepts and procedures provide
challenges in the involvement of the different actors and their
consequent responsibilities.
• How existing laws regulate the allocation of liabilities in ATM, and
the assessment of whether such laws and regulations provide an
adequate normative framework.
• How to optimally allocate responsibilities in present and future
highly-automated socio-technical systems. Allocation of
responsibilities, not only as a way to distribute risks and
sanctions, but also as a means to prevent accidents and to
increase levels of safety and performance in ATM.

8. The development of the “Legal Case”, a methodological tool
including recommendations and guidelines to ensure that relevant
legal aspects are taken into consideration at the right stage of the
design, development and deployment process.
The creation of a Network of legal research in socio-technical
systems with the purpose of creating a multidisciplinary
community that will support knowledge construction and
distribution, sharing of cases and best practices, discussion on
the topics of interest, archiving of documents and references
useful to develop this research area.

9. The first activity of the Network will be the publication of a position
paper that is intended to “seed” the discussion forum, leading
hopefully to a collaborative effort from the network community in
developing a rich community corpus of stories arguments, and
interpretations.
The process of people recruiting for the ALIAS Network will start
during the SESAR Innovation Days 2011. We invite you to
register to the network and propose topics of discussion.
On line registration:
www.aliasnetwork.org/register.html

11. Socio-technical systems: examples

12. Socio-technical systems: basic structure
Institutions Technology
(rules, tasks, (hardware,
procedures) software)
People
(managers,
operators, users)

13. The problem of liability
• Who is responsible in case something goes
wrong
• Organisations (companies/agencies)?
• Operators and managers?
• Providers?
• Users?
• Who is liable to compensate the damage to
persons and goods
• Organisations (companies/agencies)?
• Operators and managers?
• Providers?
• Should anybody be sanctioned

14. Kinds of legal liability
Criminal liability Detention, fine
Compensatory
damages
Civil liability
Punitive damages
Legal
liability
Administrative
liability Fines, withdrawal of
privileges
Reprimand,
Disciplinary liability suspension,
dismissal

15. What grounds for liability
Negligence
Fault liability
Intentional tort
Grounds of civil Vicarious
liability Liability
Strict liability

16. Implications of automation
• Delegation of task from operators to technology
• Hybrid agency (designing man-machine symbiosis/
coagency)
• Humans as controllers and supervisors
• Machine intelligence and autonomy
• The challenge of complexity
• How to maintain control, prevent and mitigate failures

17. Automation and liability
• Who should be responsible liable in ATMS as STS
• Operators vs managers
• Individuals vs organisations (should we reject that myth of individual
ultimate responsibilty)
• Actors vs providers (of goods and services)
• Public authorities (States)
• Ways for allocation liabilities
• Organisation (tasks)
• Contract
• Laws (regulations)

18. Automation and liability (2)
• Objectives of the allocation of liabilities
• Prevent accidents
• Provide compensation
• Promote safety culture
• What to do about liability
• New public regulation is required, or
• Self-regulation, coupled with contractual mechanisms
• Reorganisation of tasks and internal responsibilities

19. Liability and software
• What kind of software liability?
• Service liability vs product liability
• Strict vs fault liability
• Limited vs unlimited liability
• Who is liable?
• Software producers vs software users
• Could damage be mitigated?
• Was the software acting autonomously?

20. Some questions (1)
• How automation transforms operators’ roles and tasks? What
impact on liability?
• Who is responsible for the behaviour of systems that humans
cannot fully monitor and control?
• Who is responsible for information supplied by automated
sydtems that the human cannot verify?
• Who is responsible for harms resulting from defective design
of human-machine interaction?
• Does strict liability constrains innovation?

21. Some questions (2)
• Who is responsible when operators fail to cope with
emergencies involving degraded automation?
• Who is responsible when operators defectively make up for
a non-performing automated systems? Will these be treated
as organizational, automation or human failures?
• Does compliance with current technical standards and
regulation exonerate from liability?
• How can insurance systems complement the liability
system? Can insurance fully protect from liabilities?

22. ALIAS Objectives
• Investigate liability and automation in ATM and in other
domains
• Build a“Network of Legal Research in ATM”, a
multidisciplinary community of practice to stimulate
international debate around liability and automation.
• provide the “Legal Case”, a methodological tool to support
the introduction of any technology in ATM, ensuring that
relevant legal issues are taken into consideration.

23. Chinook crash (Scotland 1994)
Unjust accusation to pilots, software failure

24. B-2 Spirit Stealth Bomber Crash, Guam (2008)
Unreported sensor failure, Hardware failure

25. Ariane 5 software failure (1996)
Faulty design, software failure

26. Ueberlingen mid-air collision(1996)
Human error, organizational failure, hardware failure

27. The team
• Giovanni Sartor, EUI - Project Leader
• Liam Bannon, DBL
• Giuseppe Contissa, EUI
• Paola Lanzi, DBL
• Patrizia Marti, DBL
• Hans Micklitz, EUI
• Francesco Quarta, EUI
• Marta Simoncini, EUI
• Emanuele Tarducci, DBL
Thanks for your attention
Please register to the ALIAS network!
www.aliasnetwork.org/register.html