Safety is defined as a system's ability to operate without causing human injury or environmental damage. Safety and reliability are related but distinct, as a reliable system can still be unsafe if requirements are incorrect. There are three approaches to safety critical systems development: hazard avoidance through design, hazard detection and removal before accidents occur, and damage limitation to minimize harm from any accidents.

1. System safety
System safety, 2013
Slide 1

2. Safety
• Safety is a property of a system that
reflects the system’s ability to
operate, normally or abnormally, without
danger of causing human injury or death
and without damage to the system’s
environment.
System safety, 2013
Slide 2

3. Principal dependability
properties
System safety, 2013
Slide 3

4. Safety criticality
• Primary safety-critical systems
– Embedded software systems whose
failure can cause the associated
hardware to fail and directly threaten
people.
System safety 2013
Slide 4

5. System safety 2013
Slide 5

6. • Secondary safety-critical systems
– Systems whose failure results in faults in other
(socio-technical) systems, which can then have
safety consequences.
– For example, a medical records system is safetycritical as failure may lead to inappropriate
treatment being prescribed.
System safety 2013
Slide 6

7. System safety 2013
Slide 7

8. Safety and reliability
• Safety and reliability are related but
distinct
• For a system to be safe, it must
generally be reliable
• However, this is not enough. It must
also be safe in the event of system
failure
System safety 2013
Slide 8

9. • Reliability is concerned with
conformance to a given specification
and delivery of service
• Safety is concerned with ensuring
system cannot cause damage
irrespective of whether or not it
conforms to its specification
System safety 2013
Slide 9

10. • A reliable system can be unsafe if the
requirements are incorrect or
incomplete
System safety 2013
Slide 10

11. Unsafe reliable systems
The software on this plane operated ‘reliably’ yet the plane
crashed- covered in a separate video
System safety 2013
Slide 11

12. Unsafe reliable systems
• There may be dormant faults in a system
that are undetected for many years and
only rarely arise.
• Specification errors
– If the system specification is incorrect then
the system can behave as specified but still
cause an accident.
System safety 2013
Slide 12

13. • Hardware failures generating spurious inputs
– Hard to anticipate in the specification.
• Context-sensitive commands i.e. issuing the
right command at the wrong time
– Often the result of operator error.
System safety 2013
Slide 13

14. Accident
• An unplanned event or sequence of events
which results in human death or
injury, damage to property, or to the
environment.
• An overdose of insulin by a softwarecontrolled insulin injection system is an
example of an accident.
System safety 2013
Slide 14

15. Hazard
• A condition with the potential for causing or
contributing to an accident. A failure of the
sensor that measures blood glucose is an
example of a hazard.
System safety 2013
Slide 15

16. Damage
•
A measure of the loss resulting from a mishap.
Damage can range from many people being killed as
a result of an accident to minor injury or property
damage.
•
Damage resulting from an overdose of insulin could
be serious injury or the death of the user of the insulin
pump.
System safety 2013
Slide 16

17. Safety critical systems
development
• Hazard avoidance
– The system is designed so that some
classes of hazard simply cannot arise.
System safety 2013
Slide 17

18. • Hazard detection and removal
– The system is designed so that hazards are
detected and removed before they result in
an accident.
System safety 2013
Slide 18

19. • Damage limitation
– The system includes protection features
that minimise the damage that may result
from an accident.
System safety 2013
Slide 19

20. Summary
• Safety is a property of a system that reflects
the system’s ability to operate, normally or
abnormally, without danger of causing human
injury or death.
• Safety and reliability are related but reliable
systems may be unsafe.
System safety 2013
Slide 20

21. Summary
• Hazards are conditions that may (but need
not) lead to an accident
• Safety is achieved through
– Hazard avoidance
– Hazard detection and removal
– Damage limitation
System safety 2013
Slide 21