The Challenges of SoS Assurance:   A Safety Case Perspective         George Despotou             © The University of York
Overview• Systems of Systems    – Example•   Safety lifecycle•   (Top down) safety case•   SoS hazards and related argumen...
Systems of Systems                                                       • Geographical dispersion• Distinct class of syst...
Challenges in SoS Analysis• Evolving systems   – Constantly evolving and adapting• Emergent behaviour   – Behaviour affect...
Preliminary     ABM Stages                                                                        Scenario-based  SoS Safe...
SoS Safety Case• Safety case principles  are the same                                              SoS Safety Argument• Ar...
Top down start                                                       • Definition of                                      ...
Definition of safety context                                                           MODAF Ov1• Scenario based approache...
How is the capability offered?• Many systems        Supplier A                     Supplier B       Supplier C  collaborat...
Multiple supplier safety case                                                                            Definition of Saf...
Multiple supplier safety case                                                                            Definition of Saf...
SoS Hazard Anlysis• Focusing on individual systems is a starting point (e.g.  DDA)   – But not sufficient• Obscure (comple...
The role of simulation• Exhaustive  – Cannot claim that  – Only to a degree of confidence• (Exhaustive) manual analysis al...
Challenges for simulation• Two well known challenges   – Valid      • Do the simulated models represent the right system  ...
Safety Requirements                                                                    • System AC will support the safety...
(Addressing) Cross supplier               hazards       Definition of Safety           (argument)                         ...
Hazard controls• Discharging (safety related) requirements  – Potential dependencies among suppliers     • Coordination th...
Definition of Safety                                                                     (argument)SoS Safety  Case       ...
Safety Case - Responsibility• Not clear  – Except for assurance cases for individual systems  – Interweaved interest     •...
SoS Acquisition Challenges• Increasingly apparent that collaboration amongst  stakeholders is inevitable• Commercial/compe...
Synopsis• SoS demonstrate a distinct combination of  characteristics• Challenged to safety analysis• Safety case principle...
Further Info• SoS community  – www.dependablesos.org     • Informal; contributions welcome• www-users.cs.york.ac.uk/~georg...
Upcoming SlideShare
Loading in …5
×

The challenges of so s assurance

294 views

Published on

Published in: Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
294
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

The challenges of so s assurance

  1. 1. The Challenges of SoS Assurance: A Safety Case Perspective George Despotou © The University of York
  2. 2. Overview• Systems of Systems – Example• Safety lifecycle• (Top down) safety case• SoS hazards and related arguments• Overview of the safety case• Related challenges © The University of York
  3. 3. Systems of Systems • Geographical dispersion• Distinct class of systems • Overall objectives demonstrating certain • Complexity attributes • Multiple elements • Autonomy • Collaboration • Communication • Heterogeneity• Common methods often fall short for these type of systems © The University of York
  4. 4. Challenges in SoS Analysis• Evolving systems – Constantly evolving and adapting• Emergent behaviour – Behaviour affected by this of its components• Functional composition – SoS functions composed of elements’ functions• Network centred – Heavy reliance on communications• Multi-attribute failures – Element failures propagate and manifest as different types – Trade-offs © The University of York
  5. 5. Preliminary ABM Stages Scenario-based SoS Safety Hazard Identification Overall Scenario and Capabilities Lifecycle Common Mode & Emergent (SoS) Hazards Activity-based Activities, Hazard Actors and Analysis Information• Extends ARP Preliminary System lifecycle SoS Safety Assessment Functions, Data and connectivity• Incorporates SoS System modelling Safety Assessment System approaches SoS Safety Realisation and Infrastructure (MODAF/DODAF) Assessment often black box Safety Synthesis © The University of York
  6. 6. SoS Safety Case• Safety case principles are the same SoS Safety Argument• Argument: series of statements (claim) about properties (safety) of the system• Evidence supporting Evidence the claims © The University of York
  7. 7. Top down start • Definition of safety always Definition of Safety (argument) requiresSoS Safety Argument consideration of operation Elaboration (relevance) • Capability based in SoS Evidence Evidence • Scope not always clear © The University of York
  8. 8. Definition of safety context MODAF Ov1• Scenario based approaches• Different acceptability criteria for each• Safety tolerance (risk)• Operational fitness – Performance, security, •Hidden sub-scenarios may be present (roles) availability etc. •e.g. S&R, intelligence, suppression, maintenance etc.• Identification of stakeholders goals •Understanding the variations of requirements in each •Crucial for later stages (ALARP) © The University of York
  9. 9. How is the capability offered?• Many systems Supplier A Supplier B Supplier C collaborating• Different vendors Army Operator Facets• Different technologies, Air Force standards, methods Comms• Often in isolation Command (competition?) with Allies each other• However, common operator/customer Supplier D Supplier E – Customer structure © The University of York
  10. 10. Multiple supplier safety case Definition of Safety (argument)Assurance Case for System X Assurance Case for System Y Assurance Case for System Z Argument for Argument for Argument for System System System X Y Z Evidence for Evidence for Evidence for system X system Y system Z Supplier A Supplier B Supplier C © The University of York
  11. 11. Multiple supplier safety case Definition of Safety (argument) This is a challenging stepAssurance Case for System X Assurance Case for System Y Assurance Case for System Z Argument for Argument for Argument for System System System X Y Z Evidence for Evidence for Evidence for system X system Y system Z Supplier A Supplier B Supplier C © The University of York
  12. 12. SoS Hazard Anlysis• Focusing on individual systems is a starting point (e.g. DDA) – But not sufficient• Obscure (complex) causal chain to hazards – Combination, propagation and transformation of failures • The degree of this phenomenon can be profound – Controls for a single hazard may need to be implemented by multiple suppliers – Often difficult to predict behaviour (e.g. autonomy)• Not always deviant behaviour – An SoS can operate as designed • Yet resulting in (emergent) hazards © The University of York
  13. 13. The role of simulation• Exhaustive – Cannot claim that – Only to a degree of confidence• (Exhaustive) manual analysis almost impossible – Scale – Configurations • Known and unknown (adaptability)• Simulation offers a ‘brute force’ approach to identifying hazardous conditions – Fault injection © The University of York
  14. 14. Challenges for simulation• Two well known challenges – Valid • Do the simulated models represent the right system – Verified • Has the simulation been implemented correctly?• Both may undermine our confidence in simulation• Effectiveness of simulation is unknown – Cannot argue whether the potential risk reduction from simulation justifies the cost• Need for more empirical evidence about simulation• And tactics to argue/achieve validation and verification © The University of York
  15. 15. Safety Requirements • System AC will support the safety Definition of Safety argument (argument) – i.e. hazard oriented argument Dependency Definition • Follows identification of SoS(hazard controls argument) hazards – And the contribution of each system to them Assurance Case X Assurance Case Y Assurance Case Z • Assumes ‘straight forward’ contribution of system operation to hazards – Discharging safety requirements to suppliers © The University of York
  16. 16. (Addressing) Cross supplier hazards Definition of Safety (argument) • Hazards occurring from Dependency Definition combination of failures (hazard controls argument) • Addressing the hazard may result in depending on another suppliers assurance caseAssurance Case X – Challenges during acquisition Assurance • Cannot anticipate specific Case Y dependencies in advance – Impact on acquisition processEvidence (and contracts) Evidence © The University of York
  17. 17. Hazard controls• Discharging (safety related) requirements – Potential dependencies among suppliers • Coordination throughout acquisition is crucial• Safety policy – Implemented on top of the SoS elements – Who will be responsible to monitor/enforce it • Controller vs. distributed policy • Any additional requirements to suppliers? – Onus on the operator to analyse hazards and argue safety • May result in the operator having to produce evidence © The University of York
  18. 18. Definition of Safety (argument)SoS Safety Case Hazard Controls Argument Other means SoS Safety (e.g. SDR Policy controller) Argument Argument Operational SC X Operational SC Y Operational SC Z AC about other safety related Assurance Case X Assurance Case Y Assurance Case Z systems © The University of York
  19. 19. Safety Case - Responsibility• Not clear – Except for assurance cases for individual systems – Interweaved interest • Supplier needs to be aware of hazard controls – Dependencies on their system » SDR and policy – Claims made, which may depend on other supplier• Who provides the contextual information? – E.g. SoS hazard analysis • operator: ability to do so? • supplier: access to information? – Inevitable assumptions © The University of York
  20. 20. SoS Acquisition Challenges• Increasingly apparent that collaboration amongst stakeholders is inevitable• Commercial/competition issues• Customer/operator driven process – Clear identification of stakeholder interfaces – Cannot be a single demonstration • Continuous process and support• Not zero sum (?) – It is in the interests of the suppliers to increase collaboration (?) © The University of York
  21. 21. Synopsis• SoS demonstrate a distinct combination of characteristics• Challenged to safety analysis• Safety case principles are the same• SoS level hazards are difficult to identify• Allocation of requirements can involve multiple stakeholders• Often, dependencies between suppliers• Responsibility of the safety case is unclear• Some of these issues can be incorporated in the acquisition process © The University of York
  22. 22. Further Info• SoS community – www.dependablesos.org • Informal; contributions welcome• www-users.cs.york.ac.uk/~george• Assurance cases – GSN: www.goalstructuringnotation.info – AC editor: http://code.google.com/p/acedit/ © The University of York

×