Adam Evans & Kristian Cruickshank, Nova Systems - Developing UAV safety cases
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Adam Evans & Kristian Cruickshank, Nova Systems - Developing UAV safety cases

on

  • 247 views

Adam Evans & Kristian Cruickshank, Nova Systems delivered the presentation at the 2014 UAV Triple Zero Summit. ...

Adam Evans & Kristian Cruickshank, Nova Systems delivered the presentation at the 2014 UAV Triple Zero Summit.

The 2014 UAV Triple Zero Summit had a theme and focus on ‘Mobilising and Regulating UAVs in Australian Emergency Response’. It drew on government policy, current legislation and privacy protocol in establishing an informed analysis of the current and future scope surrounding the utilization of unmanned systems in this sector.

For more information about the event, please visit: http://www.informa.com.au/uavtriplezero14

Statistics

Views

Total Views
247
Views on SlideShare
247
Embed Views
0

Actions

Likes
1
Downloads
14
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Adam Evans & Kristian Cruickshank, Nova Systems - Developing UAV safety cases Presentation Transcript

  • 1. http://www.novasystems.com.au Experience Knowledge Independence Developing UAV Safety Cases UAV Triple Zero Summit Mr Adam Evans Mr Kristian Cruickshank 1
  • 2. http://www.novasystems.com.au Experience Knowledge Independence Overview 2 Nova’s Background What is a Safety Case? When is a Safety Case Required? UAV Safety Case Paradigm Safety Case Process Levels of Acceptable Risk Emergency Services Risk Context UAV Operation Risk Analysis Treating Unacceptable Risk UAV Safety Management Systems Consolidating the Safety Case
  • 3. http://www.novasystems.com.au Experience Knowledge Independence Terminology UAV vs UAS vs RPA vs RP vs RPAS Throughout this presentation: UAV = RPA UAS = RPAS UAV Controller = RP 3
  • 4. http://www.novasystems.com.au Experience Knowledge Independence Origins in Defence T&E Involved in all significant ADF UAS projects to date: Heron Shadow 200 Aerial Targets Nova contracted by ADF to develop UAV regulatory framework Specialists in Military and Civil Airworthiness, inclusive of operational and technical risk management Aeronautical Engineers Australia specialists in Civil Airworthiness and CASRs Practitioners in various aerospace engineering and operational domains Nova’s Background 4
  • 5. http://www.novasystems.com.au Experience Knowledge Independence What is a Safety Case? Broad Definition: A structured argument of compiled evidence demonstrating that a system is acceptably safe No CASA definition for UAV Safety Case CASA Airworthiness Circular for Aerodromes: “A documented body of evidence that provides a demonstrated and valid argument that a system is adequately safe for a given application and environment over its lifetime” (AC 139-16(1)) Propose that the definition used in AC 139- 16(1) is suitable for UAVs 5
  • 6. http://www.novasystems.com.au Experience Knowledge Independence What is a Safety Case? Elements of a UAV Safety Case Adequate Level of Safety. Benchmark is ‘acceptable’* level of risk posed to the general public. Given Application and Environment. Safety case must define the types of UAV operations and the environmental factors present in those operations Statement of Operating Intent (SOI) or Concept of Operations (CONOPS) or equivalent Key environmental factors are – population densities, physical environment, airspace category. Lifetime. UAV context may lessen the importance of this element – possibly more ‘disposable’ than most aircraft? Still requires consideration. * ‘Acceptable’ may vary depending on a given emergency services scenario 6
  • 7. http://www.novasystems.com.au Experience Knowledge Independence What is a Safety Case? Elements of a UAV Safety Case (cont) System. Unmanned Aerial System plus the Safety Management System or equivalent implemented. Demonstrated Argument. Logical, valid, and defensible argument constructed from applicable body of evidence. No specific CASA guidance on what the argument must consider Experience with Military UAS provides a reasonable basis for considerations 7
  • 8. http://www.novasystems.com.au Experience Knowledge Independence When is a Safety Case Required? Implied by NPRM1309OS (regulations and guidance not published yet) Intent of once-off Area Approval is the same as a safety case, but safety case can be enduring Operation of Large UAV (> 150kg) Operating outside of Standard Operating Conditions Over Populous Areas Beyond Visual Line Of Sight Greater than 400ft Other than Class G airspace Closer than 3NM from aerodrome 8
  • 9. http://www.novasystems.com.au Experience Knowledge Independence When is a Safety Case Required? Put Simply: UAV OPERATIONS THAT WOULD BE OF MOST BENEFIT TO EMERGENCY SERVICES! 9
  • 10. http://www.novasystems.com.au Experience Knowledge Independence Likely Scenarios for Safety Case Search and Rescue BVLOS, Over Populous Areas, Above 400ft Fire Spotting Restricted Airspace? Police Tactical Operations BVLOS, Over Populous Areas, Controlled Airspace Natural Disasters BVLOS, Over Populous Areas, Above 400ft, Launching from Aerodromes Others? 10
  • 11. http://www.novasystems.com.au Experience Knowledge Independence UAV Safety Case Paradigm Different approach than regular aircraft – Why? Aircraft Type Certification and Operational Management Regulations established and industry complies UAV origins – Hobby and Military No internationally recognised Type Certification Requirements established ‘Risk Management Approach’ instead of a ‘Compliance to Standards’ approach 11
  • 12. http://www.novasystems.com.au Experience Knowledge Independence UAV Safety Case Paradigm The Future Safety Paradigm – Establishing Compliance with Technical Airworthiness Requirements Confidence in Integrity of System Design Confidence in Quality of Manufacture Design of Maintenance Schemes that maintain aircraft reliability Same process as normal Aircraft Challenges with ‘The Future’ Cost Establishing requirements for different UAV categories (Small, Medium, Large, Commuter?) Detect and Avoid + more 12
  • 13. http://www.novasystems.com.au Experience Knowledge Independence UAV Safety Case Paradigm The Current Safety Paradigm – Technical and Operational Risk Management Defining Acceptable Levels of Risk to Public Determine worst credible Consequence of UAV accident Determine Probability of worst credible Consequence occurring Reliability of UAV (hardware reliability combined with integrity of software) – if possible to determine Probability of fatality/injury given impact Population density + more Technical and Operational risk treatments Plus ‘normal’ aircraft requirements (maintenance, flight operations system, Safety Management System, etc) 13
  • 14. http://www.novasystems.com.au Experience Knowledge Independence UAV Safety Case Process 14 Develop SOI / CONOPS Define Acceptable Levels of Risk System Safety Assessment Compare Risk to Acceptable Levels Risk Acceptable ? Develop Risk Mitigations Consolidate Safety Case Operational, maintenance, design, SOI change, etc Yes No Evidence SOI, Acceptable Risk, UAV design, Maintenance System, Safety Management System, Operators Manual, OEM Documentation
  • 15. http://www.novasystems.com.au Experience Knowledge Independence Statement of Operating Intent Analogous to Concept of Operations Derived from Military Context Defines types of operations and informs risk assessment process 15
  • 16. http://www.novasystems.com.au Experience Knowledge Independence Statement of Operating Intent Key Aspects Role. Function(s) or purpose(s) assigned to system – SAR, Fire Spotting, Surveillance, etc. Tasks are a sub-element of Role. Tasks to be conducted under a given role. Environment. Totality of surroundings/conditions of operations (airspace, areas of operation, physical environment, etc) Flight Envelope. Defines outermost boundary of flight conditions for UAV to remain airworthy. Flight Usage Spectrum. Flight Profiles for each task/role, frequency of profiles, Rate of Effort, etc. 16
  • 17. http://www.novasystems.com.au Experience Knowledge Independence Safety Case Process 17 Develop SOI / CONOPS Define Acceptable Levels of Risk System Safety Assessment Compare Risk to Acceptable Levels Risk Acceptable ? Develop Risk Mitigations Consolidate Safety Case Operational, maintenance, design, SOI change, etc Yes No Evidence SOI, Acceptable Risk, UAV design, Maintenance System, Safety Management System, Operators Manual, OEM Documentation
  • 18. http://www.novasystems.com.au Experience Knowledge Independence Safety Targets – Example Maximum acceptable Individual probability of fatality or serious injury to the General Public: 1 X10-7 per flight hour Maximum acceptable Collective fatality expectation to the General Public: 1000 X10-6 (1x10-3) per annum OR 5x10-7 per flight hour Maximum acceptable Individual probability of fatality or serious injury to the Mission Personnel: 1 X10-6 per flight hour Maximum acceptable Collective fatality expectation to the Mission Personnel: 10000 X10-6 (1 x10-2) per annum OR 1 X10-5 per flight hour 18
  • 19. http://www.novasystems.com.au Experience Knowledge Independence Defining Levels of Acceptable Risk What ‘level of safety’, integrity or reliability do we need to operate a 20kg UAV in an sparsely populated rural environment? What if the operation is attempting to prevent an assault? What if the operation is attempting to prevent a homicide? What if the operation is attempting to prevent multiple homicides What if the aircraft has sufficient range to fly into densely populated area? 19
  • 20. http://www.novasystems.com.au Experience Knowledge Independence Emergency Service Risk Context May be quite simple to balance risk When exposing the public to risks, the basis for determining the risk as acceptable must be able to stand up to public scrutiny 20 Public risk benefit from UAV operation Public risk exposure without UAV operatio
  • 21. http://www.novasystems.com.au Experience Knowledge Independence Example safety target Homicide Assault Probable 1x10-3 1x10-4 Likely 1x10-4 1x10-5 Unlikely 1x10-5 1x10-7 Rare 1x10-7 1x10-9
  • 22. http://www.novasystems.com.au Experience Knowledge Independence Explanation of Table UAV operations to prevent a Homicide are reasonable if the risk to the general public is less than 1x10-3 and it is determined that the assailant will Probably commit the crime. UAV operations to prevent 5 Assaults are reasonable if the risk to the general public is less than 5x10-7 and it is determined that the assailant could, but is Unlikely commit the crime. 22
  • 23. http://www.novasystems.com.au Experience Knowledge Independence Safety Case Process 23 Develop SOI / CONOPS Define Acceptable Levels of Risk System Safety Assessment Compare Risk to Acceptable Levels Risk Acceptable ? Develop Risk Mitigations Consolidate Safety Case Operational, maintenance, design, SOI change, etc Yes No Evidence SOI, Acceptable Risk, UAV design, Maintenance System, Safety Management System, Operators Manual, OEM Documentation
  • 24. http://www.novasystems.com.au Experience Knowledge Independence 24 System Safety Assessment Unrecoverable Failure Rate Unrecoverable Failure Rate unknown? Fault tree analysis to identify safety critical systems Engines/Navigation/Airframe/Autopilot/etc Various techniques to assess overall reliability / integrity of design Consider existing Operational Mechanisms Use Casualty/Fatality Expectation Rate Analysis to quantify risks to personnel End Product is Unmitigated Risk Software reliability?
  • 25. http://www.novasystems.com.au Experience Knowledge Independence FTA for Military UAS operations – Air Vehicle Escape 25
  • 26. http://www.novasystems.com.au Experience Knowledge Independence Casualty Expectation Methodologies Once Unrecoverable Failure Rates for the vehicle are known Used to determine Collective and Individual Risks to General Public and Mission Essential Personnel Based on population densities and Lethal Area of Vehicle 26
  • 27. http://www.novasystems.com.au Experience Knowledge Independence Casualty Expectation CE = λ x PCasualty|Strike x PStrike|Impact x PImpact CE – Casualty Expectation (collective risk) λ – Reliability PImpact – Probability of high energy crash given a failure PStrike|Impact - Probability of striking an individual PCasualty|Strike - Probability of killing someone 27
  • 28. http://www.novasystems.com.au Experience Knowledge Independence Casualty Expectation Pimpact Difficult to determine (e.g. reliability of FTS) Substantial computational resources Integrate over all possible crash locations Pstrike/impact Exposure time Population density Lethal Area of Vehicle 28
  • 29. http://www.novasystems.com.au Experience Knowledge Independence Casualty Expectation Pcasualty/strike Depends on debris KE and explosive energy in the Air Vehicle Requires analysis of various materials in Air Vehicle CASA paper assists
  • 30. http://www.novasystems.com.au Experience Knowledge Independence Risk Comparison 30 Develop SOI / CONOPS Define Acceptable Levels of Risk System Safety Assessment Compare Risk to Acceptable Levels Risk Acceptable ? Develop Risk Mitigations Consolidate Safety Case Operational, maintenance, design, SOI change, etc Yes No Evidence SOI, Acceptable Risk, UAV design, Maintenance System, Safety Management System, Operators Manual, OEM Documentation
  • 31. http://www.novasystems.com.au Experience Knowledge Independence Risk Comparison CE = λ x PCasualty|Strike x PStrike|Impact x Pimpact 31 Public risk benefit from UAV operation Public risk exposure without UAV operatio Prevent possible homicide (1x10-4)
  • 32. http://www.novasystems.com.au Experience Knowledge Independence Risk Mitigation 32 Develop SOI / CONOPS Define Acceptable Levels of Risk System Safety Assessment Compare Risk to Acceptable Levels Risk Acceptable ? Develop Risk Mitigations Consolidate Safety Case Operational, maintenance, design, SOI change, etc Yes No Evidence SOI, Acceptable Risk, UAV design, Maintenance System, Safety Management System, Operators Manual, OEM Documentation
  • 33. http://www.novasystems.com.au Experience Knowledge Independence Risk Mitigation Develop Risk Treatments if Acceptable threshold exceeded Operational Treatments: Restrict range of UAV Extended VLOS Operations only up to (X) population density Etc Technical Treatments: Different UAV OEM redesign (datalink reliability) 33
  • 34. http://www.novasystems.com.au Experience Knowledge Independence Minimising Risk (below acceptable?) Emergency services have a duty of care to minimise the risk to the public Further work could be done in order to identify risk levels at the front line Flight plans could be optimised to reduce risk to public Aircraft type or configuration selected to reduce risk Possibility for assumed clearance for flight if specific criteria is satisfied (as specified by Operations Manual/Safety Management System) 34
  • 35. http://www.novasystems.com.au Experience Knowledge Independence UAV Safety Management Systems Likely that specific risk mitigating processes or techniques will need to be enacted during operations If there is an ongoing need to identify and treat risks, or an ongoing Operational Risk Management process – this will form a large portion of the UAV Safety Management System Best place for these to be documented and enforced (forming part of the safety case) will be in a Safety Management System 35
  • 36. http://www.novasystems.com.au Experience Knowledge Independence UAV Safety Management System Ongoing Risk Assessments may include: Mission Planning Processes and Tools Onsite Risk Assessments Particularly relevant to Emergency Services Risk Context Risk Assessment and Treatment on repairs or maintenance More? 36
  • 37. http://www.novasystems.com.au Experience Knowledge Independence UAV Safety Management Systems No specific requirement in CASR 101 (or NPRM1309OS) However, intent of identifying and managing safety risks associated with UAVs is applicable SMS Gap Analysis Tool provided by CASA Operations Manual and other corporate plans/procedures may be sufficient without a dedicated SMS for most operations Likely that most Operations Manuals would include these considerations, but a dedicated SMS may be advisable for large UAVs and non-standard operating conditions 37
  • 38. http://www.novasystems.com.au Experience Knowledge Independence Consolidating the Safety Case 38 Develop SOI / CONOPS Define Acceptable Levels of Risk System Safety Assessment Compare Risk to Acceptable Levels Risk Acceptable ? Develop Risk Mitigations Consolidate Safety Case Operational, maintenance, design, SOI change, etc Yes No Evidence SOI, Acceptable Risk, UAV design, Maintenance System, Safety Management System, Operators Manual, OEM Documentation
  • 39. http://www.novasystems.com.au Experience Knowledge Independence Consolidating the Safety Case Don’t make it too scenario dependent What if you haven’t thought of all scenarios? Flexibility where operational functions and risks remain valid Structured argument Outline the process SOI/CONOPS Justify Risk Acceptability (where did safety targets come from?) Describe Risk Mitigations (where necessary, show that they’ve been incorporated into design/operations) 39
  • 40. http://www.novasystems.com.au Experience Knowledge Independence Consolidating the Safety Case Various techniques for ‘Structuring’ argument (Goal Structured Notation is a good method) 40 Safe UAV Operation SOI/CONOPS Safety Target(s) System Reliability / Integrity System Limitations Op Risk Manage Ops/Maint Processes System Safety Assess
  • 41. http://www.novasystems.com.au Experience Knowledge Independence Consolidating the Safety Case If residual risk is unacceptable – Talk to CASA In some cases the risk may simply be too high Back to the drawing board The UAV you intended to use may not be the answer UAVs may not be the answer Finally – Submit the Safety Case! Hopefully this is not the first time CASA has seen it... Get them involved from the start for planned ‘high’ risk operations 41
  • 42. http://www.novasystems.com.au Experience Knowledge Independence Questions? 42
  • 43. http://www.novasystems.com.au Experience Knowledge Independence Developing UAV Safety Cases UAV Triple Zero Summit Mr Adam Evans http://www.linkedin.com/pub/adam-evans/12/850/b73 Mr Kristian Cruickshank http://www.linkedin.com/pub/kristian-cruickshank/34/922/ba4 43