Principles of Independent Safety Assessment (ISA) MAY 2011 Written by Billy Fong Senior Consultant Edgilis Pte. Ltd. E firstname.lastname@example.org W edgilis.comAbstractIndependent Safety Assessment (ISA) has become increasingly common for railway projects in recent years.The main objective of commissioning an ISA is to provide assurance that safety management processes havebeen adequately implemented and that the risk associated with the project has been reduced to a level AsLow As Reasonably Practicable (ALARP).This paper details the ISA concept, the role of an ISA, the benefits of commissioning an ISA, and someexamples of ISA methodologies that can be adopted.
Principles of Independent Safety Assessment Table of Contents 1. Introduction 3 2. What is ISA and Do I Need One? 4 3. Role of ISA 5 4. ISA Competency: who should do it? 5 5. ISA Methodologies 6 6. Closing Thoughts 7 7. References 8edgilis.com | May 2011 Page 2 of 8
Principles of Independent Safety Assessment1. Introduction Rail is one of the most complex sectors, from both a technical and business standpoint. Railway authorities, which are generally public institutions, have to leverage a supplier base composed of worldwide technology suppliers in order to build rail networks which can compete with other transportation modes. These systems, which are already increasingly complex, also need to be able to operate with neighbouring railway networks, and to support a never-ending pursuit of higher operational efficiency and decreased maintenance costs. Amidst all this, railway authorities are held accountable for the safety of their networks and systems, with ever-more stringent safety regulations. Assessing the safety of system is therefore a constantly renewed exercise during which railway authorities must identify the main risks and ensure they are mitigated over time. There are a seemingly endless number of books and manuals on system safety standards and best practices, but experience demonstrates that even a strict enforcement of those stringent rules does not guarantee a safe system. Indeed, applying rigorous requirement specification is a prerequisite, but does not mitigate system risks to acceptable levels. Some typical problem areas within system safety management are: Inaccurate or incomplete identification of hazards and requirements; Inappropriate depth of analysis; Incomplete safety argument; Inadequate evidence supporting the argument; and Insufficient competency or experience of safety engineers. One way of reducing the occurrence of such problems is to have an independent assessment of these elements of the system safety process. Independent Safety Assessment (ISA) can play a major role in ensuring that these elements are not compromised on a project. In order to provide this level of assurance, the Assessor needs to access all the project material, as it is being produced, and understand the design decisions that were made during the early stages of development.edgilis.com | May 2011 Page 3 of 8
Principles of Independent Safety Assessment2. What is ISA and Do I Need One? ISA helps to make a judgement about the safety adequacy of a product, system or process in a particular context and environment and against a set of requirements. During an ISA, it may be required to demonstrate compliance to recognised standards. Depending on projects’ requirements, full compliance may not be necessary. The purpose of an ISA is to audit and assess processes used in a project to show compliance to best and appropriate practice and to assess the adequacy of the evidence that has been generated during application of those processes. An ISA offers an independent view of the safety processes on a project based on experience and a thorough understanding of the relevant standards. The main objective is to provide assurance that a contractor/supplier not only considers but also addresses safety issues appropriately. Other motivations or benefits for undertaking an ISA include: To comply with a Standard that requires an ISA – for instance, when carrying out work in accordance with Defence Standard 00-56 or for safety critical systems for the UK railway industry; To provide added confidence that safety claims are justified and that any weaknesses are identified and dealt with, as it is done independently from existing safety analysis and assessment; and To demonstrate to a regulator that your system is safe – although there is no mandate for the use of an ISA for all railway projects, there are advantages and benefits that can be gained from effective use of the ISA roles (e.g. identifying and closing potential gaps at early design stages, which is naturally more cost-effective than making a change in manufacturing or later project stages; and providing assurance to stakeholders that a sound safety management is being professionally managed for the project throughout the lifecycle). In many cases, conducting an ISA is just good practice. Using an ISA may help the contractor/supplier in safety planning and analyses. This tends to happen naturally during the audit/assessment process during which the contractor/supplier is providing information to address the queries by the Assessor. Additionally, during the early stages of a project, an Assessor can often provide generic guidance or advice, as long as independence is not being compromised.edgilis.com | May 2011 Page 4 of 8
Principles of Independent Safety Assessment3. Role of ISA Commissioning an ISA and defining its role should be done as early as possible in the project lifecycle development. Generally, the frequency and depth of the safety audits and assessments as well as the level of independence of the Assessor is based on the varying levels of complexity and risk presented by the project. Typically, projects of less complexity or with lower risks can be handled by a single assessor, who may well be working for the contractor directly. However, undertaking safety audits and assessments of very complex and high risk projects will likely to involve a team of assessors from an independent organisation. The organisation commissioning an ISA should prepare a remit with the requirements of the ISA including, but not limited to: qualifications, experience and level of independence of the Assessor including any references to previous audits and assessments; the scope of the audit/assessment. This could be limited to certain subsystems within a system (e.g. subsystems that have undergone a design change since last release); the purpose of the audit/assessment (e.g. as a supporting document to be submitted for management for approval); and the basis of the audit/assessment (e.g. the documents that the project will be audited against and the safety management framework within which the project is being run). The Assessor needs to be convinced that the process captures, understands and mitigates the hazards and identifies safety requirements associated with a system. This is carried out by a review of the safety analysis and supporting documents that leads to the development of the system Safety Case.4. ISA Competency: who should do it? An Assessor should be able to evaluate the safety activities free from conflicts of interest. Even if a client is paying for the ISA’s services, there should be a level of professional independence such that the ISA is not influenced by project timescale and pressure from management. Certain organizations have built a track record for delivering sound and professional audits.edgilis.com | May 2011 Page 5 of 8
Principles of Independent Safety Assessment The Assessor has to provide an authoritative, expert opinion on safety, and therefore has to be properly qualified. The Assessor needs both technical and managerial skills in order to plan, arbitrate, moderate meetings, and defend his position in a firm but non-confrontational manner. A balanced team should be managed and coordinated by a team leader and engineers qualified to provide in-depth knowledge of the individual systems and functions. Competency requirements for ISA generally include the following: Technical competency in safety engineering, including knowledge of the principles and concepts of safety management (e.g. ALARP, risk and safety requirements), and of the safety analysis techniques (e.g. HAZOPs, QRA, and Hazard Log Management). The ability to judge the scope and depth of analyses carried out is also important. Technical competency in the application domain, which should cover an understanding of the specific technologies used and their context in the particular domain. Assessors need to have the engineering knowledge and relevant experience in the application area and technology. Auditing and assessment competency in managing the various ISA steps from determining the scope and objectives to collecting and analysing the evidence to support the expert opinion to making a judgement on the safety of a system and being able to document the findings in a clear and unambiguous manner. Behavioural competence – Assessors will need to rely on their interpersonal skills, their ability to communicate and interview personnel at all levels of the organisation and their reporting and presentation skills. They must also have demonstrated their integrity and trustworthiness. ISA organisations should be able to supply evidence of competence covering these attributes, supported by verifiable examples, as part of their proposal when bidding for an ISA role.5. ISA Methodologies ISA comprises of two main activities: Process review and auditing for compliance to standards and safety plan. Independent analysis in order to assess the implementation and results of project safety tasks. In many cases, ISAs are performed by a team, as opposed to a single person, in order to review the technical data and the processes separately and to allow for a more effective peer review. Toedgilis.com | May 2011 Page 6 of 8
Principles of Independent Safety Assessment supplement the basic ISA methods of auditing and assessment (i.e. documentation review), the following tools can also be applied: Sampling – Assessing all the related evidence may not be practical on some projects, for instance for a well-defined engineering process which generates large volumes of evidence or documentation (from the Hazard Analysis and Hazard Log, FMECA, etc), which is an opportunity to use sampling. Should the sampling reveal significant problems or issues, then further detailed assessment could be conducted. Vertical Slice Analysis – The objective of this activity is to trace the mitigation of a hazard throughout the system lifecycle. The Assessor assesses the safety and design requirements derived from a particular hazard, the specifications for implementation and the supporting verification and validation evidence. The Assessor then builds an overall picture of how the safety argument for a particular hazard was developed throughout the lifecycle. This approach is useful when assessing a hazard of particular concern, or when assessing the overall effectiveness of a project’s system engineering process. Diverse Analysis – This can increase confidence in some critical aspect by performing an analysis that differs from the one performed by the Project. The analysis is not entirely repeated, but conducted enough to gain confidence under review. For example, the ISA Team could perform an independent HAZOP on an area of particular concern and compare the results with the Project’s analysis. This may increase confidence significantly more than an individual assessor reviewing the Project’s HAZOP report.6. Closing Thoughts Early ISA involvement in a project can identify potential risks, especially with complex systems. Effective usage of the ISA role can help to significantly de-risk a project. ISA increases the ability to deliver a system in-line with international standards for Safety and RAM, they help to identify and closes potential gaps in the Railway Authority’s requirements at the early design stages, when design changes are easier and more cost-effective to implement and they help Railway Authorities gain confidence that the as-built system will meet their aspirations.edgilis.com | May 2011 Page 7 of 8
Principles of Independent Safety Assessment7. References i. Engineering Safety Management (The Yellow Book), Fundamentals and Guidance Issue 4. ii. MOD, Def Stan 00-56/2, Safety Management Requirements for Defence Systems, 13 th December 1996. iii. IEE/BCS, Safety, Competency and commitment, competency guidelines for Safety related systems practitioners. 1999.8. Author Biography Billy is currently appointed as the team leader responsible for managing the System Assurance Centre of Excellence within Edgilis. He leads a team of System Assurance specialists in delivering a wide range of RAMS services to various industries. Billy has acquired significant experience in performing RAMS studies/activities across a variety of railway projects in a number of Billy Fong Senior Consultant countries including Australia, Dubai, Johannesburg, Hong Kong, Edgilis Pte. Ltd. Taiwan and Singapore. 3 Fusionopolis Way Symbiosis #05-20 Singapore 138633 In the delivery of these projects, he has undertaken an assortment of T +65 6304 5311 F +65 6467 8900 project roles including Project Manager, Project Coordinator, Lead E email@example.com Safety Consultant/Analyst, System Assurance Manager and Independent Safety Assessor.edgilis.com | May 2011 Page 8 of 8