(Soon to be in the new Office of Nuclear Safety and Environmental Policy, HS-21) I have been involved in identifying and generating ES&H performance measures for several years. Over this time, I have been particularly interested in how we can identify those elusive “leading indicators” we all keep searching for Through my readings and discussion, I’ve come to some conclusions about “best practices,” which I want to share today I hope that you will find them stimulating and persuasive. But I should add that these are my personal view. I’m not speaking for EH, (or HSS) or for “headquarters”
My intent is not to survey the specific performance measures used in these different examples, but rather to step back and identify a key organizing principle, and to cite a few examples along the way.
Grace Wever worked for Kodak at the time her book was written, and was active with the Great Lake Council of Industries. She later moved to Peat, Marwick.
Wever adds a third level (additional leading indicators). These reflect fuller integration of ES&H into corporate management, and fuller integration with stakeholders and customers. I’m not going to address this third level of performance measurement in any detail. (Although I do think that some DOE sites and contractors are currently doing things that fall within level 3.) But it’s important to be reminded that there are higher levels of excellence to which we can aspire. Transition: Remind ourselves why we are on this search for the ‘holy grail’ of ‘leading indicators’??
Transition: The European Process Safety Center makes this point explicitly
DOE sites certainly include ‘operations where there may be potential for severe accidents’. Transition: The British Health and Safety Executive makes much the same point
This list encompasses a diverse collection of “best practice” approaches. None of these is directly transferable to DOE. But each has useful elements to consider Its not my intent to describe any of them completely. Rather, I will try to highlight some important similarities and differences, and then make some summary observations. References are listed at the end of the presentation, so that you can explore any of these more fully. All of these different “best practice” approaches have a common theme: the need to measure how key elements of the management system are functioning.
The first “best practices” example I want to discuss is an excellent guide produced by the British health and Safety Executive.
A broad definition of performance measurement, and of its purpose.
Health and Safety Executive takes a broad systems approach. For example, they explicitly monitor -- not just identify, but track -- what they call ‘inputs’ that is, the hazards faced or created by the organization’s activities. And they monitor the elements of the H&S management system And they measure ‘outcomes’ or ‘lagging indicators.’
H&SE identifies five key elements of a H&S management system. These look very much like “Plan – Do – Check – Act.” Or like ISO 14001. Or like Integrated Safety Management. Within the ‘process’ part of framework, the performance measurement system must cover each element of this management system.
Measurement, in this approach, is not as easy as collecting outcome (lagging) data. (And I acknowledge that this has its challenges as well.) It’s a qualitatively different, and much more wide-ranging, activity. It takes advantage of a broader range of information, and systematically seeks it out.
Transition: The next “best practice” system I will discuss is the Total Quality Environmental Matrix, developed by the Great Lake Council of Industries.
Transition: In Australia, the government occupational safety and health office is promoting the use of “positive performance indicators”
Note that ‘inputs’ has a different meaning here than in the British system.
Transition: At the ISM meeting in April, Kurt Krueger from GE described GE’s corporate Health and Safety Framework.
Computerized reporting and
CCPS was formed in 1985 by the American Institute of Chemical Engineers to promote the improvement of chemical process safety Also rolls up into score for each element
Under each element, you can click down to detailed questions, definitions, and guidance.
DOE has many pieces in place which could support a system such as this. We don’t consistently use a management system framework to organize and pull all this information together Office of ES&H Evaluations in the Office of Security and Safety Performance Assurance Moving to the new Office of Health, Safety and Security as HS-64)
Office of Independent Evaluations (SP-44) recently published the report of their inspection of the Savannah River Site The Inspector General’s office recently issued a report on the Corrective Action Program at the Office of Civilian Radioactive Waste Management
As an example, the inspection at Savannah River Site (report issued February 2006) Looked at four SRS activities Tritium facilities H-canyon operations H-canyon maintenance and construction SRNL and assessed them against the five ISM core functions Also looked at five elements related to Essential System Functionality Configuration management programs and supporting processes Engineering design and authorization basis Surveillance and testing Maintenance and improvement Operations Office of Independent Oversight rates each as Effective Performance Needs Improvement Significant Weakness Transition: If we look at 17 inspections, over three years, we begin to see a pattern
I tabulated these last year, in connection with the Department’s Compliance Improvement Initiative. This summarizes the ratings given by the Office of Independent Oversight in these four areas in the 17 ES&H inspections from 2002 through 2004.
Many of these will be qualitative measures.
Patty Bubar, in the next room, is describing a proposal for a new set of DOE Corporate Performance Indicators. These have the potential to improve our use of precursor information which we already collect through ORPS and other systems. The broader vision I’m talking about today is not inconsistent with this proposal. This vision generalizes from a variety of “best practices” from around the world to identify a broader conceptual framework, within which we can make use of a wider range of information to assess how well we are really managing risk. It is a framework, or a mindset, which can inform our improvement efforts over the coming years.
Steven R. Woodbury DOE Office of Nuclear Safety and Environmental Policy Leading Indicators for ES&H Performance: Measuring How Well Our Management System Is Performing ISM Best Practices Workshop September 12, 2006
“When safety is good and injury and loss rates are low, then [outcome] measurements are not sufficient to provide adequate feedback for managing safety.
For operations where there may be potential for severe accidents, the likelihood of such events must be extremely low. This means that the absence of very unlikely events is not, of itself, a sufficient indicator of good safety management. ”
“Health and safety differs from many areas measured by managers because success results in the absence of an outcome (injuries or ill health) rather than a presence. But a low injury or ill-health rate, even over a period of years, is no guarantee that risks are being controlled and will not lead to injuries or ill health in the future. This is particularly true in organizations where there is a low probability of accidents but where major hazards are present. ”
– A Guide to Measuring Health & Safety Performance Health & Safety Executive (UK), p 5
Some Best Practices in “Next Level” Performance Measurement
Guide to Measuring Health & Safety Performance (British Health and Safety Executive)
Total Quality Environmental Management Matrix (Grace Wever, Council of Great Lakes Industries)
Positive Performance Indicators (Australia, Department of Employment and Workplace Relations)
General Electric Company
ProSmart software (Chemical Process Safety Council)
“ The primary purpose of measuring H&S performance is to provide information on the progress and current status of the strategies, processes and activities used by an organization to control risks to health and safety”
– A Guide to Measuring Health & Safety Performance Health & Safety Executive (UK), p 7
PPIs are not limited to Occupational Health and Safety – completely applicable to management across ES&H
PPIs focus on identifying key elements of the ES&H management system, and measuring how well these key elements are functioning
PPIs are non-prescriptive and open-ended: approach provides great flexibility at the facility and site level
Most PPIs are qualitative, based on assessments, record reviews, surveys, or audits
The Guidance document assumes a fairly basic management system. DOE sites generally have more sophisticated ES&H management systems in place, with the opportunity to develop more sophisticated measures.
General Electric – Health and Safety Framework
At the April ISM meeting in Albuquerque, Kurt Krueger described GE’s corporate-wide Health and Safety program
“ One standard, one program, one set of metrics for every GE facility around the world – No exceptions!”
GE’s Health and Safety Framework includes a mix of 21
ES&H Evaluations of 17 DOE Sites 2002-2004 15 2 Feedback and Improvement 5 12 Work within Controls 1 11 5 Develop and Implement Controls 7 10 Analyze the Hazards Significant Weakness Needs Improvement Effective Performance Management System Area
There is a common thread of best practice, across many different organizations: to move beyond outcome (lagging) indicators, we need to start systematically measuring how well key elements of our management system are performing.
This is not as easy as counting occurrences (injuries, enforcement actions): it must involve walk-throughs, audits, surveys, and other varied approaches.
This ‘next level’ of leading performance measurement complements, and does not replace, outcome (lagging) measures.
Approaches developed for process safety, worker safety, environment, can be applied across ES&H. We need to be eclectic and ecumenical in our thinking.
Nobody has a turnkey system that’s right for DOE. We need to identify and build on the best of these various approaches.
Grace Wever, 1996, Strategic Environmental Management: Using TQEM and ISO 14000 for Competitive Advantage , John Wiley and Sons, NY
European Process Safety Centre (Jacques van Steen, ed.), 1996, Safety Performance Measurement , Institution of Chemical Engineers, Rugby (UK) (distr by Gulf publishing Co., PO Box 2608, Houston, TX 77252-2608)
Health and Safety Executive (United Kingdom), 2001, A Guide to Measuring Health & Safety Performance http:// www.hse.gov.uk/opsunit/perfmeas.pdf
Commonwealth of Australia (Department of Employment and Workplace Relations), 2005, Guidance on the Use of Positive Performance Indicators to Improve Workplace Health and Safety http://www.nohsc.gov.au/PDF/Standards/GuidanceNotes/ASCCPPIGuidanceBooklet.pdf
Kurt Krueger, “Creating a Culture of Safety Excellence: The Journey and the Prize,” talk at DOE ISM Champions Workshop, Albuquerque NM, April 2006 http://www.eh.doe.gov/ism/workshops/Creating_a_Culture_of_Safety_Excellence.pdf
George Eckes, The Six Sigma Revolution: How GE and Others Turned Process into Profits , New York: John Wiley and Sons, 2000.
Center for Chemical Process Safety, ProSmart software http://www.aiche.org/CCPS/Publications/Software/ProSmart/index.aspx