Good morning. Due to the key NOHC metric requiring completion within 20 minutes my presentation today is only an overview of the topic of metrics design and their use in supporting industrial hygiene programs within a larger business organization. There has been a good deal of activity in recent years related to the topic of IH metrics in the form of conference courses, PDC’s and AIHA work and publications. This short survey of the topic is a more experience based rather than rigorous academic approach.
So, if you have been a free range IH for years, doing whatever you think is important, what is the point of adopting IH metrics? Point 1 --- With no key IH/OH metrics, program value and accomplishments are invisible outside your functional group. Point 2 - Well designed key metrics continuously advertise IH as key to achieving outcomes or goals of value to the larger organization. Point 3 - When corporate leaders review sites based on a basket of performance measures, those things in the basket have a better chance for local resource allocation than those issues not included. … IH needs to get in the basket!
1) Well chosen metrics support a continuous improvement cycle, helping sustain improvement year over year … By providing measures of performance and progress ….an d 3) Feeding the next improvement planning cycle
Typical HSE Metrics… Are legacies…who’s use may have become more ritual than value adding Lack a significant and useful IH element Are lagging not leading indicators of performance Are not “smart” metrics..... These last two assertions will be made clearer to those not familiar with the terms.
Lagging metrics or indicators - Are Measures of past developments and effects/results , e.g. reflects history or outcomes of actions and processes …. Examples include: Occupational illnesses case rate Samples or risk assessments completed Percent of JSAs including health hazards Rate of hearing exams with threshold shifts Number of IH’s employed
Leading metrics or indicators - Are measures of future developments and drivers / causes: Number of personnel signed up for future training Number of new exposure assessments planned or committed to. IH surveys scheduled in coming Qtr /year Engineering controls scheduled for completion
What are some marks of a poor metric? Metrics that: Produce a glut of information and activity not linked to strategic goals Drive resources into non-strategic activity Fail to promote systematic review and reduction of risks across all jobs
Now, just what are “smart” metrics? Smart metrics are simply proof measures of SMART goals. These are goals that are: Strategic, Measureable, Actionable, Relevant, and Time based: Smart metrics will measure & drive progress towards core IH objectives and goals over set time periods
Smart IH objectives and goals align with: Company IH & Occ. Health high level standards Company IH & Occ. Health strategic initiatives. Business strategic objectives So…we can say smart metrics measure & drive progress towards core IH objectives and goals over set time periods
Some basic considerations in design of Key metrics or performance indicators are: Universality across all business units. Don’t set up an “apples and oranges comparison” situation Defined ownership – Who is responsible and for what. KPIs need instructions or policies for implementation. Analyzable – A typical metric tells you what is happening.. can analysis tell you why? Or yield any other useful insights? Actionable - After being analyzed a KPI should be able to lead to the decisions and actions to improve or sustain the performance on the given KPI Optimal in number – High level KPIs should be linked to your top few root objectives. They should have little or no overlap. Collecting and analyzing data for KPI reporting generates additional work so that performance benefits diminish rapidly as you increase the number of KPI reported. Simplicity aligned to level of use - Design KPI complexity for your audience. The higher up the managerial structure it is targeted the more elegantly simple, and familiar in design I would recommend you make it.
For good or bad, metrics are apt to be permanent or at least difficult to change once established as part of an organizations’ culture; for example LWCIR or TRR Make sure your metrics are structured to address fundamental factors immune to time. And, if you want metrics to support resource allocations…….
..Make sure the whole picture the metrics send is internally consistent with your intent. poor example: - performed 1,000 repeat hexane tests YTD of which 99.5% were < detect level, versus; YTD, Completed 112 new/updated job/task risk assessments, corrected 4 uncontrolled exposures
I have a few specific examples addressing core IH concerns… Anticipation: MOCs and project planning include IH reviews Recognition % of job tasks with current QEAs Evaluation Risk based priority sampling completion….AND:
Control…examples include: Unprotected Overexposure Incidents Driving Down Risks Across All Job tasks Driving Exposure Controls Implementation
IH metrics can drive improvement in respirator use during all potential exposures including release response.
IH metrics can integrate with exposure assessment and control programs, to encourage risk reduction measure implementation. This performance indicator encourages business units to drive risk rankings down to avoid reporting poorly controlled high priority risks.
Leading / Lagging metrics combinations can be used to drive and track qualitative assessment driven sampling for quantitative assessments.
A combination Leading / lagging metric can be designed to encourage and track risk reduction controls efforts.
For a deeper understanding and other viewpoints I suggest the following resources, and would be happy to send requestors a copy of the poster that was presented at AIHCE 2009, presenting ConocoPhillips Co’s journey to IH metrics design and adoption.
Using Key Metrics to Support Industrial Hygiene Dale Rhodes MSPH, CIH NOHCE 2009 1
IH Metrics, What’s the Point?Visibility of IH program andaccomplishmentsValue advertising Consideration in resourceallocation 2
IH Metrics, What’s the Point?Well chosen metrics support acontinuous improvement cycle… PLAN ACT DOBy providing measures ofperformance and progress CHECK….andFeeding the next improvementplanning cycle 3
Typical HSE MetricsAre legaciesAre lagging not leading indicators of performanceAre not “smart” metricsLack a significant and useful IH element Business Employee Tot. Rec. Contractor Tot. Rec. Combined Unit Rate Rate Tot. Rec. Rate 2009 2008 Goal 2009 2008 y-e Goal 2009 2008 y-e Goal ytd y-e variance ytd variance ytd variance Company 0.26 0.27 4% 0.46 0.65 29% 0.39 0.52 25% Central US 0.26 0.25 -4% 0.57 1.02 44% 0.48 0.80 40% East US 0.36 0.34 -6% 0.36 0.51 29% 0.36 0.43 16% Europe 0.00 0.20 100% 0.00 0.00 0% 0.00 0.19 100% Western US 0.16 0.17 6% 0.55 0.44 -25% 0.27 0.26 -4% Corporate 0.00 0.36 100% 0.00 0.00 0% 0.00 0.35 100% Project 0.00 0.00 0% 0.32 0.19 -68% 0.28 0.17 -65% Development 5
Leading / Lagging MetricsLagging metrics or indicators -Measures of past developments and effects/results, e.g. reflects history or outcomes of actions and processes – Occupational illnesses case rate – Samples or risk assessments completed – Percent of JSAs including health hazards – Rate of hearing exams with threshold shifts – Number of IH’s employed 6
Leading / Lagging MetricsLeading metrics or indicators -Measures of future developments and drivers / causes: – Number of personnel signed up for future training – Number of new exposure assessments planned or committed to. – IH surveys scheduled in coming Qtr /year – Engineering controls scheduled for completion 7
Poor MetricsProduce a glut ofinformation and activity notlinked to strategic goalsDrive resources into non-strategic activityFail to promote systematicreview and reduction ofrisks across all jobs 8
“Smart” MetricsSmart metrics are simply measures of SMART goals. These Goals are: – Strategic, – Measureable, – Actionable, – Relevant, and – Time based: 9
“SMART” MetricsSmart IH objectives and goalsalign with: • Company IH & Occ. Health high level standards • Company IH & Occ. Health strategic initiatives. • Business strategic objectivesSmart metrics measure & driveprogress towards core IHobjectives and goals over set timeperiods 10
Key Metrics – Other Desireable AttributesUniversalDefined ownershipAnalyzableOptimal in numberSimplicity aligned to level of use 11
Caveats..Make sure the whole message your metrics send is consistent with your core reason for metrics. POOR: - YTD performed 1,000 repeat hexane tests of which 99.5% were < 10% of OEL BETTER: - YTD, Completed 112 new/updated exposure assessments, corrected 4 uncontrolled exposures 13
Core IH Issues to AddressAnticipation:– MOCs and project planning include IH reviewsRecognition– % of job tasks with current QEAsEvaluation– Risk based priority sampling completion 14
Core IH Issues to AddressControl– Unprotected Overexposure Incidents– Driving Down Risks Across All Job tasks– Exposure Controls Implementation 15
Lagging IH Metric: Example Exposure Incident Rates Business Unit Employee Exposure Contractor Total Count Count Incident Rate1 CountEast Coast 0 0 0 0Central 1 0.04 0 1Gulf Coast 6 0.10 1 7Project 0 0 3 3DevelopmentResearch 0 0 0 0West 0 0 1 1Coast/RockiesTotal Company 7 0.02 5 12 1 Calculated from company employee work hours onlyIndustrial Hygiene Exposure IncidentsThere were no employee or contractor unprotected exposureincidents in September. Total incidents are 12 for a rate of 0.02. 16
Lagging IH Metrics:Addressing Basic Risk Reduction 17