This presentation explores the use of data to evaluate ergonomic risk factors and how PG&E collected this data to help create an algorithm that accurately predicts the risk of ergonomic discomfort.
7. Typical Methods to Identify Risk
In-Person
Assessment
Self
Assessment
Remote
Assessment
Combination of
Methods
8. Typical Methods to Identify Risk
In-Person
Assessment
Self
Assessment
Remote
Assessment
Combination of
Methods
Equals
Employee
Risk Status
9. “Before the computer age, progress in
science was achieved mainly by: gathering
empirical data and crafting [hypotheses to
explain] our observations...
This theory-observation-refine (TOR) cycle
has provided many of our most profound
insights into how the universe works.
It has not worked so well, however, for
developing our understanding of complex
systems.”
Christoph Adami, Professor of Microbiology &
Molecular Genetics, Michigan State
Are there Better Ways to Predict Risk?
10. PG&E Study
• Step 1: Collected as much risk factor data as practical, using an
epidemiological study model, with premise that we don’t know
which factors influence risk or why
• Step 2: Using predictive analysis tools (a la Netflix) to consider each
factor separately and in combination with others to see where
factor(s) predict risk
• Step 3: Using these results to create an algorithm that accurately
predicts risk of discomfort and time-to-onset of discomfort
What We Learned
• Factors with predictive value aren’t necessarily intuitive
• We can use predictive analysis to quantitatively guide the degree to
which an ergonomics program should consider different factors
Understanding Optimal Risk Factors
11. Initial Findings –
Example 1
• Disc = Discomfort
• OR = Odds Ratio
Key Take-away:
An employee’s perception of how
often they take breaks is a
significant predictor of risk of injury
12. Initial Findings –
Example 2
• Disc = Discomfort
• OR = Odds Ratio
Key Take-aways:
• Not all questions are valuable risk
predictors (surprisingly, this one wasn’t)
• Since this is a survey question, this
doesn’t mean that external devices aren’t
important (could be the question, or
inaccurate reporting)
13. Initial Findings –
Unexpected Predictors
What question would
you imagine results
in this discomfort
distribution?
• Disc = Discomfort
• OR = Optimal Risk
Key Take-away:
• It’s important to consider all factors
without bias as to which will be the
strongest risk predictors
14. Future Use of Predictive Data
• Shorten assessments to focus on
questions with significant predictive value
• When possible, use automatically collected
data to predict risk (timeliness, easier on
employee)
• Focus on interventions that are shown to
reduce discomfort incidence
• Look at multiple factors (e.g. notebooks +
exposure hours)
• Rely on software solutions to automatically
take measures to reduce detected risks
15. Our Preliminary Learnings
Collect as much ergonomic data as possible
before making any assumptions about what
factors cause risk
What your data reveals may surprise you!
After AEC Conference organizers introduce Ron and Arnie, Arnie explains the nature of this field study that PG&E has been conducting with Remedy Interactive using desktop assessment software. And that they are going to review the correlations between what they found in their self assessment results and perceived discomfort reporting..
[Arnie’s slide]As ergonomists, we have the primary goals of keeping employees comfortable and preventing injuries. To achieve this, we typically focus on identifying risk factors that cause discomfort and then use a range of methods to continually decrease those risks. This preso is focused on the identification process..
[Arnie’s slide]The methods we use to identify risk involve collecting information, and then comparing the results to research to predict where risk exists. [Audience interaction question] Are there methods not listed here? YES: For example, automated data collection using software/hardware. Here’s what PG&E does/uses.Using this data we can direct limited resources where they’ll be of most value [examples? How does PG&E use risk data like OES results, break compliance, exposure hours?]
[Arnie’s slide]The methods we use to identify risk involve collecting information, and then comparing the results to research to predict where risk exists. [Audience interaction question] Are there methods not listed here? YES: For example, automated data collection using software/hardware. Here’s what PG&E does/uses.Using this data we can direct limited resources where they’ll be of most value [examples? How does PG&E use risk data like OES results, break compliance, exposure hours?]
[Arnie’s slide]The methods we use to identify risk involve collecting information, and then comparing the results to research to predict where risk exists. [Audience interaction question] Are there methods not listed here? YES: For example, automated data collection using software/hardware. Here’s what PG&E does/uses.Using this data we can direct limited resources where they’ll be of most value [examples? How does PG&E use risk data like OES results, break compliance, exposure hours?]
[Arnie’s slide]The methods we use to identify risk involve collecting information, and then comparing the results to research to predict where risk exists. [Audience interaction question] Are there methods not listed here? YES: For example, automated data collection using software/hardware. Here’s what PG&E does/uses.Using this data we can direct limited resources where they’ll be of most value [examples? How does PG&E use risk data like OES results, break compliance, exposure hours?]
[Arnie’s slide]The methods we use to identify risk involve collecting information, and then comparing the results to research to predict where risk exists. [Audience interaction question] Are there methods not listed here? YES: For example, automated data collection using software/hardware. Here’s what PG&E does/uses.Using this data we can direct limited resources where they’ll be of most value [examples? How does PG&E use risk data like OES results, break compliance, exposure hours?]
[Arnie’s slide]The methods we use to identify risk involve collecting information, and then comparing the results to research to predict where risk exists. [Audience interaction question] Are there methods not listed here? YES: For example, automated data collection using software/hardware. Here’s what PG&E does/uses.Using this data we can direct limited resources where they’ll be of most value [examples? How does PG&E use risk data like OES results, break compliance, exposure hours?]
[Arnie says] We realized that we have all this data but what we REALLY needed was a way to use this data better for risk prediction. This all begs the question, are there better ways to predict risk ?[Ron then continues and quotes from Science article.] The method that this article proposes for complex systems like ergonomics is [click] Predictive Analytics.Unlike the traditional and assumptive ways of identifying risk factors, this new approach allows us to use our own data and analyze it in a different way using predictive analytics to identify risk much more precisely.
[Ron’s slide]As Arnie mentioned, we started this field study with PG&E about 2½ years ago to analyze all the data. First, we agreed that we would not make any assumptions about what causes risk. Instead of starting with the factors that we think cause risk and looking for data points to substantiate or refute it, we decided to start with the premise that we don’t actually know what causes risk. So we came up with [how many?] questions for users to answer during an online self-assessment then we used predictive analytics tools within the program to understand which factors ---- like arm height or monitor positioning ---- predicted risk, and what combinations of factors predicted risk. We also looked at how --- or to what degree ---- certain factors contributed to risk of injury without worrying about “which ones” or “why”[After click “What We Learned” shows up] What we learned in this process ---- is that certain kinds of questions have predictive value, and that we can use science to help guide what questions we ask, and what the inputs might be
[Ron’s slide]So for example, [take audience through example, describe data]….The key thing we learned here is that this question is a GREAT predictor of risk. When you realize a question doesn’t have predictive value, you may have to reword it so that you get clearer info. Or you may decide not to continue asking this question, so that you can focus on having employees answer the self assessment questions that DO predict risk behavior.
[Ron’s slide]Now in this next example, we asked users to answer: [Read out question, then pinpoint the few areas where risk occurred based on the answers][Takeaways:] On this example, I have a confession to make – this question doesn’t predict risk well. It might be that the question is confusing, or that people don’t answer this accurately. We don’t know why, but we know we have to either figure it out, or stop asking the question. We have objective data we haven’t analyzed yet.But this helps highlight that unless you examine the data, you may be missing out on an opportunity to come across a great discovery in your risk analysis process.
[Ron’s slide]As you can see here, the results on this last example show a range of issues (highlight issues)[Ask someone in the audience] What do you think the question was?Based on the results we can see, the question may surprise you. The question was….. [read monitor glare question.]What we learned here is that how users answer this question can be a predictor of risk. We don’t need to know “why” this question is a strong risk predictor… it might be attributed to a number of other factors related to monitor glare. But it shows why it’s important to ask a range of questions to determine whether any patterns occur that are worth paying attention to, and to know which questions yield major risk factor data.
[Arnie’s slide]Thanks Ron. While we’re still in the midst of analyzing all this data we’ve collected, there are a few things we’ve learned and plan to do in the future to improve our office ergonomics programs. [Arnie picks and chooses which ones he wants to highlight]
[Ron presents slide; Arnie to add commentary as appropriate]In summary, there are some big learnings coming out of this study that are already improving the way PG&E manages its ergonomics program. You can achieve these same insights if you include a range of questions in your assessments and collect as much ergonomic usage data as possible. It also helps a great deal to use software to collect this data, then some kind of analytics program to make sense of all of it.When you really take a look at your data and all of the correlations, the answers about risks of injury may surprise you!
