2. Assessing
Controller
Workload
2011 AGA Operations Conference
Charles Alday
3. AGA Program Description
• The CRM rule requires that a company
monitor the general level of controller activity
on an annual basis. This presentation will
p
describe a method for assessing tasks and
mental workload.
4. Rule, FAQs, Inspection Protocols
• “monitor the content and volume of general
monitor
activity being directed to and required of each
controller…that will assure controllers have
sufficient time to analyze and react to
incoming alarms.”
g
• PHMSA representatives have remarked that
“general activity” refers to all tasks for which
a controller is responsible.
5. AGA White Papers
• The SCADA system provides alarms, which
alarms
are preset and represent deviations outside
of expected p
p parameters.
• The alarm screens are monitored constantly,
and critical alarms require immediate and
q
continuous actions.
• Non-critical alarms require action as
q
determined by the controller.
6. AGA White Papers
• Review the content and volume of activity
being directed to
b i di t d t each G C t ll t
h Gas Controller to
assure sufficient opportunity to analyze and
react to incoming alarms
• Volume of activity being directed to
Controllers.
Controllers This can include:
– Phone calls
– Reports
– Visitors
– Load forecasting and supply
7. FAQs and Inspection Protocols
• These draft documents contain more
detailed guidance that are not
supposed t be required… BUT????
d to b i d
8. Rule, FAQs, Inspection Protocols
• “General activity” means any activity that is
required of the controller This includes but is
controller. includes,
not limited to, pipeline operations, handling
SCADA alarms, conducting shift change,
, g g ,
greeting and responding to visitors,
administrative tasks, impromptu requests,
telephone calls, f
l h ll faxes, or other activities such as
h i ii h
monitoring weather and news reports, checking
security and video surveillance systems using
systems,
the internet, and interacting with colleagues,
supervisors, and managers.
9. Rule, FAQs, Inspection Protocols
• Process must have a sufficient degree of
formality and documentation.
• Describe the level of activity for each console,
including (in
i l di (i cases of control rooms with multiple
f t l ith lti l
consoles) which console has the most activity
and which has the least
least.
13. Assessing Task Workload
1. Identify controller tasks during normal
operations.
2. Assess the “difficulty” of tasks.
3. Count the number of occurrences, over a
specific amount of time.
a. Use data that is available in your system.
4. Measure the amount of time per task.
5. Project that ti
5 P j t th t time over a month, a quarter, a
th t
year.
14. Assessing Task Workload
• Information Access (how difficult is it to get
information to perform task?)
• Mental Loading (how great is the “mental
workload” to complete the task?)
p )
• Action Loading (how complex are the physical
actions required?)
• Communication L di (h
C i ti Loading (how complex and
l d
demanding are communication requirements?)
• Stress (how great is the psychological stress
due to time demands, abnormal conditions, or
exposure to hazardous conditions?)
15. “Ten Most Common Tasks
Ten Tasks”
1. Line Operations
2. Line Adjustments
3. Log Sheet/Schedule
4. Sampling, Calibrate, Proving,
4 Sampling Calibrate Proving Testing
5. Abnormal Operations
6. Monitoring
7. Phone Calls
8. Communications
9. Administrative Tasks
10. Miscellaneous
16. Monitoring?
• Let me tell you a story
– “These controllers are spending 49% of their time
just sitting there watching screens.”
• “Therefore…”
17. Monitoring?
• “Monitoring for the controllers is a continuous
Monitoring
and cognitively demanding task performed
while also performing mathematical
p g
calculations, paperwork, phone calls, and
deliveries.”
• This does not include the abnormal
conditions to which the controller must
immediately attend.
18. Mental Workload
• Some of you won’t like this method
won t method.
• It’s not purely quantitative.
• It s
It’s subjective!
• It involves asking individual controllers to rate
their workload
workload.
• Then those ratings are analyzed and
compiled quantitatively
quantitatively.
• This is combined with the task workload.
19. Mental Workload
The NASA TLX is a multi-dimensional rating
procedure th t provides an overall workload
d that id ll kl d
score based on a weighted average of ratings
on six subscales:
– Mental demand
– Physical demand
– Temporal demand
– Effort
– Performance
– Frustration level
20. NASA TLX Dimensions
• Mental Demand: The amount of mental
and/or perceptual activity (e.g., thinking,
deciding, calculating, remembering, looking,
g, g, g, g,
searching, etc.) that is required by the task or
tasks.
– Were the tasks easy or demanding, simple or
complex, precise or forgiving?
21. NASA TLX Dimensions
• Physical Demand: The amount of physical
activity that is required by the task or tasks
( g,
(e.g., writing, p
g, punching the calculator buttons,
g ,
walking, moving chairs, moving the mouse,
p
pushing, p
g pulling, turning, controlling,
g g g
activating, etc.).
– Was the task easy or demanding, slack or
strenuous?
22. NASA TLX Dimensions
• Time Demand: The amount of time pressure
due to the rate or pace at which the tasks
happened.
pp
– Was the pace slow and leisurely or rapid and
frantic?
– How pushed for time was the controller?
– Was the controller “under the gun”?
23. NASA TLX Dimensions
• Effort: How hard the controller had to work
(mentally and physically) to accomplish their
level of performance.
p
– Did the controller have to expend a great deal of
effort to complete the task?
24. NASA TLX Dimensions
• Performance: How well the controller
performed the task.
– How satisfied was the controller was with his or
her performance?
25. NASA TLX Dimensions
• Frustration: How insecure or secure
secure,
discouraged or gratified, irritated or content,
stressed or relaxed, and annoyed or
, y
complacent the controller felt.
– Was the controller “on his last nerve”?
– How worried was she?
26. Example of Results
Dimension System 1 System 2
Performance 285 233
Time Demand 254 212
Mental Demand 231 245
Effort
Eff 129 175
1
Frustration 70 104
Physical Demand 35 72
27. Workload Assessment
• Phase 1: The goal is to understand workload in
the normal control and monitoring of the pipeline
and refinery processes. Workload will be
assessed in the actual operating environment of
the control center during all shifts.
• Task analysis will be conducted first in order to
y
get a full list of the controller tasks. The task
analysis would be based on these procedures
and records, assembled and provided b th
d d bl d d id d by the
Company
28. Workload Assessment
• Phase 2: The goal will be to determine the five
most common tasks from Phase 1 and introduce
abnormal and emergency events into the system
and measure controller reaction time. This will
be conducted in a controlled and planned
environment of the training simulator where
infrequent abnormal events can be programmed
to occur. If a simulator is not available, tabletop
scenarios will b d
i ill be developed f
l d from abnormal and
b l d
emergency procedures.
29. Workload Assessment
• Another consideration that affects workload is
the workspace and environment and the number
of hours worked. The method for this study
would include an assessment with appropriate
evaluation methods of environmental,
ergonomic,
ergonomic and spatial factors These factors
factors.
affect controller workload, situation awareness,
vigilance, and attention.
g
30. AGA Program Description
• The CRM rule requires that a company
monitor the general level of controller activity
on an annual basis. This presentation will
p
describe a method for assessing tasks and
mental workload.
