Pipeline Safety: Control Room Management /Human Factors                                                  “Final Rule December 3, 2009”Pipeliners Club of AtlantaJanuary 11, 2010
Presentation ObjectivesProvide background information on development of the rule.Present elements of “new” control room management rule.Comment on ways to comply with rule, and go beyond compliance.
Background on Rule DevelopmentPipeline and Hazardous Materials Safety Administration has been examining and evaluating and advising on Control Room operations for at least 14 years. (1996)NTSB has been making recommendations about “human factors” for at least 12 years. (1998)Pipeline Safety Improvement Act of 2002Pipeline Inspection, Protection, Enforcement, and Safety Act of 2006 (PIPES Act)
Safety Rec. P-05-001 (NTSB) Recommends that PHMSA:Require operators of hazardous liquids pipelines to follow the API Recommended Practice 1165 for the use of graphics on the SCADA* screens.NTSB had performed a safety study due to leaks going undetected after indications of a leak on the SCADA interface.New rule addresses this recommendation.*SCADA = Supervisory Control and Data Acquisition
Safety Rec. P-05-002 (NTSB)Recommends that PHMSA:Require pipeline companies to have a policy for the review/audit of alarms.New rule addresses this recommendation.
Safety Rec. P-05-003 (NTSB)Recommends that PHMSA:Require controller training to include simulator or non-computerized simulations for controller recognition of abnormal operating conditions, in particular, leak events.New rule addresses this recommendation.
Safety Rec. P-05-004 (NTSB)Recommends that PHMSA:Change the liquid accident reporting form and require operators to provide data related to controller fatigue.This recommendation is being addressed in a change to accident reporting requirements for gas and liquids pipelines.
Safety Rec. P-99-12 (NTSB)Urges PHMSA, to:“establish within two years scientifically based hours of service regulations…that set limits on hours of service,…provide predictable work and rest schedules…and consider circadian rhythms and human sleep and rest requirements.”New rule may or may not address this recommendation, depending on whether you are NTSB or PHMSA.
Safety Rec. P-98-30 (NTSB)Urges PHMSA, to:“assess the potential safety risks associated with rotating pipeline controller shifts and…establish industry guidelines for the development and implementation of pipeline controller work schedules… that reduce the likelihood of accidents attributable to controller fatigue.”Closed May 3, 2006, due to P-99-12 being issued.
Reasons for NTSB BulletinsThe NTSB noted that scientific research has shown that certain sleep factors can affect fatigue and performance, such as insufficient sleep, irregular schedules, and unpredictable schedules.
Reasons for NTSB BulletinsRecommendations resulted from the NTSB’s review of all transportation accidents reported to U.S. Department of Transportation (DOT) modal administrations over a 10 year period. NTSB noted that it had issued over 70 fatigue-related safety recommendations that resulted from its investigations of major accidents, special investigations, or safety studies that identified operator fatigue as a causal factor.
Reasons for NTSB BulletinsWere there any pipeline accidents where Control Room issues could have contributed to the accident?Several examples cited:Colonial Pipeline      1996 (Controller fatigue)Marathon Pipeline    1998 (SCADA & alarms)Olympic Pipeline      1999 (SCADA maintenance)Marathon-Ashland    2000 (Controller fatigue)NTSB study “10 of 18 hazardous liquids accidents have potential Control Room involvement”
Federal Most Wanted Transportation Safety ImprovementsReduce Pipeline Accidents Caused by Human Fatigue
NTSB Most WantedNo Federal hours-of-service regulations exist for controllers of pipeline systemsRegulations exist in other transportation modes.Set working hour limits for pipeline controllers based on fatigue research, circadian rhythms, and sleep and rest requirements. “Acceptable Response – Progressing Slowly”
PHMSA Bulletins “Pipeline Safety:  Countermeasures to Prevent Human Fatigue in the Control Room”  ADB-05-06“Potential Service Disruptions in SCADA Systems”  ADB-03-09“Potential Service Interruptions in Supervisory Control and Data Acquisition Systems”  ADB-99-03
PHMSA & Industry ActivitiesController Certification StudyParticipation with industry associationsAPI 1165 RP SCADA displaysAPI 1167 RP Alarm Mgt.(in progress)API 1168 RP Control Room Mgt.Battelle Human Factors AnalysisAPI Control Room ForumsSGA Gas Control Framework DocumentAGA White Papers
CongressPipeline Safety Improvement Act of 2002Study control room operationsOperator Qualification RequirementsPIPES Act of 2006“Reduce risks associated with human factors, including fatigue, for pipeline controllers and other employees.”NTSB recommendations on display, alarms, and trainingIssue regulations
Notice of Proposed RulemakingPHMSA issued NPRM PHMSA-2007-27954 on September 12, 2008.Received 144 comments, mostly from liquids and gas pipeline industry groups and companies.Majority of comments said “withdraw the proposed rule and start over.”Rule was not withdrawn, but PHMSA did take some comments into consideration before issuing the final rule.
Initial Summary StatementsMust define the roles and responsibilities of controllers…And provide controllers with the necessary information, training, and processes to fulfill these responsibilities.…must also implement methods to prevent controller fatigue
Initial Summary (continued)…manage SCADA alarms…assure control room considerations are taken into account when changing pipeline equipment or configurations……review reportable incidents or accidents to determine whether control room actions contributed to the event.
Effective DatesFinal rule is effective February 1, 2010.Develop control room management procedures by August 1, 2011.Implement the procedures by February 1, 2012.Plenty of time for compliance?Most companies still have questions about what parts of the rule “really mean.”
Definitions in Parts 192 & 195Important for determining if rule applies:AlarmControllerControl RoomSupervisory Control and Data Acquisition System
Definitions in Parts 192 & 195Alarm means an audible or visible means of indicating to the controller that equipment or processes are outside operator-defined, safety-related parameters.Control Room means an operations center staffed by personnel charged with the responsibility for remotely monitoring and controlling a pipeline facility.
Definitions in Parts 192 & 195Controller means a qualified individual who remotely monitors and controls the safety-related operations of a pipeline facility via a SCADA system from a control room, and who has operational authority and accountability for the remote operational functions of the pipeline facility.
Definitions in Parts 192 & 195Supervisory Control and Data Acquisition (SCADA) system means a computer-based system or systems used by a controller in a control room that collects and displays information about a pipeline facility and may have the ability to send commands back to the pipeline facility.
Rule for Parts 192 & 195If you have controller(s) in a control room monitoring and controlling with a SCADA system;Applies to natural gas pipelines, exceptDistribution with less than 250,000 services, orTransmission without a compressor stationMust still implement sections about fatigue, compliance validation, compliance and deviations.Applies to hazardous liquids pipelines.
Control Room Mgt. SectionWritten control room management planDefine controller roles and responsibilities:Normal, abnormal, and emergency operationsAuthority to make decisions and take actionsController’s role during abnormal operating conditionsController’s role during emergency operating conditionsMethod of recording shift-changes and hand-over of responsibility.I would also define roles and responsibilities of other jobs in relation to controllers so that every person understands roles and responsibilities.
“Provide adequate information”Implement API 1165 (partially for gas) whenever SCADA is changed.Conduct point-to-point verification between SCADA displays and field equipment whenever something is changed.Test and verify internal communication plan for manual operation of the pipeline.Test any backup SCADA systems.Implement section 5 of API RP 1168 to establish procedures for when different controller assumes responsibility.Very specific in this section, because it is easier to be specific about SCADA systems & equipment.
“Fatigue mitigation”Establish shift lengths and schedule rotations that…provide time…to achieve 8 hours of continuous sleep.  Work time + commute timeEducate controllers and supervisors in fatigue mitigation strategies and how off-duty activities contribute to fatigue.  Regular, ongoingTrain controllers and supervisors to recognize the effects of fatigue.  Recognize signs and risksEstablish a maximum limit on controller hours of service.   Day, week, dayshift, nightshiftLess specific in “socio-technical” system.
“Alarm Management”Written alarm management planReview SCADA safety-related alarm operations using a process that ensures alarms are accurate…Monthly identification of alarms that are off-scan, inhibited, false, etc. Verify correct safety-related alarm set-point values and alarm descriptions annually.Review alarm management plan annually.Monitor content and volume of general activity directed to controller annually to ensure alarm response.Address deficiencies in any of these items.Specific again
“Change Management”Establish communications between control room representatives, management, and field personnel when planning and implementing changes.   Document your plans and practicesRequire field personnel to contact control room during emergency conditions or when making field changes.   How will you “require?”Seek control room or control room management participation in planning prior to implementation of significant pipeline hydraulic or configuration changes.   What is “significant?”Less specific, due to more “socio-technical” elements
“Operating Experience”Incorporate lessons learned from operating experience into procedures and training.Review incidents that meet reporting requirements for contributions/deficiencies:Controller fatigueField equipmentOperation of any safety deviceProceduresSCADA system configuration & performance.Do you have operating experience system?
“Training”Establish, then review annually and improve training program.Responding to abnormal operating conditions likely to occur.Use of simulator or tabletop method for training controllers to recognize abnormal operating conditions.Working knowledge of pipeline system…Infrequently used setups and procedures.Most companies have a training program.
“Compliance Validation”Submit procedures, when requested, to PHMSA or appropriate State agency.
“Compliance & Deviations”Maintain records for review:Records that demonstrate complianceDocumentation to demonstrate that any deviations from requirements was necessary for the safe operation of a pipeline facility.
SummaryWas this rule necessary?  Can this rule help improve pipeline operations?Incidents and near misses are caused by any and all of the areas in the rule.Most organizations are averse to learning.Most organizations have a change management process, but sometimes do not manage changes well.Is the final rule the “final” rule?Believe SCADA and alarm management are.Do not believe fatigue mitigation will be.
2010 Pipeliners Club Crm Final Rule

2010 Pipeliners Club Crm Final Rule

  • 2.
    Pipeline Safety: ControlRoom Management /Human Factors “Final Rule December 3, 2009”Pipeliners Club of AtlantaJanuary 11, 2010
  • 3.
    Presentation ObjectivesProvide backgroundinformation on development of the rule.Present elements of “new” control room management rule.Comment on ways to comply with rule, and go beyond compliance.
  • 4.
    Background on RuleDevelopmentPipeline and Hazardous Materials Safety Administration has been examining and evaluating and advising on Control Room operations for at least 14 years. (1996)NTSB has been making recommendations about “human factors” for at least 12 years. (1998)Pipeline Safety Improvement Act of 2002Pipeline Inspection, Protection, Enforcement, and Safety Act of 2006 (PIPES Act)
  • 5.
    Safety Rec. P-05-001(NTSB) Recommends that PHMSA:Require operators of hazardous liquids pipelines to follow the API Recommended Practice 1165 for the use of graphics on the SCADA* screens.NTSB had performed a safety study due to leaks going undetected after indications of a leak on the SCADA interface.New rule addresses this recommendation.*SCADA = Supervisory Control and Data Acquisition
  • 6.
    Safety Rec. P-05-002(NTSB)Recommends that PHMSA:Require pipeline companies to have a policy for the review/audit of alarms.New rule addresses this recommendation.
  • 7.
    Safety Rec. P-05-003(NTSB)Recommends that PHMSA:Require controller training to include simulator or non-computerized simulations for controller recognition of abnormal operating conditions, in particular, leak events.New rule addresses this recommendation.
  • 8.
    Safety Rec. P-05-004(NTSB)Recommends that PHMSA:Change the liquid accident reporting form and require operators to provide data related to controller fatigue.This recommendation is being addressed in a change to accident reporting requirements for gas and liquids pipelines.
  • 9.
    Safety Rec. P-99-12(NTSB)Urges PHMSA, to:“establish within two years scientifically based hours of service regulations…that set limits on hours of service,…provide predictable work and rest schedules…and consider circadian rhythms and human sleep and rest requirements.”New rule may or may not address this recommendation, depending on whether you are NTSB or PHMSA.
  • 10.
    Safety Rec. P-98-30(NTSB)Urges PHMSA, to:“assess the potential safety risks associated with rotating pipeline controller shifts and…establish industry guidelines for the development and implementation of pipeline controller work schedules… that reduce the likelihood of accidents attributable to controller fatigue.”Closed May 3, 2006, due to P-99-12 being issued.
  • 11.
    Reasons for NTSBBulletinsThe NTSB noted that scientific research has shown that certain sleep factors can affect fatigue and performance, such as insufficient sleep, irregular schedules, and unpredictable schedules.
  • 12.
    Reasons for NTSBBulletinsRecommendations resulted from the NTSB’s review of all transportation accidents reported to U.S. Department of Transportation (DOT) modal administrations over a 10 year period. NTSB noted that it had issued over 70 fatigue-related safety recommendations that resulted from its investigations of major accidents, special investigations, or safety studies that identified operator fatigue as a causal factor.
  • 13.
    Reasons for NTSBBulletinsWere there any pipeline accidents where Control Room issues could have contributed to the accident?Several examples cited:Colonial Pipeline 1996 (Controller fatigue)Marathon Pipeline 1998 (SCADA & alarms)Olympic Pipeline 1999 (SCADA maintenance)Marathon-Ashland 2000 (Controller fatigue)NTSB study “10 of 18 hazardous liquids accidents have potential Control Room involvement”
  • 14.
    Federal Most WantedTransportation Safety ImprovementsReduce Pipeline Accidents Caused by Human Fatigue
  • 15.
    NTSB Most WantedNoFederal hours-of-service regulations exist for controllers of pipeline systemsRegulations exist in other transportation modes.Set working hour limits for pipeline controllers based on fatigue research, circadian rhythms, and sleep and rest requirements. “Acceptable Response – Progressing Slowly”
  • 16.
    PHMSA Bulletins “PipelineSafety: Countermeasures to Prevent Human Fatigue in the Control Room” ADB-05-06“Potential Service Disruptions in SCADA Systems” ADB-03-09“Potential Service Interruptions in Supervisory Control and Data Acquisition Systems” ADB-99-03
  • 17.
    PHMSA & IndustryActivitiesController Certification StudyParticipation with industry associationsAPI 1165 RP SCADA displaysAPI 1167 RP Alarm Mgt.(in progress)API 1168 RP Control Room Mgt.Battelle Human Factors AnalysisAPI Control Room ForumsSGA Gas Control Framework DocumentAGA White Papers
  • 18.
    CongressPipeline Safety ImprovementAct of 2002Study control room operationsOperator Qualification RequirementsPIPES Act of 2006“Reduce risks associated with human factors, including fatigue, for pipeline controllers and other employees.”NTSB recommendations on display, alarms, and trainingIssue regulations
  • 19.
    Notice of ProposedRulemakingPHMSA issued NPRM PHMSA-2007-27954 on September 12, 2008.Received 144 comments, mostly from liquids and gas pipeline industry groups and companies.Majority of comments said “withdraw the proposed rule and start over.”Rule was not withdrawn, but PHMSA did take some comments into consideration before issuing the final rule.
  • 20.
    Initial Summary StatementsMustdefine the roles and responsibilities of controllers…And provide controllers with the necessary information, training, and processes to fulfill these responsibilities.…must also implement methods to prevent controller fatigue
  • 21.
    Initial Summary (continued)…manageSCADA alarms…assure control room considerations are taken into account when changing pipeline equipment or configurations……review reportable incidents or accidents to determine whether control room actions contributed to the event.
  • 22.
    Effective DatesFinal ruleis effective February 1, 2010.Develop control room management procedures by August 1, 2011.Implement the procedures by February 1, 2012.Plenty of time for compliance?Most companies still have questions about what parts of the rule “really mean.”
  • 23.
    Definitions in Parts192 & 195Important for determining if rule applies:AlarmControllerControl RoomSupervisory Control and Data Acquisition System
  • 24.
    Definitions in Parts192 & 195Alarm means an audible or visible means of indicating to the controller that equipment or processes are outside operator-defined, safety-related parameters.Control Room means an operations center staffed by personnel charged with the responsibility for remotely monitoring and controlling a pipeline facility.
  • 25.
    Definitions in Parts192 & 195Controller means a qualified individual who remotely monitors and controls the safety-related operations of a pipeline facility via a SCADA system from a control room, and who has operational authority and accountability for the remote operational functions of the pipeline facility.
  • 26.
    Definitions in Parts192 & 195Supervisory Control and Data Acquisition (SCADA) system means a computer-based system or systems used by a controller in a control room that collects and displays information about a pipeline facility and may have the ability to send commands back to the pipeline facility.
  • 27.
    Rule for Parts192 & 195If you have controller(s) in a control room monitoring and controlling with a SCADA system;Applies to natural gas pipelines, exceptDistribution with less than 250,000 services, orTransmission without a compressor stationMust still implement sections about fatigue, compliance validation, compliance and deviations.Applies to hazardous liquids pipelines.
  • 28.
    Control Room Mgt.SectionWritten control room management planDefine controller roles and responsibilities:Normal, abnormal, and emergency operationsAuthority to make decisions and take actionsController’s role during abnormal operating conditionsController’s role during emergency operating conditionsMethod of recording shift-changes and hand-over of responsibility.I would also define roles and responsibilities of other jobs in relation to controllers so that every person understands roles and responsibilities.
  • 29.
    “Provide adequate information”ImplementAPI 1165 (partially for gas) whenever SCADA is changed.Conduct point-to-point verification between SCADA displays and field equipment whenever something is changed.Test and verify internal communication plan for manual operation of the pipeline.Test any backup SCADA systems.Implement section 5 of API RP 1168 to establish procedures for when different controller assumes responsibility.Very specific in this section, because it is easier to be specific about SCADA systems & equipment.
  • 30.
    “Fatigue mitigation”Establish shiftlengths and schedule rotations that…provide time…to achieve 8 hours of continuous sleep. Work time + commute timeEducate controllers and supervisors in fatigue mitigation strategies and how off-duty activities contribute to fatigue. Regular, ongoingTrain controllers and supervisors to recognize the effects of fatigue. Recognize signs and risksEstablish a maximum limit on controller hours of service. Day, week, dayshift, nightshiftLess specific in “socio-technical” system.
  • 31.
    “Alarm Management”Written alarmmanagement planReview SCADA safety-related alarm operations using a process that ensures alarms are accurate…Monthly identification of alarms that are off-scan, inhibited, false, etc. Verify correct safety-related alarm set-point values and alarm descriptions annually.Review alarm management plan annually.Monitor content and volume of general activity directed to controller annually to ensure alarm response.Address deficiencies in any of these items.Specific again
  • 32.
    “Change Management”Establish communicationsbetween control room representatives, management, and field personnel when planning and implementing changes. Document your plans and practicesRequire field personnel to contact control room during emergency conditions or when making field changes. How will you “require?”Seek control room or control room management participation in planning prior to implementation of significant pipeline hydraulic or configuration changes. What is “significant?”Less specific, due to more “socio-technical” elements
  • 33.
    “Operating Experience”Incorporate lessonslearned from operating experience into procedures and training.Review incidents that meet reporting requirements for contributions/deficiencies:Controller fatigueField equipmentOperation of any safety deviceProceduresSCADA system configuration & performance.Do you have operating experience system?
  • 34.
    “Training”Establish, then reviewannually and improve training program.Responding to abnormal operating conditions likely to occur.Use of simulator or tabletop method for training controllers to recognize abnormal operating conditions.Working knowledge of pipeline system…Infrequently used setups and procedures.Most companies have a training program.
  • 35.
    “Compliance Validation”Submit procedures,when requested, to PHMSA or appropriate State agency.
  • 36.
    “Compliance & Deviations”Maintainrecords for review:Records that demonstrate complianceDocumentation to demonstrate that any deviations from requirements was necessary for the safe operation of a pipeline facility.
  • 37.
    SummaryWas this rulenecessary? Can this rule help improve pipeline operations?Incidents and near misses are caused by any and all of the areas in the rule.Most organizations are averse to learning.Most organizations have a change management process, but sometimes do not manage changes well.Is the final rule the “final” rule?Believe SCADA and alarm management are.Do not believe fatigue mitigation will be.