The document discusses several incidents involving software errors that led to accidents, including a dropped cable that killed a dockworker due to inconsistent speed readings by sensors, injuries caused by erratic behavior of elevators and bales from an outdated software patch, and a generator trip caused by an unofficial software change made by a vendor. It emphasizes the importance of software configuration management, requirements gathering, failure mode analysis, and change control to prevent such incidents.

1. Accidents Caused by Software Errors Software Kills! 15th Annual American Industrial Hygiene Association - Rocky Mountain Section (AIHA-RMS), and the American Society of Safety Engineers (ASSE) Colorado Chapter FALL TECHNICAL CONFERENCE September 16th & 17th, 2009 Arvada Center “Environmental Health & Safety Broadening Our Alliances” Don Shafer, CSDP Chief Technology and HSE Officer Athens Group, Inc.

2. A Safety Minute – 17 September 2009 Safety - Dropped object fatality in the Keppel FELS shipyard Since arriving in Singapore, I’ve be mildly shocked at the nonchalance shown around lifting operations. It’s quite common to see crane operations lifting loads over active walkways; walkways that are not taped off and often there’s little notice by the workers using the walkways of the lifting operations occurring above them. This resulted in tragedy yesterday. A basket of scrap cable was being transferred from a rig to the dock. Reportedly, the crane was blowing its horn as per protocol for transfer operations. The load shifted and a chunk of cable dropped, landing on the head of a dockworker underneath. I don’t believe the dockworker was participating in the lifting operation. Even more disturbing – no one on the dock came to his aid, rig personnel ran over an attempted CPR. The ambulance arrived without paramedics, rig personnel accompanied the dockworker to the hospital where he was pronounced dead. Be careful out there, your PPE will protect you from many things, but your awareness will save your life.

3. Presentation Outline Examples of Software Related Incidents Software you can “see” Software you cannot “see” Proven Practices to Reduce Software Risk Life Cycle Recognition Configuration Management FMECA

4. Those old Software Safety Chestnuts!

5. And, some not so old! Air France - What is known about the crash of an Air France airbus on 1 June bears similarities with the little-noticed loss much earlier of two computer-controlled passenger jets. Those two crashes raised questions of whether the pilots or systems were really in control. Airbus said this data showed that the pilots might have received conflicting information about their speed. There was a “divergence in airspeed measurement” by the onboard systems of the Air France aircraft. This is one of the matters being investigated, said Airbus. Data to the onboard computers about air speed came from sensors called pitot tubes, at least one of which was due for replacement. French authorities have suggested that inconsistent air speed readings are not dangerous.

6. Software you can “see” 6

7. Safety Incident: Injured Rig Hands Incident The elevators and bales of an older-model top drive reacted erratically to a rapid and erroneous user command. The vendor had released a software patch to that model to prevent this erratic behavior, but somehow it had not been communicated or installed on that drilling unit. There was little or no initial design and testing of the control software and the software interlock issue was not discovered. Little or no system requirements gathering were done on the control system and no FMECA was done on the control portion of the top drive. There was no consistent management of change treating software as an asset on the MODU between the supplying vendor and the operator. Solution Result Initial requirements definition; FMECA of the control system and software change control protocols would have avoided this incident. The bales swung around and injured two of the rig hands, resulting in reportable LTIs. Estimated Lost Time: 5 days Day Rate: $310,000. Minimum Cost: $1,550,000.

8. Safety Incident: Potentially Deadly Mishap Incident A driller was performing a test with a riser joint suspended 70 feet (21 meters) above the drill floor. Prior to leaving the drill cabin for a Job Risk Analysis meeting with the roughnecks, the driller selected “standby” mode on the drilling chair. While doing so, he inadvertently pressed the keypad button that activates Pipe Handling mode. In this mode, the drill control system sends a pressure monitoring command to the pipe elevator every three minutes. The driller stepped out onto the drill floor and three minutes later the pressure monitoring command was sent to the riser handling equipment which mistook it for an unlock command. Solution Result The riser tool released the joint which fell through the well center into the ocean. The joint fell perfectly through the slips. Neither personal injury nor collateral equipment damage was experienced. Estimated Lost Time: 1.5 days Day Rate: $310,000 Minimum Cost: $465,000 An FMECA of the equipment covering operational states and message flow could have prevented this incident. 3 people 4 people

9. Safety Incident: Dropped Blocks Incident The semisubmersible MODU was in the final stages of pulling the BOP. The BOP was being lifted the last meter to gain clearance for access to the BOP transporter in the moonpool. With the travelling block at the uppermost limit, the Kinetic Energy Management System was ‘tripped’, and the resulting action was not as expected. The anti-bird nesting components were incorrectly installed thus limiting the 1200 psi used to function the service brake to 200psi. There was no operator error and the incident was a result of a disc brake system failure. Solution Result Traveling blocks, complete with riser and suspended BOP, descended approximately 50 meters in an uncontrolled manner, until the Top Drive impacted against the riser gimbal at the rig floor level. Estimated Lost Time: 5 days Day Rate: $477,000 Minimum Cost: $2,385,000 An FMECA of the equipment covering operational states and message flow could have prevented this incident. Regression testing of software upgrades and formal change control should have taken place.

10. Safety Incident: Top Drive Out of Control Incident During the voyage to location, a technician was ‘tweaking’ the zone management parameters on a newbuild. A few minutes later the top drive started rotating by itself. The technician in his zeal to fix one thing had broken another – thereby introducing regression into the system. He was also unable to quickly recover to a previous known state as he wasn’t following software change control protocols. Solution Result Following software change control and testing protocols would have prevented this. The technician and the team had to scramble to correct the issue. Fortunately there was no equipment damaged or personnel injured. Estimated Lost Time: 2 days Day Rate: $380,000 Minimum Cost: $760,000

11. Safety Incident: Generator Trip Incident A vendor arrived onboard a rig after having been officially requested to make changes to the rig’s automation system. While onboard, an unofficial request was made by a system operator regarding the numbering of main engine cooling system valves. The vendor either hadn’t completely understood the request or had been distracted and inadvertently made the change to the wrong valve. Some time later a different operator attempted to give a close command to the valve in preparation for maintenance of the system. Solution Result If formal control procedures had been adopted no unofficial change requests should have been carried out. Closing of the incorrect valve caused a generator trip. Estimated Lost Time: .5 days Day Rate: $310,000 Minimum Cost: $155,000

12. Safety Incident: Control System Reset Kills 4 Incident A control system failure occurred on a large, off-shore construction vessel. Two control units were restarted twice, unsuccessfully. A blinking red lamp on the PLC indicated that a memory reset was required, even though a memory reset had NEVER been requested by control system diagnostics during equipment operations. As soon as the hydraulic power packs started, a loud bang was heard. A quadruple joint of pipe dropped approximately one meter to the welding deck below. A second quadruple joint of pipe in the pipe elevator was released (all clamps opened and the hydraulic safety stop swung away) and fell the full length of the tower, smashing through a crowded access platform to the deck below. The initialization instruction was pre-loaded in PLC EPROM memory and the initialization included instructions to OPEN ALL CLAMPS. Solution Result An FMECA of the equipment covering operational states and message flow could have prevented this incident. Document the impact of resetting control systems during operations. Eight personnel were injured – four fatally. All were located on the access platform and several were thrown overboard by the impact. Estimated Lost Time: 20 days Day Rate: $510K Minimum Cost: $10,200,000

14. Predefined, Fail-set or Fail-safe state?

16. Your IT Network is Safe? IT contractor indicted for sabotaging offshore rig management system, Company had refused to offer him a permanent job, feds say, March 18, 2009: Mario Azar, 28 of Upland, Calif., was charged with illegally accessing and compromising a computer system used by Pacific Energy Resources Ltd. (PER) to monitor offshore platforms in California and Anchorage and to detect oil leaks. The indictment papers allege that Azar's actions affected the "integrity and availability" of the system and resulted in it becoming temporarily unavailable. Though no oil spill or environmental hazard occurred while the system was compromised, Azar's actions caused thousands of dollars in damage, the indictment said.

17. Cyber criminals targeting energy – 15 March 2009 Based on an analysis of more than 240 billion requests for analysis by the corporate users, there was near 600% malware growth between like quarters in 2007 and 2008, and a 300% volume ratio increase from January 2008 through December 2008. A vertical industry analysis of malware growth found the energy and oil sector to rank in the top five targets in all threat categories. But energy and oil leads the pack by a long shot when it comes to one important category: encounters with unique new variants of data theft Trojans. With advances in the technology and sophistication of cyber attacks, malware delivered through the web can be remotely customized and configured once in place, based on the victim’s identity.

18. What do the Authorities Say? How to implement these activities and processes is not prescribed by the recommended practice. The recommended practice is primarily focused on the ‘what to do’. DNV RP D-201 Recommended Practice for Integrated Control Systems

19. How can Software become Safer? Awareness of Development Life Cycle Software Configuration Management (SCM) Failure Mode and Effects Criticality Analysis (FMECA)

20. Athens Group Deliverables Life Cycle Model Design Acceptance Construction Operation Contractual Software Standards Controls & Network Commissioning Vendor Software Process Assessment Integration and Test Startup Support FMECA FMECA FMECA Module Development Unit Test Detailed Design Hardware Path Vendor Management Preliminary Design Software Change Management Hardware Requirements Requirements Validation Design Verification Acceptance Planning Deployment Integrated System Testing Operations Maintenance System Requirements Concept Definition Activity Software Requirements Commissioning Planning Preliminary Design Alarm Management Detailed Design Software Path Factory Acceptance Testing Coding Unit Test Troubleshooting and Remediation Integration and Test

21. Software Change Management

22. Failure Mode and Effects Criticality Analysis (FMECA)

23. In Conclusion You can – and MUST - make Software Safer Awareness of Development Life Cycle Software Configuration Management (SCM) Failure Mode and Effects Criticality Analysis (FMECA)

24. Don Shafer, CSDP Chief Technology Officer 5608 Parkcrest Drive, Suite 200 Austin, Tx 78731 don.shafer@athensgroup.com www.athensgroup.com 512.345.0600 x117

25. References NORA Symposium 2008: Public Market for Ideas and Partnerships - http://www.cdc.gov/niosh/nora/symp08/posters/035.html Fatalities Among Oil and Gas Extraction Workers --- United States, 2003 - 2006 - http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5716a3.htm Therac 25 - http://sunnyday.mit.edu/papers/therac.pdf Air France 2009 - http://www.computerweekly.com/Articles/2009/06/01/236245/air-france-crash-thought-to-be-caused-by-system-failure.htmhttp://www.computerweekly.com/Articles/2009/06/16/236447/air-france-airbus-pitot-sensor-linked-to-two-fatal-crashes.htm 8 Software Related Death Incidents - http://www.baselinemag.com/c/a/Projects-Processes/Eight-Fatal-SoftwareRelated-Accidents/

26. Speaker Bio Don Shafer, CSDP, developed Athens Group's oil and gas practice and leads Athens Group engineers in delivering superior rig software services and oil and gas exploration as well as production and pipeline monitoring systems for clients such as BP, Noble, Transocean, Maersk, ExxonMobil, Conoco Phillips and Shell. Prior to co-founding Athens Group, Don led groups developing and marketing hardware and software products for Motorola, AMD and Crystal Semiconductor. He was responsible for managing a $129 million-a-year PC product group that produced the award-winning audio components for Apple. From the development of low-level software drivers in yet-to-be-released Microsoft operating systems to the selection and monitoring of Taiwan semiconductor fabrication facilities, Don has led key product and process efforts. Don earned a BS degree from the USAF Academy and an MBA from the University of Denver. Treasurer of the IEEE Computer Society Board of Governors, Past Editor-in-Chief of the IEEE Computer Society Press, IEEE Senior Member and software engineering book series author, Shafer is an adjunct professor in the Cockrell School of Engineering at the University of Texas at Austin. An avid writer, Don has contributed to three books, written over 20 published articles, and is co-author of Quality Software Project Management, recently released by Prentice-Hall. He is a contributor to the 2010 edition of the multi-volume Encyclopedia of Software Engineering. His latest patents are in state-based machine control.

28. Offices in Houston and Austin, TX

29. Pioneered Drilling Technology AssuranceSM (DTA) Services

30. 100% Referenceable Customers

31. Over 70% have completed more than one project with us

33. What We Deliver