Downloaded 10 times
Uploaded byNSW Environment and Planning
Irony of automation
This presentation examines how the automation of industry systems, whether by computers or robots, can have very unexpected, comical or even destructive effects.
Irony of automation
- 1. The Ironies of Automation DrSean Brady Brady Heywood Pty Ltd
- 2.
- 3.
- 7.
- 10.
- 11.
- 12.
- 14.
- 18. Emergency feedwater pumps activated PORVopens Reactor Scrams
- 20. Pressure in reactordropping PORV light indicates closure High pressure injection activated Temp is climbing
- 21.
- 22.
- 23. Reactor coolant pump Pressurein reactor dropping
- 25. Reactor coolant pump Pressurein reactor dropping
- 26. Reactor coolant pump Pressurein reactor dropping
- 27. This is UnitTwo. We are declaring a site emergency. This is not a drill.
- 31. This is UnitTwo. We are declaring a site emergency. This is not a drill. 30 Minutes
- 32.
- 33.
- 36.
- 37.
- 38.
- 39.
- 40. Emergency feedwater pumps activated 4am, 28 March 1979 – Water leak Blocked valves – no flow
- 42. Emergency feedwater pumps activated 4am, 28 March 1979 – Water leak Blocked valves – no flow Temp & Pres rise in core Reactor Scrams
- 43.
- 44.
- 45.
- 46.
- 47.
- 48.
- 49.
- 50.
- 51.
- 52.
- 53.
- 54.
- 55.
- 56.
- 58. PORV stuck open PORVindicator light shows its closed
- 59.
- 65.
- 70. Pressure in reactordropping High pressure injection activated Temp is climbing PORV stuck open PORV indicator light shows its closed
- 71. High pressure injection activated Pressurizer PORVstuck open PORV indicator light shows its closed
- 72. High pressure injection activated Pressurizer PORVstuck open PORV indicator light shows its closed It’s not going ‘solid’
- 73. High pressure injection activated Pressurizer PORVstuck open PORV indicator light shows its closed
- 74.
- 76. PORV remains open121,000 litres (1/3 reactor capacity) is streaming out
- 78. PORV had beenopen for 2 hours and twenty minutes Block Valve
- 88. Thank you Sean Brady BradyHeywood sbrady@bradyheywood.com.au www.bradyheywood.com.au Listen to the Brady Heywood Podcast on iTunes
Editor's Notes
- #4 Shift Supervisor Bill Zewe, 33 years old, ex nuclear navy, plant running “Hot, Straight, and Normal”. Been with Met Ed 6 years. Zewe is clean-shaven – he looks like an airline pilot Fred Scheimann. Shift Forman William Zewe, Shift supervisor Edward Frederick, Control room operator Craig Faust, Control room operator
- #5 Zewe runs both units.
- #6 Harrisburg is 155 mile from New York City and 106 miles from Philadelphia
- #7 Over 600,000 people in area surrounding this plant in a 20 mile radius
- #8 William Zewe, Shift supervisor Edward Frederick, Control room operator Craig Faust, Control room operator
- #9 The fuel rods—36,816 of them—contain enriched uranium in little pills, all stacked within a thin liner, like the cigarette paper around tobacco, only about 12 feet long. Over 3 m tall 150 tonnes of uranium
- #12 Water in the Core is under pressure. The secondary loop is not.
- #14 4 am, 28 March 1979 the control rooms at 3 mile Island are immense. This room has a 90 foot wall of gauges and lights (174) These are a top team: Many of them were ex navy. They were all trained in simulators at Babcock & Wilox and their test scores were near the top. Faust, Schiemann, Frederick, and Zewe have over 25 years of Navy training Zewe is an office at rear of control room eating a sandwich and looking at stacks of paperwork. He’s watching the control room (175) Zewe went on shift at 11 PM (172) at the centre console Ed Frederick is on the edge of his chair. He is looking at 1100 dials gauges and switch indicators. This place was designed for normal operation. (177). Eight floors up, Ed Frederick is rolling a swivel chair along the console to adjust the cooling system when he notices something on Faust’s face. Faust says ‘something is happening ‘. High-pitched warble shatters the silence, then every alarm on panel 15 comes on. Bill Zewe walks out of the office and says “turbine trip. We lost a reactor”. Frederick and Faust are running the controls, Zewe is standing back and watching. (186)
- #15 Fred Scheimann. Shift Foreman William Zewe, Shift supervisor Edward Frederick, Control room operator Craig Faust, Control room operator
- #16 Eight floors up, Ed Frederick is rolling a swivel chair along the console to adjust the cooling system when he notices something on Faust’s face. Faust says ‘something is happening ‘. High-pitched warble shatters the silence, then every alarm on panel 15 comes on. Bill Zewe walks out of the office and says “turbine trip. We lost a reactor”. Frederick and Faust are running the controls, Zewe is standing back and watching. (186)
- #17 At 3:58 am March 28, 1979. Fred Scheimann is on level 281, a level that’s usually really noisy, when he hears a sudden silence. He realises the turbine has tripped off. Then a huge slope of water, propelled by steam, sounds like a freight train moving through the pipe. The conduit leaps from its mounts, rips out valve controls, fractures a pump scalding water everywhere. (184)
- #18 Main Safeties open and dump 1 million pounds of steam into the air. It actually sounds like the buildings tearing itself apart.
- #19 Eight floors up, Ed Frederick is rolling a swivel chair along the console to adjust the cooling system when he notices something on Faust’s face. Faust says ‘something is happening ‘. High-pitched warble shatters the silence, then every alarm on panel 15 comes on. Bill Zewe walks out of the office and says “turbine trip. We lost a reactor”. Frederick and Faust are running the controls, Zewe is standing back and watching. (186)
- #20 Moisture got into the air system in the plant – probably from a leak. This moisture interrupted the air pressure which ‘told’ the pumps that something was amiss, when it wasn’t Without pumps, cold water was no longer flowing in the secondary loop, so an automated safety device kicks in an shutsdown or trips the turbine and generator. In order to keep water cycling in the secondary loop the emergency feedwater pumps come on. But pipes are blocked because valves are closed. Operators did not know they are pumping water into a blocked pipe. There were two indicator lights on the console saying these two valves were closed (8 min later they discover the problem). But why would they even look, these values are always open! If there were no indicator light then they would have prob checked it – this happened in plant one and a half years previous. Heat rises in the core. Reactor scrams (check) – automatic. Because heat has risen the pilot-operated relief valve (PORV) opened and vented.
- #21 Fred Scheimann. Shift Forman William Zewe, Shift supervisor Edward Frederick, Control room operator Craig Faust, Control room operator
- #22 The decaying radioactive materials still produce some heat, enough to generate electricity for 18,000 homes. But now cold water is depresurising the reactor. If that happens the water starts to boil. Then you’ll get steam. You don’t want that. Not designed to cope with steam. So they need to keep pumping in water to get the pressure back up. But they are losing ground. Frederick is still losing ground with, the water level is don’t want 60 inches and dropping. He gets the control value for the high-pressure injection. The needle slows as descent. (187) High Pressure injection: It goes in at about 1,000 gallons a minute. 3,800 litres a minute. We are now 2 minutes into the crisis
- #23 At this point the door bursts open and Fred Shearman rushes in. It’s taken three minutes to reach the control room from level 281. He grabs the emergency manuals and looks for the procedures for turbine and react trip. Item by item he shouts out the emergency procedures so that the operators can confirm this what they’ve done. (187) Fred Scheimann. Shift Foreman William Zewe, Shift supervisor Edward Frederick, Control room operator Craig Faust, Control room operator
- #24 by now Ed Frederick is satisfied that the water level is on the rebound. Now they need to manage the water level in the pressuriser. Scheimann showed that the numbers: 180 inches, 190, 200. Frederick knows it’s going up too fast, by the time he gets to the pressuriser panel the level is 300 inches. This cannot let it get solid. Frederick and Scheimann produced a high-pressure injection system but the water still arises. 340 inches, 350. Frederick knows something is really wrong, where is this water coming from. 380, 390. Frederick says “okay, we’re going solid”. Before Craig Faust the boilers are running dry, but how can the pressuriser be full and the boilers are running dry. He checks emergency pumps are running, they are. Then he checks the valves and the valves are closed, he opened the valves. Cold water heats superheated tubes, and the here machine gun like explosions inside the boilers. (188) By now pressure in reactor is coming back up. They’re watching the gauges. Schiemann is shouting out the numbers: “We got 180 inches” “A hundred ninety” “Two hundred” But this is filling up too fast now. By t he time Ed Frederick gets to the pressuerizer panel to look at the numbers its 300 inches. Got to stop it going solid. This is what they’re entire training has told them. So Frederick and Schiemann throttle back on the high pressure injection – reduce the flow into the reactor loop. But level keeps rising. They throttle back on makeup pumps. Still rising. 340 inches 350 inches Frederick feels fear! Where is this water coming from. Why is it still rising? 380 inches 390 inches Frederick says “Ok, we’re going solid.”
- #26 Shift Supervisor Bill Zewe on right control room operator Ed Frederick
- #34 These are a top team: Many of them were ex navy. They were all trained in simulators at Babcock & Wilox and their test scores were near the top.
- #40 Human Error replaced by machine strengths
- #42 cardiotocography (CTG)
- #55 Then they discover they’ve lost feedwater.” So water in the secondary loop is not circulating, which means that no water to take heat from core. They verify emergency feed. Water now flowinging again. But this is a rough ride. This water is way cooler, like throwing cold water into a red hot saucepan.
- #78 Data overload – alarms are going off everywhere
- #79 40 or 50 alarms always lit – chronic malfunction in alarm or underlying equipment. He can see 600 alarm lights, 1,100 separate dials, gauges and switches. He doesn’t like the control room, everythings too far apart and there is too much information. Sometimes it takes two operators to adjust something because the meter they need to monitor is too far away from the switch that controls is – sometimes its on the back of a panel. Prob is room is designed for normal operation, not for managing an emergency – e.g., you get a shutdown and 30 alarms can light up – what does that mean? And no rhyme or reason to the alarms – reactor coolant (NB) beside the alarm saying the lift in the turbine building!
- #81 Data overload Compters have got faster but we haven’t Cognitively aligned with operator.
- #82 Edward Frederick complained about the alarms the alarm printer takes four seconds to type alone in the allowance of incoming into a three times that fast. Computer is running 30 minutes behind. (198)
- #83 We are not concerned with data, we are concerned with meaning Meaning is found in contrasts, not isolated data We need better alarms, not more alarms
- #84 Automation needs to behave in a way to make the human smarter We are not concerned with data, we are concerned with meaning Meaning is found in contrasts, not isolated data We need better alarms, not more alarms
- #85 We are not concerned with data, we are concerned with meaning Meaning is found in contrasts, not isolated data We need better alarms, not more alarms
- #86 Interestingly we create out own keyholes like these drs do Figure 12.1 H ow practitioners cope with complexity in computerized devices. This figure illustrates a portion of the menu space for a computerized patient-monitoring information system. The highlighted areas are the items actually used by practitioners during observations of device use in cardiac surgery over three months. Note that the space of possibilities is very large compared with the portion practitioners actually use. (From Cook and Woods, 1994) Johannesen, Leila, Dr, et al. Behind Human Error, Ashgate Publishing Ltd, 2010. ProQuest Ebook
- #94 The problem of meaning When users problem solve in a system, they are looking for the structure of the problem. Only when they can see the structure can they effectively solve it. Dekker (2014) points out that the status of a single component of a system “may not be that interesting for an operator. In fact, it may be highly confusing. Rather, the operator must see, through a forest of seemingly disconnected failures, the structure of the problem so that a solution or countermeasure becomes evident.”
- #95 The problem of meaning When users problem solve in a system, they are looking for the structure of the problem. Only when they can see the structure can they effectively solve it. Dekker (2014) points out that the status of a single component of a system “may not be that interesting for an operator. In fact, it may be highly confusing. Rather, the operator must see, through a forest of seemingly disconnected failures, the structure of the problem so that a solution or countermeasure becomes evident.”
- #97 We get a shift change, and they can see the structure of the problem. Mehler receives call from the control room at 5 AM. He drinks coffee puts on close and heads for the plant. When he reaches control rooms he immediately noticed the contradiction of the instruments. The pressure in the coolant loop is going down, while the pressure in the building is going up. He checks the pressuriser relief valve, seems too hot. He leans over the console and says to Scheimann “shut the block filed in the top of the pressuriser”. The reactor pressure bottoms out. It took two hours and 18 minutes to find the problem. (1102)
- #99 Partial meltdown Core 2/3 uncovered
- #100 Partial meltdown Core 2/3 uncovered 45% of the core – 62 tonnes
- #104 Use this for steam dump
- #105 Designers argued that it would have been ok, just as long as the humans had stayed out of it. Similar to Sweden id the computers had stayed out out of it It was difficult to predict
- #110 people adapt their practices and relationships to cope with the new technology, and as a consequence, the new technology gets adapted to fit the new practices and relationships. Users “Practitioners tailor their activities to insulate the larger system from device deficiencies and peculiarities of the technology.”