Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Metcalf Substation Incident
The Disappearing Elephant
Chuck Petras
Gabriel: Have you ever heard of Harry Houdini? Well he wasn't
like today's magicians who are only interested in television...
The Incident (16 Apr 2013)
• 500kv/230kv substation located south of San Jose
• Unknown perpetrators shot at transformers ...
Source:
http://soq
uelbythecr
eek.blogsp
ot.com/20
14/02/is-
2013-
california-
attack-on-
pg-and-
at.html
The Narrative
"The underground [fiber optic] cables, protected by manhole covers, were cut
shortly before 1:30 a.m. in two...
Timeline
• 12:58 a.m. AT&T fiber-optic telecommunications cables were cut not far from U.S. Highway 101 just outside south...
Source:
http://www.
wsj.com/arti
cles/SB1000
1424052702
3048511045
7935914194
1621778
Source: Google Earth Pro
Damage (Substation)
• 500kV Yard: 10 transformers
• 230kV Yard: 7 transformers
• 115kV Yard: 6 circuit breakers
• Total of...
Damage (Substation)
Sources: https://www.youtube.com/watch?v=RQzAbKdLfW8
http://www.cpuc.ca.gov/NR/rdonlyres/E5190E64-D4EF...
Damage (Telecom vaults)
• AT&T Communications Fiber-Optic Cables Cut
• Six fiber-optic cables cut
• Took approx. 24 hours ...
Damage (Telecom vaults)
Source: http://www.norscan.com/_uploads/images/IMG_2845.jpg
NOTE: Representative (not Metcalf)
Fiber-Optic Network
Source: http://maps.level3.com/default/
Fiber-Optic Network
Source: http://www.zayo.com/network/interactive-map
The Neighborhood
Source: Google Earth Pro
The Neighborhood
Source: Google Earth Pro
The Neighborhood
Source: Google Earth Pro
Upcoming SlideShare
Loading in …5
×

Metcalf substation incident : the disappearing elephant

685 views

Published on

On the morning of the 16th of April 2013 the PG&E Metcalf Transmission Substation in San Jose, Calif. was attacked and temporarily put out of service as were fiber-optic cables owned by AT&T and LEVEL 3 Communications.

See article at https://www.linkedin.com/pulse/metcalf-substation-incident-disappearing-elephant-chuck-petras?published=t

Published in: Engineering
  • Interesting narrative. It would seem the monitoring alarm systems in place were not up to the task of protecting vital services. The type of surveillance in use were probably suitable to protect against meth heads looking for copper or curious kids snooping around. Protecting the nations infrastructure against a coordinated attack requires much more in the way of sophistication and redundancy. Auditory and seismic sensors come to mind. It shouldn't require a manual report of gunfire. Critical facilities should have the ability to sense and alert on the sound and vibrations gun shots or explosions. Redundant systems with backup power with alternative means of communication in the event of a primary failure of the monitored facilities. A major alarm that is communicated on the system being monitored is useless. A security system can be disabled by cutting the line feeding it. A loss of communication alarm will not trigger the same level of response as an active alarm and is inadequate for critical facilities. Terrestrial radio systems operated by backup batteries could provide a system that was independent from the primary telecommunications infrastructure. While the primary system provides tremendous capabilities under normal operating conditions, the primary backbone links are a very attractive and vulnerable point of failure, if targeted in a coordinated assault. Disabling the fiber feeds that provide connectivity to cell towers can cripple one of the main sources of connectivity. Any system that relies entirely on cell communications may have great functionality at the cost of extreme vulneralbility. It is a unfortunate consequence of the success of mobile phones that the terrestrial radio network was financially devasted and subsequently dismantled. Sure, some capabilites are out there but for the most part the mobile transceivers were discarded. While the forces that can bring the infrastructure down, can be the work of nefarious individuals, infinitely worse damage may be the result of natural events. Wide scale power outages due to ice or wind storms can cripple an entire regions. A large ice storm crippled the Puget Sound region, when branches weighed down by ice fell and snapped power lines throughout the region. It knocked power out for days. The backup power systems at cell towers were depleted in a very short amount of time and cellular communcation was rendered usesless. The wide spread loss of electrical power required hundreds of portable generators to be deployed to restore cell communications. It took days for them to arrive and most were never deployed. As the power was gradually restored, the cable breaks could start to be restored. Huge delays. The direct connect option of our old school Nextel phones was greatly missed. Mobile radio systems (cell phones are radios, but are low power and require many repeaters) could operate off of vehicle 12V power. The repeater stations located on high points throughout the area, can provide a backup to the omnipresent cell infrastructure. The resurrection and updating of these facilities could be a critical component of alternative emergency communications in the event of a major loss of the primary systems. Of course, these sites would require a much more robust form of monitoring to keep them secure and operational. Storing large amounts of fuel to keep the generators going in the midst of an extended power outage is critical. Perimeter alarms and sophisticated motion detectors with capabilities beyond keeping out nosy kids and hikers. Redundant systems are often budgeted into the future and get cut ruthlessly. You don't need them till you need them. Protecting our nation is beyond hand to hand fighting. The infrastructure can bring us to our knees in a hurry. Intensive, redundant surveillance monitoring of key infrastructure components are critical and mandatory. The slow responses detailed in this incident are alarming, but not surprising. The telecommunication POPs in large cities are another point of vulnerability. A well placed unit of destruction, at a critical point in a major POP (Point of Presence) would wreak havoc. Perhaps a parallel system for priority emergency commmunications could be co-located at cell sites, so the emergency personnel could continue to operate. A parallel communication channel with a separate backup power system could allow extra runtime for the emergency responders. Private enterprise can cut backup maintenance to reduce expenditures. We the people, should not be left vulnerable due to maintenance budget cuts. While point to point microwave systems are in decline due to the increasing proliferation of fiber, they are not as easily subjected to total failure by an individual armed with bolt cutters or an axe. Enough rambling....thanks for sharing the story and hopefully it increases public awareness of the vulnerability of the power and communications grid.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Metcalf substation incident : the disappearing elephant

  1. 1. Metcalf Substation Incident The Disappearing Elephant Chuck Petras
  2. 2. Gabriel: Have you ever heard of Harry Houdini? Well he wasn't like today's magicians who are only interested in television ratings. He was an artist. He could make an elephant disappear in the middle of a theater filled with people, and do you know how he did that? Misdirection. Stanley: What the f*** are you talking about? Gabriel: Misdirection. What the eyes see and the ears hear, the mind believes. Source: http://www.imdb.com/title/tt0244244/quotes?item=qt0320356
  3. 3. The Incident (16 Apr 2013) • 500kv/230kv substation located south of San Jose • Unknown perpetrators shot at transformers and breakers • 116 impact points on 22 pieces of equipment • Lost 52,000 gallons of transformer oil • 10 of 11 transformers were struck • Unusually well informed attackers • Two fiber lines cut before the attack • Telecom vaults were resealed, garbage spread to draw attention away • Left the scene minutes before law enforce arrived • Targeted only the ‘hot’ transformers (one was down for maintenance) • Nearby Generating station was on outage Source: https://midwestreliability.org/MRODocuments/NERC%20CIPC%20Report%20to%20the%20Board%2006.27.2013.pdf
  4. 4. Source: http://soq uelbythecr eek.blogsp ot.com/20 14/02/is- 2013- california- attack-on- pg-and- at.html
  5. 5. The Narrative "The underground [fiber optic] cables, protected by manhole covers, were cut shortly before 1:30 a.m. in two locations along Monterey Highway, Smith said. Some of the cables, comprised of bundled fiber optic cords, were cut near Monterey Highway and Coyote Ranch Road, AT&T spokesman George Ross said. ... A short time after the [L3 Communications fiber optic] cables were cut, around 1:45 a.m., the sheriff’s office and San Jose police received reports of gunfire in the area of Monterey Highway and Blanchard Road, sheriff’s spokesman Deputy Kurtis Stenderup said. ... Two hours later, PG&E contacted the sheriff’s office to report a problem at the utility’s nearby substation in the 100 block of Metcalf Road. ... which is located near both a public gun range and the sheriff’s shooting range. All of the incidents occurred within a half-mile radius of each other, Smith said." Source: http://sanfrancisco.cbslocal.com/2013/04/16/gunshots-cause-oil-spill-at-san-jose-pge-substation/
  6. 6. Timeline • 12:58 a.m. AT&T fiber-optic telecommunications cables were cut not far from U.S. Highway 101 just outside south San Jose. • 1:07 a.m. Some customers of Level 3 Communications, an Internet service provider, lost service. Cables in its vault near the Metcalf substation were also cut. • 1:31 a.m. A surveillance camera pointed along a chain-link fence around the substation recorded a streak of light that investigators from the Santa Clara County Sheriff's office think was a signal from a waved flashlight. It was followed by the muzzle flash of rifles and sparks from bullets hitting the fence. • 1:37 a.m. PG&E confirms received an alarm from motion sensors at the substation, possibly from bullets grazing the fence. • 1:41 a.m. San Jose Sheriff's department received a 911 call about gunfire, sent by an engineer at a nearby power plant that still had phone service. • 1:45 a.m. The first bank of transformers, riddled with bullet holes and having leaked 52,000 gallons of oil, overheated - at which time PG&E's control center about 90 miles north received an equipment-failure alarm. • 1:50 a.m. Another apparent flashlight signal, caught on film, marked the end of the attack. More than 100 shell casings of the sort ejected by AK-47s were later found at the site. • 1:51 a.m. Law-enforcement officers arrived, but found everything quiet. Unable to get past the locked fence and seeing nothing suspicious, they left. • 3:15 a.m. A PG&E worker arrives to survey the damage. Source: http://en.wikipedia.org/wiki/Metcalf_sniper_attack
  7. 7. Source: http://www. wsj.com/arti cles/SB1000 1424052702 3048511045 7935914194 1621778
  8. 8. Source: Google Earth Pro
  9. 9. Damage (Substation) • 500kV Yard: 10 transformers • 230kV Yard: 7 transformers • 115kV Yard: 6 circuit breakers • Total of 52,000 gallons of oil • Mineral oil • $15.4 million in restoration costs • 27 days to restore substation Source: http://www.ksg.harvard.edu/hepg/Papers/2014/Sandoval_HEPGJune2014.pdf
  10. 10. Damage (Substation) Sources: https://www.youtube.com/watch?v=RQzAbKdLfW8 http://www.cpuc.ca.gov/NR/rdonlyres/E5190E64-D4EF-4112-B4FE-341C673E24F7/0/SafteySlidesfromPowerPointforthe22714Meeting3331.pdf
  11. 11. Damage (Telecom vaults) • AT&T Communications Fiber-Optic Cables Cut • Six fiber-optic cables cut • Took approx. 24 hours to return to service (needed to install new cable to work around the affected area) • LEVEL 3 Communications Fiber-Optic Cables Cut • One fiber-optic cable cut(?) • Took approx. 10 hours to return to service Sources: https:///corp.sonic.net/status/2013/04/16/fusionflexlink-outage-in-the-santa-cruz-area ftp://ftp2.cpuc.ca.gov/PG&E20150130ResponseToA1312012Ruling/2014/02/SB_GT&S_0641100.pdf
  12. 12. Damage (Telecom vaults) Source: http://www.norscan.com/_uploads/images/IMG_2845.jpg NOTE: Representative (not Metcalf)
  13. 13. Fiber-Optic Network Source: http://maps.level3.com/default/
  14. 14. Fiber-Optic Network Source: http://www.zayo.com/network/interactive-map
  15. 15. The Neighborhood Source: Google Earth Pro
  16. 16. The Neighborhood Source: Google Earth Pro
  17. 17. The Neighborhood Source: Google Earth Pro

×