Dragos & SRP, PI World 2019: Utilizing Operations Data for Enhanced Cyber Threat Detection
•Download as PPTX, PDF•
3 likes•737 views
Adversary groups and activities targeting industrial control systems are on the rise. Security teams are now tasked with defending increasingly complex and critical control systems without interrupting operations. This presentation highlights plans and progress of a large public electric utility to extend threat detection capability using PI system data sets. Integration with a threat detection platform improves situational awareness and adds value in three ways. It first provides confidence for quickly eliminating threat activity as a root cause of operational upsets. The second benefit is improved likelihood of detecting malicious tradecraft targeting control systems. Finally, the integrated approach provides data in support of control system incident response and forensic activities. Video for presentation here: https://youtu.be/Inn6FPaXN1w Learn more www.dragos.com
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Dragos & SRP, PI World 2019: Utilizing Operations Data for Enhanced Cyber Threat Detection
Similar to Dragos & SRP, PI World 2019: Utilizing Operations Data for Enhanced Cyber Threat Detection (20)
More from Dragos, Inc.
More from Dragos, Inc. (20)
Recently uploaded
Recently uploaded (20)
Dragos & SRP, PI World 2019: Utilizing Operations Data for Enhanced Cyber Threat Detection
- 1. #PIWorld ©2019 OSIsoft, LLC Utilizing operations data for enhanced cyber threat detection and response in industrial control systems (ICS). Mark Johnson-Barbier & Dan Gunter 1
- 2. #PIWorld ©2019 OSIsoft, LLC About 2 • Dan Gunter • Principal Threat Analyst • Dragos • @dan_gunter • Mark Johnson-Barbier • Sr. Principal Analyst • Salt River Project • @PulseOut101
- 3. #PIWorld ©2019 OSIsoft, LLC About SRP 3 • Founded 1903 (10 Years before AZ statehood): First multipurpose project under the National Reclamation act of 1902 • 5089 employees • 1,041,342 customers • 2,900 sq mile service area • 375 sq mile water service area • 13,000 sq mile watershed • Salt River Valley Water Users’ Association • 10 member board and 30 member council – elected by landowners • Canals largely follow 500 miles of ditches built 400-1450AD by the Hohokam • 2018 Water delivery: 773,527 acre-feet • 8 dams and lakes • Salt River Project Agricultural Improvement and Power District • 14 member board and 30 member council – elected by landowners • Generation Owner/Operator: 1 Nuclear, 12 Fossil, 8 hydro plants • Generation: Biomass, Utility Solar, Wind, Geothermal, Rooftop Solar • Transmission & Distribution • Peak Power System: 7,610 MW • Sustainable Portfolio 17.25% of retail requirements
- 4. #PIWorld ©2019 OSIsoft, LLC About SRP 4 • Founded 1903 (10 Years before AZ statehood): First multipurpose project under the National Reclamation act of 1902 • 5089 employees • 1,041,342 customers • 2,900 sq mile service area • 375 sq mile water service area • 13,000 sq mile watershed • Salt River Valley Water Users’ Association • 10 member board and 30 member council – elected by landowners • Canals largely follow 500 miles of ditches built 400-1450AD by the Hohokam • 2018 Water delivery: 773,527 acre-feet • 8 dams and lakes • Salt River Project Agricultural Improvement and Power District • 14 member board and 30 member council – elected by landowners • Generation Owner/Operator: 1 Nuclear, 12 Fossil, 8 hydro plants • Generation: Biomass, Utility Solar, Wind, Geothermal, Rooftop Solar • Transmission & Distribution • Peak Power System: 7,610 MW • Sustainable Portfolio 17.25% of retail requirements = .05 Texas
- 5. #PIWorld ©2019 OSIsoft, LLC About SRP 5 • Founded 1903 (10 Years before AZ statehood): First multipurpose project under the National Reclamation act of 1902 • 5089 employees • 1,041,342 customers • 2,900 sq mile service area • 375 sq mile water service area • 13,000 sq mile watershed • Salt River Valley Water Users’ Association • 10 member board and 30 member council – elected by landowners • Canals largely follow 500 miles of ditches built 400-1450AD by the Hohokam • 2018 Water delivery: 773,527 acre-feet • 8 dams and lakes • Salt River Project Agricultural Improvement and Power District • 14 member board and 30 member council – elected by landowners • Generation Owner/Operator: 1 Nuclear, 12 Fossil, 8 hydro plants • Generation: Biomass, Utility Solar, Wind, Geothermal, Rooftop Solar • Transmission & Distribution • Peak Power System: 7,610 MW • Sustainable Portfolio 17.25% of retail requirements = .021043125 QLD
- 6. #PIWorld ©2019 OSIsoft, LLC About Dragos 6
- 7. #PIWorld ©2019 OSIsoft, LLC • 3 Business/Security Challenges • Integration of PI System and Threat Detection assists with these challenges • SRP Test implementation (Proof of Concept) • Solution, Plans, Ideas for future use cases Agenda 7
- 8. #PIWorld ©2019 OSIsoft, LLC 3 Business/Security Challenges 8
- 9. #PIWorld ©2019 OSIsoft, LLC RESULTSCHALLENGE SOLUTION SRP 1. Eliminate threat activity as direct cause of operational outages 2. Improve detection of adversary tradecraft targeting OT 3. Provide data supporting fast & accurate Incident Response Integrate PI data with the Threat detection platform • PI Event Frames notify on specific events • Dragos Platform correlates PI data with network and endpoint data Enhance Cyber Threat detection with PI data 9
- 10. #PIWorld ©2019 OSIsoft, LLC Business Challenge: Eliminate Threat activity as cause of operational upsets • July 2004 Substation Fire High temps: 111°F/44°C Avg Temps: 101°F/38°C • Sep 8 2011 San Diego • July 2018 Transformer bushing 10
- 11. #PIWorld ©2019 OSIsoft, LLC Business Challenge: Preventing Breach 11
- 12. #PIWorld ©2019 OSIsoft, LLC Business Challenge: Incident Response 12
- 13. #PIWorld ©2019 OSIsoft, LLC 13 • German Steel Mill • Trisis • Crashoverride (Ukraine 2016) Event • Ukraine 2015 Observation Question Hypothesis Prediction Test Iterate ICS Events
- 14. #PIWorld ©2019 OSIsoft, LLC 2015 Ukraine Attack Summary 225 K Customer Outages 3.5 hr Outage Duration. 135 MW Load impact 100’s Server and Workstation Damage 10’s Field Device Damage 50 Substations Impacted 3Utilities Attacked
- 15. #PIWorld ©2019 OSIsoft, LLC 2016 Ukraine Attack Summary TBD Customer Outages 1.25 hr Outage Duration. 200 MWLoad impact TBD Server and Workstation Damage TBD Field Device Damage 1 Substation(s) Impacted 1Trans Co. Attacked
- 16. #PIWorld ©2019 OSIsoft, LLC How can we prevent, detect, and respond to a cyber attack at SRP? 16 Observation Question Hypothesis Prediction Test Iterate
- 17. #PIWorld ©2019 OSIsoft, LLC Adversary will utilize similar tactics as Electrum during an intrusion and will attempt to open breakers from the EMS system 17 Observation Question Hypothesis Prediction Test Iterate
- 18. #PIWorld ©2019 OSIsoft, LLC 1. If prevention fails, SRP can detect an adversary who opens a breaker by sending DNP3 operate commands from an abnormal source computer 2. SOC analysts will quickly gather data to prove or disprove cyber attack as the cause of disruption 18 Observation Question Hypothesis Predictions Test Iterate
- 19. #PIWorld ©2019 OSIsoft, LLC Test: Prevent, Detect, Respond to adversary using Result Existing corporate controls • Prevent: most, but not all, adversaries • Detect: untargeted att&cks • Respond: Slow Add Threat Focused network monitoring platform • Prevent: adds active defense capability to prevent att&ck techniques • Detect: targeted att&cks • Respond: Med Integrate data from PI system • Respond: Expect Fast but TBD 19 Observation Question Hypothesis Prediction Test Iterate
- 20. #PIWorld ©2019 OSIsoft, LLC Integrating: PI + Dragos 20
- 21. #PIWorld ©2019 OSIsoft, LLC Test: Event Frame on breaker open event 21 Test
- 22. #PIWorld ©2019 OSIsoft, LLC Test: Notification 22 Test
- 23. #PIWorld ©2019 OSIsoft, LLC Test: Query Focus Dataset 23 Test
- 24. #PIWorld ©2019 OSIsoft, LLC Test: Detections 24 Test
- 25. #PIWorld ©2019 OSIsoft, LLC Playbooks 25
- 26. #PIWorld ©2019 OSIsoft, LLC Start Test: Breaker Trips Event Frame Sent 26
- 27. #PIWorld ©2019 OSIsoft, LLC 27
- 28. #PIWorld ©2019 OSIsoft, LLC 28
- 29. #PIWorld ©2019 OSIsoft, LLC 29
- 30. #PIWorld ©2019 OSIsoft, LLC 30
- 31. #PIWorld ©2019 OSIsoft, LLC 31
- 32. #PIWorld ©2019 OSIsoft, LLC Quality of the Assessment (simulated) drives appropriate response actions 32 Defcon 5 Normal Operations – all connections active through firewalls Defcon 4 Add email content filtering Additional web proxy filtering/Only critical web use Reduce remote access to OT zones (vendors or employees) Disable non-critical external access Increase Geo-IP blocking Defcon 3 Strict email sanitization (reduce use, block attachments, TXT only) Limit remote access to corp Disable remote access to OT zones (vendors or employees) Reduce public internet surface area Defcon 2 Unplug all OT network connections Disable all corp remote access Defcon 1 Full internet disconnect Defcon 0 Go home, hug kids, grab bug-out bag
- 33. #PIWorld ©2019 OSIsoft, LLC Quality of the Assessment (simulated) drives appropriate response actions 33
- 34. #PIWorld ©2019 OSIsoft, LLC Solution, Plans, Future 35
- 35. #PIWorld ©2019 OSIsoft, LLC RESULTSCHALLENGE SOLUTION SRP 1. Eliminate threat activity as direct cause of operational outages 2. Improve detection of adversary tradecraft targeting OT 3. Provide data supporting fast & accurate Incident Response Integrate PI data with the Threat detection platform First (small) test has proven that this integration can add value • PI Event Frames notify on specific events • Dragos Platform correlates PI data with network and endpoint data. • Were able to launch an investigation based on operational events • Provided data that allowed the analyst to make better assessments Enhance Cyber Threat detection with PI data 36
- 36. #PIWorld ©2019 OSIsoft, LLC Future Use Cases 37 Safety • Can PI + Dragos identify unauthorized entry into a substation? Contract Compliance • Can PI + Dragos assure outside contractors are operating properly? Cyber Attack • Can PI + Dragos detect abnormal load shedding events from substations, meters, or solar inverters?
- 37. #PIWorld ©2019 OSIsoft, LLC Contact 39 • Dan Gunter • Principal Threat Analyst • Dragos • @dan_gunter • Mark Johnson-Barbier • Sr. Principal Analyst • Salt River Project • @PulseOut101
- 38. #PIWorld ©2019 OSIsoft, LLC Questions? Please wait for the microphone State your name & company Please remember to… Complete Survey! Navigate to this session in mobile agenda for survey DOWNLOAD THE MOBILE APP 40
- 39. #PIWorld ©2019 OSIsoft, LLC 41
Editor's Notes
- It seems like all operational outages are all cyber now… This transformer fire in July 2004 was the first time anyone asked me if it could have been caused by a cyber attack. My answer at the time, was “I don’t know”. Direct cause of the fire ended up being a Blue Huron pooping on an insulator and causing a fault. Gila Power Plant visit We made some changes in the am … testing complete, went to lunch Driving back to plant … black smoke coming from Unit 2 transformer “was it a cyber attack” Photo credit: APS Substation technician
- Talk about adversary capabilities and activity This is a software fight – to scale it we need data We want to use threat into to improve our path forward Value of red teaming
- After a breach Need to be able to quickly identify adversary actions IT security tools don’t do that well in OT If we are going to enhance IR, we need data – just like a picture needs more pixels to enhance IRL. We are looking at PI to provide some of that data. Meme credits: Battlestar Galactica The X-Files
- So as we follow the scientific method in our efforts to solve this business problems, we look to known ICS attacks. German Steel Mill OT team didn’t consider cyber Insurance company RCA discovered cyber intrusion as cause of equipment damage Trisis Vendor contacted 3 times prior to consideration of cyber attack Crashoverride (Ukraine 2016) Event Ukraine 2015
- Give a quick overview of Ukraine attacks – many attendees won’t know details What we are doing is based on the tactics of Explain why this matters Reference: Wall Street Journal: http://www.wsj.com/articles/cyberattacks-raise-alarms-for-u-s-power-grid-1483120708 For nine months, the hackers studied the Ukrainian electric system. When the attack finally happened on December 23, 2015, hackers remotely took control of three of Ukraine’s 30 power distribution utilities within a half-hour. During the attack, the first time that power systems had been blacked out through cyber means, control room engineers sat helplessly as ghostly hands moved cursors across their computer screens, opening circuit breakers at 50 substations and shutting off electricity to about 700,000 people.
- “Shortly before midnight on December 17, someone started disconnecting circuit breakers through remote means until the electrical substation was completely disabled, Mr. Kovalchuk said.” “Mr. Kovalchuk said he believes the latest attack was well planned because the targeted substation is one of the utility’s most automated.” Reference: Kovalchuk said the outage amounted to 200 megawatts of capacity, equivalent to about a fifth of the capital's energy consumption at night.
- After observing adversary instruction into other ICS, I’m left with a simple question – how to prevent, detect, and respond to these attacks.
- Prevention is our first line of defense – we believe we can prevent many adversaries and we are working on being a hard target. However, we need to test all levels of defense including response and to do that we have to assume that our preventive barriers fail Given what we know about ICS adversaries, and assuming that prevention fails I predicted that we will 1. detect some adversary tactics 2. Gather data to respond
- After testing the previous theory against existing controls and against the Dragos threat platform, we realized that we still have room for improvement. Quickly summarize tests of previous efforts: 1 – controls using existing corporate controls was effective but didn’t lower risk enough 2 – adding Threat intel and threat based detections improved detection rates and response 3 – if an event occurs we expect to need faster responses – therefore we are investigating the integration with PI
- So let’s talk about how the PI system was applied to help solve this problem.
- One step for preparing for a direct attack on our EMS would be to prepare alerts and integrations in advance that would allow a SOC analyst to Quickly confirm if a breaker operation is/not the result of a cyber attack Quickly gather necessary data that will help identify the extent of the intrusion and prepare an eviction plan
- In a simulated attack the PI event frame would connect to the Dragos platform to display a notification.
- When a behavior is associated to a threat behavior, the analyst would start investigating the behavior to determine if it is legitimate or not
- Creating an alert is only the first step of preparing for an event such as Ukraine 2015. We must also plan our response and ensure that the SOC analyst has all the data she needs to confirm the attack and provide necessary data to learn the extent of the intrusion. The pre-planned actions are recorded in a playbook. Playbook steps SOC analyst through analysis steps: Confirm communications to breaker is from expected hosts Confirm all data flows to breaker on uses expected ports/services Confirm that the DNP3 operations are only the expected operations Review PI data to observe historical state of breaker If necessary contact EMS to confirm operation was expected Review endpoint information from the source computer
- Playbook Step 1 – soc analyst validates that the source of the dnp3 breaker operate function was the expected source computer. Because of the visual diagram of traffic flows the analyst quickly identifies a source computer that began remote communications with the automated breaker just prior to the event.
- Playbook Step 2 The analyst reviews the content of the communications and identifies new protocols that are unexpected
- The analyst reviews the content of the valid traffic and finds that it uses a different operate command (direct operate) than the normal EMS system uses (select then operate)
- SOC analyst reviews history of breaker as part of investigation and finds that the system should be right at the start of an extended operational window and likely should have remained closed.
- SOC analyst reviews connections to the source computer Are there any unexpected remote connections Review Alerts for the computer – brute force attacks, etc Alert on EWS for interactive logon
- SOC analyst reviews data – assessment is made. Because all 5 steps in the playbook shows some level of anomalous activity, the analyst is able to provide a high confidence assessment. Quality of the assessment and amount of data contributes to a more appropriate decision - without necessary data we might overreact to a squirrel attack or underreact to a cyber adversary
- We also need to consider information sharing – how much of what we’ve gathered needs to be shared with partners, government, other utilities, regulators, and customers.
- Few companies actually monitor their vendor access in a way that would prevent Intel is never going to be perfect