Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

3.3_Cyber Security R&D for Microgrids_Stamp_EPRI/SNL Microgrid

375 views

Published on

Presentation from the EPRI-Sandia Symposium on Secure and Resilient Microgrids: Cyber Security R&D for Microgrids, presented by Jason Stamp, Sandia National Laboratories, Baltimore, MD, August 29-31, 2016.

Published in: Science
  • Be the first to comment

  • Be the first to like this

3.3_Cyber Security R&D for Microgrids_Stamp_EPRI/SNL Microgrid

  1. 1. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. Cyber  Security  R&D  for  Microgrids     Panel  Session:  Emerging  System  Design     Requirements  –  Security,  Resiliency,  and  Reliability    Jason  Stamp,  Ph.D.   Sandia  NaDonal  Laboratories   1
  2. 2. Sandia’s  Control  System  Security  Research   Mission: To reduce the risk of critical infrastructure disruptions due to cyber attacks on control systems. Provide decision makers with actionable information •  Red Team Assessments •  Field Device Analysis •  PLC monitoring and forensics •  PLC firmware forensics •  ICS network detection for ICS traffic •  Emulytics (SCEPTRE) •  Exercise/Test Bed support Design resilient systems to withstand cyber-attacks •  Research next generation security solutions •  Partner with industry to “push” solutions to market 2
  3. 3. Control  System  Architecture   Human-Machine Interface (HMI) software Status displays Switches and dials User Interfaces Field Devices Programmable Logic Controllers (PLC) Remote Telemetry Units (RTU) Intelligent Electronic Devices Sensors Thermocouples Accelerometers Photoresistors Physical Process Oil & Gas Refining Electrical Distribution and Transmission Manufacturing Actuators Breakers/Switches Motors Valves Supervisory Control and Data Acquisition (SCADA) Distributed Control Systems (EMS/DCS) Data Historians Control System Apps 3
  4. 4. RepresentaDve  ICS  TesDng  Environments   Emuly&cs™/SCEPTRE 4
  5. 5. SCEPTRE  OperaDonal  Overview   §  SCEPTRE  provides  a  cyber-­‐physical  environment  to  show  interacDon  between   cyber-­‐iniDated  events  and  the  physical  world   §  Balances  need  for  M&S  accuracy  against  tesDng  resources   §  Live  system  tesDng:  potenDal  damage  to  the  real  system  and  dangers  to  human  life   §  Test  bed  systems:  Expensive  to  build,  maintain,  configure,  and  operate   §  Labscale  hardware  tesDng  setups:  May  require  the  context  of  a  larger,  networked   system   §  Devices  (simulated,  emulated,  real)  communicate/interact  via  ICS  protocols   §  All  ICS  devices  are  able  to  interact  with  the  process  simulaDon,  providing  both   updates  and  subscribing  to  the  current  state  of  the  simulaDon   §  Overall  simulaDon  is  able  to  bridge  mulDple  infrastructures  into  the  same   experiment  to  show  interdependencies   §  Use  cases:   §  Test  and  evaluaDon   §  Mission  rehearsal   §  Other  analysis:  understand  vulnerabiliDes  and  exploitable  avenues,  idenDfy  criDcal   components  on  the  control  network,  model  infrastructure  interdependencies,  etc.   5
  6. 6. SCEPTRE  Cyber  Security  Analysis  for  ICS   §  Control  systems  devices:  simulated  RTUs,  PLCs,  relays;  emulated  PLCs,  FEPs,  HMI   services;  real  HITL  relays,  PLCs,  RTUs   §  High  fidelity  SCADA  protocols:  ModbusTCP,  DNP3,  IEC61850   §  Process  simulaDon:  industry  standard  so_ware  where  possible,  PowerWorld,   PyPower,  PSSE  for  electricity,  water  treatment,  refining,  oil/gas  pipelines   6
  7. 7. Cyber  Security  Architecture   §  Microgrid  cyber  security  reference   architecture   §  In  addiDon  to  DoD  IA  controls,   addiDonal  rigor  will  be  applied  to   protecDng  data-­‐in-­‐moDon  and  data-­‐ at-­‐rest,  along  with  ensuring  such   addiDonal  rigor  does  not  impede  the   operaDonal  data  exchange   requirements  of  the  SPIDERS   microgrid   §  Defense-­‐in-­‐depth  using:   §  Enclaves   §  FuncDonal  Domains   7 4 V. DESIGN APPROACH AND DEFENSE-IN-DEPTH Best practices for securing ICSs leverage network segmen- tation; for example, see [3], [6], and [7]. In most cases, however, network segmentation is focused on separation of the control system network from other less-trusted networks, such as the enterprise network and the Internet. The concept of network segmentation within the control system network itself is addressed to a minimal degree in a recommended practices document [3] published by the DHS Control System Security Program (CSSP), but the additional complexities of configur- ing and managing such a network often result in this level of defense-in-depth being dismissed. In geographically dispersed control systems and field devices, physical segmentation often inherently exists within ICS command and control networks due to the employment of third-party providers for communi- cation services. This segmentation is not leveraged to enhance security, however, as neither physical nor logical segmentation is currently used as a basis for providing additional defense- in-depth within modern ICS networks. The SNL approach to designing a secure microgrid control system network leverages segmentation to reinforce defense- in-depth practices. The microgrid control system network is segmented into enclaves defined by system functions, physical locations, and security concerns. Enclaves are then grouped to- gether into functional domains that allow actors to collaborate in operational system functions that crosscut enclaves. Data exchange worksheets describe communication between actors within enclaves and functional domains. A. Enclaves An enclave is a collection of computing environments that only by system function, rather than by physical location. For example, consider that all of the actors at Site II are grouped into a single enclave (Enclave 3) based on physical location, whereas the actors at Site I are segregated into two enclaves (Enclave 1 and Enclave 2), which may be based on physical location, system function, security concerns, or a combination of features. Fig. 2. Example segmentation of network into enclaves and functional domains. B. Functional Domains Although some enclaves are defined based on actors that participate in a particular system function, some actors neces- sarily crosscut enclaves that are defined by physical location, functional characteristics, or security concerns. For example, the EMS could interact with external actors at the electrical points of common coupling (PCCs), which could belong to
  8. 8. Cyber  Security  Data  Exchange   §  Process:   §  Designate  actors   §  Describe  data  flows  using  tables   §  Assign  enclaves   §  Develop  funcDonal  domains   §  Design  cyber  security  controls   8 TABLE IV DATA EXCHANGE ATTRIBUTES AND EXAMPLE VALUES. Attribute Description Example Values Exchange Type Type of data exchange to occur monitor, control, report, write Interval How often data exchange occurs e.g. milliseconds, seconds Method How data will be exchanged unicast, multicast, broadcast Priority Relative importance of exchanging the data high, medium, low Latency Tolerance Tolerance to delayed control or delayed data exchange high (delays do not affect operation), medium, low Data Type Type of data to be exchanged voltage, setpoint, status Accuracy Necessary precision/timeliness of data significant digits, time units Volume Amount of data to transferred per exchange e.g. bytes, kilobytes, etc. Reliability Necessity of access to control processes and data critical, important, informative InformationAssurance Confidentiality Importance of preserving restrictions to control processes and information access (based on risk to system operations and/or system security) high, medium, low Integrity Importance of preventing unauthorized changes to control processes or data, including authenticity (based on reliability with respect to operations) high, medium, low Availability Importance of timely and reliable access to control processes and data (based on priority and latency tolerance with respect to operations) high, medium, low influence of actors to a particular enclave, the consequences of both local failures and vulnerabilities are isolated within that enclave. VIII. FIRST EXAMPLE FOR THE REFERENCE ARCHITECTURE The approach to segmenting the microgrid control system network is to first identify system functions with a granularity B. System Functions Consider a basic microgrid function: Connect/Disconnect Microgrid as applied to this system. Islanding of the microgrid when the installation’s distribution system loses power and is one of the key functions of the system’s operation. The power actors typically involved in this system function include: • IEDs at the utility (PCC) used to monitor voltage/current sensors and to control breakers and disconnect switches, EMS may also receive manual control messages from an operator of an HMI system. These control messages are sent from the HMI server via the EMS to the appropriate IEDs via a FEP. TABLE V EXAMPLE FOR DATA EXCHANGE (AGMC OPERATIONS) FROM A FEP TO A GENERATOR IED Data Exchange Attributes for Automated Grid Management and Control (AGMC) Operations Source FEP FEP Destination Generator controller Generator controller Exchange Type monitor control Interval seconds seconds or minutes Method unicast unicast Priority medium medium Latency Tolerance medium low Data Type run/stop/ATS status, fuel level, active & reactive output, frequency start/stop/mode/breaker control, voltage settings, governor droop settings Accuracy 1 decimal, second 1 decimal, second Volume bytes bytes Reliability important critical Information Assurance Confidentiality medium medium Integrity medium high Availability high high TABLE VI EXAMPLE FOR DATA EXCHANGE (AGMC OPERATIONS) BETWEEN AN EMS AND A HMI SERVER Data Exchange Attributes for Automated Grid Management and Control (AGMC) Operations Source EMS HMI Server Destination HMI Server EMS Exchange network concerns because or carry Server that auto and req the EM the broa sheer vo of its o through microgri relevant The enc • Dis sys • Ren ren • Ge ing Data  Exchange  Table  Format   Data  Exchange  Example   Example  Flat  Control  System   8
  9. 9. Defense-­‐in-­‐depth:  ApplicaDon  of     Enclaves  and  FuncDonal  Domains   9 Flat Segmented
  10. 10. Cyber  Security  QuanDtaDve  Analysis   10 and “report” can be considered as “reading” (from the field to the control center) and likewise all control traffic outward to the field devices can be labeled “write.” Furthermore, “high,” “medium,” and “low” are mapped to the numerical values 1, 2, and 3 respectively (although any could be used, the simplest approach is simple incrementing values). Summarizing the data exchange characteristics for each functional domain with the read/write strategy yields the data shown in Table VII. TABLE VII SUMMARIZED DATA ATTRIBUTES FOR EXAMPLE MICROGRID CONTROL SYSTEM. Functional Domain Read/Write Confidentiality Integrity Availability Subtotal Total HMI- Read 2 3 2 7 13 Server Write 2 2 2 6 Server- Read 2 3 2 7 13 FEP Write 2 2 2 6 FEP- Read 1 3 3 7 15 RTU Write 2 3 3 8 Totals Both 11 16 14 41 41 The testing against this example system was performed by cyber security Red Teams, modeling relevant threats (Section III). The tests were scored by carefully monitoring the data flows that form the functional domains during the exercise. If any flow in a functional domain was impacted according to confidentiality, integrity, or availability, then the affected security attribute was scored as a zero; otherwise, if unaffected it was scored according to the value in Table VII. Obviously, if any security attribute was impacted, then test score was less than perfect (100% of raw value 41). During testing, both read and write flows were impacted, sometimes in different ways. (a) Flat network (b) Enclaved network Fig. 7. Red Team access locations for the quantitative testing. C. Experiment Results Per the previous discussion, a total of eight versions of the notional microgrid control system network were deployed and tested in a laboratory setting at SNL. The Red Teams were Fig. 6. Reference architecture test network (enclaved configuration). the the “Type” attribute of the “Exchange” section for the ap- plicable data exchange worksheets (Table IV). Here, “monitor” and “report” can be considered as “reading” (from the field to the control center) and likewise all control traffic outward to the field devices can be labeled “write.” Furthermore, “high,” “medium,” and “low” are mapped to the numerical values 1, 2, and 3 respectively (although any could be used, the simplest approach is simple incrementing values). Summarizing the data exchange characteristics for each functional domain with the read/write strategy yields the data shown in Table VII. TABLE VII SUMMARIZED DATA ATTRIBUTES FOR EXAMPLE MICROGRID CONTROL SYSTEM. Functional Domain Read/Write Confidentiality Integrity Availability Subtotal Total HMI- Read 2 3 2 7 13 Server Write 2 2 2 6 Server- Read 2 3 2 7 13 FEP Write 2 2 2 6 FEP- Read 1 3 3 7 15 RTU Write 2 3 3 8 Totals Both 11 16 14 41 41 The testing against this example system was performed by cyber security Red Teams, modeling relevant threats (Section III). The tests were scored by carefully monitoring the data flows that form the functional domains during the exercise. If any flow in a functional domain was impacted according • Access: where in the network the modeled adversary has access (three choices, shown in Figure 7) • Compliance: a binary variable representing the cyber security of the platforms in the system, with “hardened” representing systems that are fully patched and secured according to current best practices, and “insecure” mean- ing they are not; due to the operational reliability neces- sary from energy control systems, hardware and software patches are not always applied in a timely manner (a) Flat network (b) Enclaved network Fig. 7. Red Team access locations for the quantitative testing. constrained to reasonable threat parameters (specifically, the “Mid” range shown in Table I). The results are in Table VIII. TABLE VIII MICROGRID CYBER SECURITY TEST RESULTS. Architecture Access Compliance Confidentiality Integrity Availability Total Flat High Insecure 0 0 8 8 Hardened 9 0 14 23 Enclaved High Insecure 0 0 8 8 Hardened 9 0 14 23 Med- Insecure 7 6 11 24 ium Hardened 9 6 14 29 Low Insecure 11 6 16 33 Hardened 11 6 16 33 Maximum Possible Score ! 11 16 14 41 The results indicate that each progressive variation to the reference implementation led to an increase in system security. More interesting is the fact that adding hardened systems to the enclaved versions of the reference implementation only increased the security by a small amount, and the small The authors w tricity Delivery this work, as w Idaho National L Technology Linc mand (USPACO Warfare Center participation in t [1] Systems and N Assessing and Systems (Versi (NSA), August [2] Brian Van Leeu Sandia Report Albuquerque, N [3] Control System Improving Indu Depth Strategie (NCSD), Depa [4] CSSP, Catalo Standards Deve [5] CSSP, Comm Systems, techn [6] Smart Grid In Group (CSWG Interagency Re Standards and H/M/L  SensiDvity  Scores  for  FuncDonal  Domains   Red  Team  Scoring  Results  
  11. 11. Advanced  Field  Device  Monitoring   Network monitoring alone is not sufficient to adequately defend against a sophisticated adversary PLCs are vulnerable to targeted attacks that cost millions in equipment damage, lost operation, or injured personnel. PLCs are not monitored for security compromise. It is not enough to build “secure” products. The ability to inspect and detect is necessary for systems that will be in place for decades. A backplane analysis system examines the communication between PLC modules Cyber attacks on the control systems will result in anomalies visible on the PLC backplane. New Capabilities for PLCs: •  Forensics: After compromises, detect modifications to hardware, firmware, or logic •  Detection: Actively detect anomalies 11
  12. 12. Advanced  Field  Device  Monitoring   §  WeaselBoard  plugs  into  the  backplane  and  listens  to  the  conversaDons  between   control  system  modules     §  There  is  a  lot  of  granularity  in  these  conversaDons,  which  allows  WeaselBoard  to   uniquely  observe  behavior  of  the  control  system  independent  of  the  processor  and   alert  when  the  system  is  not  operaDng  within  a  specifically  defined  manner     §  Because  it  alerts  on  effects  of  an  adack  in  progress,  and  not  on  signatures  of  prior   adacks,  WeaselBoard  can  detect  zero-­‐day  exploits   Processor Module Runs Process Logic PLC Backplane Comms Module Connects the PLC to the Network I/O Module Connects the PLC to the Process Isolation WeaselBoard Detects Intruders 12
  13. 13. Other  ICS  Cyber  Security  RecommendaDons   §  InvesDgate  all  miDgaDon  opDons,  covering  defend,  detect,  react,  and  recover   (including  incident  management/recovery  plans)   §  Develop  and  install  detecDon  capabiliDes  for  adack/anomaly  indicators   §  Complementary  opDons  include  network  traffic  monitoring  and  advanced  hardware   monitoring   §  Reduce  troubleshooDng  duraDon   §  Develop  effecDve  environments/procedures  for  tesDng   §  Minimize  adacker  opportuniDes  for  device  configuraDon  or  firmware  access   (possibly  disallowing  such  network  traffic)   §  Develop  logic-­‐  and  tamper-­‐checking  tools  for  devices  and  systems   §  Focus  on  cyber  security  assessment  for  field  devices   13
  14. 14. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. Cyber  Security  R&D  for  Microgrids     Panel  Session:  Emerging  System  Design     Requirements  –  Security,  Resiliency,  and  Reliability    Jason  Stamp,  Ph.D.   Sandia  NaDonal  Laboratories   14

×