In this document we propose the ICS Network blueprinting as the method to get the highest availability and security awareness for our critical control assets. (SCADA, PLC, RTU, IED, etc)
Industrial Control System Network Cyber Security Monitoring Solution (SCAB)
1. Continuous Security Monitoring in Industrial Control
System Networks through Blueprinting
By Enrique Martín Garcia
Schneider Electric – Global Solutions
IT Consulting & Integration Services
C/ Valgrande, 6
28018 Alcobendas
Madrid – Spain
enrique.martingarcia@telvent.com
2. Continuous Security Monitoring in Industrial Control
System Networks through blueprinting
January the 23
th
2014
2
Contents
Abstract ......................................................................... 2
Introduction................................................................... 2
ICS Monitoring Fundamentals .................................... 3
IT versus OT IP Networks ............................. 3
Connection Matrix ......................................... 4
Activity Matrix............................................... 5
Operational Matrix......................................... 5
Monitoring Technology Tests ..................................... 6
Connection Matrix tests ................................. 6
Operational Matrix tests................................. 7
IEC 61850 MMS abnormal operation ................... 8
IEC 61850 abnormal header length....................... 8
Activity Matrix tests....................................... 9
Summary ..................................................................... 10
Acknowledgements.................................................... 10
References .................................................................. 10
Abstract
Due to the huge growth in TCP/IP [1]
connected commercial off-the-shelf
industrial control systems, and the threats
associated with those, continuous
monitoring on right behavior for these
Operations Technology (OT) Networks has
become crucial for our regular way of life
and welfare. New monitoring technologies,
as behavioral blueprint definition in Control
System Networks, can help us in performing
better these important tasks.
Introduction
When talking about security we have to
focus on availability and reliability, since
industrial control systems are designed to
work on a 24x7 basis in mission critical
tasks; from transporting electricity, gas and
petrol we need, to process the water we
drink.
All these critical tasks depend on the
industrial control systems right behavior.
This behavior can be interrupted by an
abnormal unintentional human operation or
a malicious action driven by political,
economic or terrorist motivation. In most
cases, these kinds of actions are conducted
by insider organization users [2], who have
the rights, resources and sometimes
intentions, to interrupt the normal
operations.
To control the right function of these
systems we need to keep awareness of any
problem we could find through continuous
monitoring [3].
In this document we propose the ICS
Network blueprinting as the method to get
3. Continuous Security Monitoring in Industrial Control
System Networks through blueprinting
3
the highest availability and security awareness for
our assets.
ICS Monitoring Fundamentals
IT versus OT IP Networks
On any Information Technology (IT) IP Network
there are so many distinct activities with a high
amount of variance that it is extremely difficult to
discover abnormal issues.
As an example, we have registered the open TCP 80
port connections (HTTP) generated after browsing
just 10 Web pages, and we have got these results:
The browsed web Home pages had these categories:
One personal Web page
Two different National TV channels
Two different National Banks
Two different International On line Shops
Two different On line News papers
One North American University
In the next figure we can see the established
connections after only 3 web access:
This behavior is normal today on Internet
and is caused by the use of different
distributed contents web servers (adds,
banners, etc.) and other Marketing related
technologies out of our control.
IT traffic is not only HTTP, but DNS,
SMTP, POP3, IMAP, FTP, and other new
services that can change very quickly the
network connections needed to perform our
operations.
In this scenario it is clear that we cannot
think in having stable Network models to
establish the normal or right operational
behavior (Blueprint).
4. Continuous Security Monitoring in Industrial Control
System Networks through blueprinting
4
When comparing IT Networks with Operations
Technology (OT) Networks we find the following
differences:
OT Networks IT Networks
Number of IP devices Low High
Running Services Low High
Protocols used Low High
Internet Exposure Low High
Number of Threats Medium High
Availability priority High Low
Confidentiality priority Low High
Integrity priority Medium Medium
ICS Networks are:
Smaller in devices and services
Should not be directly connected to Internet
Well defined
Performs repetitive operations.
In these conditions it is easy to see that building a
normal behavior model is possible and, because of
that, any event away from that model can be
quickly detected and communicated.
To set up our blueprinting based ICS monitoring
system, we should think about three main
principles:
What goes where (Connection Matrix)
Who is doing what (Operational Matrix)
At what time goes (Activity Matrix)
If we have a clear knowledge about these three
issues, we have more chances of getting a
monitoring system that can avoid false positives
and gives the best performance by creating our ICS
Network Blueprint.
That blueprint is going to protect our ICS Network
from any non-desired operation or malicious
disruption attempt.
Let’s take a look on these points.
Connection Matrix
When talking about TCP/IP based
Networks, we have to have in mind that the
tuple (local ip, local port, remote ip, remote
port) is what uniquely defines a TCP/UDP
connection. Every IP addressed server
usually can use up to 65.536 ports.
Inside the TCP stack, these four fields are
used as a compound key to match up
packets to connections (e.g. file
descriptors).
Ports are 16-bit numbers, therefore the
maximum number of connections any given
client can have to any given host port is
64.000.
However, multiple clients can each have up
to 64.000 connections to some server's port,
and if the server has multiple ports or either
is multi-homed then you can multiply that
further.
We can calculate the maximum potential
connections between two servers in a
Network using the following formula:
(We are not considering connections from
one server to the same server).
These values can grow exponentially
depending on the number of servers
interconnected and the open port number
each server has.
When studying the open ports in a hardened
ICS Network, we have found the following
values:
5. Continuous Security Monitoring in Industrial Control
System Networks through blueprinting
5
ICS Network Node Open
Ports
Sum
Other
ports
Potential
Network
connections
Vijeo Citect Scada
Server 16 101 1616
Vijeo Citect Client 10 107 1070
Network Switch 6 111 666
Unity Pro
Workstation 2 115 230
Radius & Syslog
server 16 101 1616
Historian Server 16 101 1616
Historian Client 9 108 972
WSUS, NTP &
SNMP Server 16 101 1616
Firewall 6 111 666
PLC Modicom
M340 8 109 872
PLC Modicom
Quantum 12 105 1260
Total 12200
This bounded connections value makes possible to
think in having a well-defined Network Blueprint.
(Every system in an ICS or critical environment has
to be hardened to the max to reduce the amount of
resources needed to be managed and to minimize
the remote intrusion risks.)
On the other hand, the existence of a well-
documented connections table makes easier the
Network maintenance and improves the resilience
in case of problems since it is easier to detect any
misconfiguration or deployment error to fix it.
Activity Matrix
Many monitoring and advisory systems relays just
in the operations that are allowed on the network
but don’t take in account when this operations have
been performed. This fact makes things more
complicated if internal users perform inappropriate
operations (In an intentional or unintentional way).
Every ICS Network has to have some approved
operational procedures that any user has to follow
when login in to perform any operation, and those
procedures has to have some very well
known timetable to be executed.
With that information we should be able of
building a table that reflects the relationship
between users, procedures and actions.
Any activity out of that table should be
detected, registered and escalated as an
abnormal event.
It is clear that some operations like
reconfiguring a PLC or RTU, updating the
SCADA Data Base or changing running
states have to be very well known and
perform in Operational Windows defined by
the Administrator.
We will show later how the use of calendar
related features make easier to highlight
unusual or abnormal operations on the
production Network, and how these
anomalies can be detected and
communicated.
Operational Matrix
Although many ICS vendors support the
existence of different roles for the operation
environment, it is a fact that the work
groups many times use the same user and
password to perform their job, and in some
cases, these roles are not used in a correct
way because of easiness or fear to be unable
to login in critical situations.
This anonymous use of operation roles it is
difficult to detect and, without the other
measures described before, can make it
almost impossible to detect a human error or
malicious intervention.
We will show you later how the Network
Blueprint can be used to find not allowed
commands for legitimate users.
6. Continuous Security Monitoring in Industrial Control
System Networks through blueprinting
6
Monitoring Technology Tests
Once we have established our ICS Network
monitoring principles and methodology, we are
going to present our solution and how it performs
every of these aspects.
The technology used by our Cyber Security Area is
based in a disruptive approach to ICS Network
monitoring that is able of building, in a shelf-
learning way, the Behavioral Network Blueprint
(Normal behavior).
The Behavioral Blueprint defines communication
patterns, protocols, message types, message fields,
and field values which are allowed in your network
(i.e. the Network whitelist). Then, whenever a
communication that diverges from the Behavioral
Blueprint occurs, sensor system reports it,
pinpointing the exact source of the problem.
This technology is known as Deep Packet Behavior
Inspection (DPBI).
All this technology relies on a sensor device
controlled by a Command Center installed as a
physical or virtual Security Control Awareness Box
(SCAB) server.
Let’s see how the main ICS monitoring
principles work on our Cyberlab testing
devices.
Connection Matrix tests
After connecting the SCAB sensor to the
Network we begin the learning phase.
In that phase SCAB is building in an
unattended manner our Network Blueprint.
The flowchart can be viewed as:
We can customize the Behavioral Blueprint,
if needed, just adding, updating or deleting
connections using a text editor:
7. Continuous Security Monitoring in Industrial Control
System Networks through blueprinting
7
We can set up different protocols to be present in
our blueprint:
OPC-DA
MMS
Modbus/TCP
IEC 101/104
ICCP
IEC 61850
RPC/DCOM,
SMB/CIFS
DNP3
HTTP
VNC
RDP
After finishing the learning phase, we got the ICS
Local Network Communication Profile.
In that moment SCAB knows every tuple allowed
in the ICS network:
Src IP,Src Port -> Dest. IP,Dest Port
This is something hard to get in a multipurpose
Local Area Network (even a Home one) without
having several changes (Alerts) per hour.
From that moment we can be alerted by:
New devices on the network.
Devices trying connections out of the
model.
Devices receiving connections out of
the model.
To get these alerts we have to switch the
sensor from the learning state to the
detection state.
We made a very simple test after building
the network model, trying to connect from
10.1.1.243 to port 502 in 10.1.31.10.
Since this is a clear model violation we got
this:
Descending on the alert tree we got a much
more detailed Communication profile
violation description:
Operational Matrix tests
Although detecting non allowed
connections in very well bounded ICS
networks is not very impressive (but useful),
we need to keep aware of that.
But this feature is very well known and
completely useless when trying to detect
operation performed from valid devices by
8. Continuous Security Monitoring in Industrial Control
System Networks through blueprinting
8
valid users. To detect such actions we need to use
Deep Packet Behavior Inspection.
Once the Network Blueprint is created there is a
collection of connections (Source IP, source port,
Destination IP, destination port) protocols and
commands allowed.
But using the Deep Protocol Behavior Inspection
technology there is also a valid value or range
values allowed for each protocol message in the
Network model (Blueprint) SCAB has already
created.
Let’s see two cases on a very critical Industrial
Control Systems Network protocol: IEC 61850
MMS.
IEC 61850 MMS abnormal operation
In this case we have activated an IEC 61850 DPBI
sensor engine in learning mode on the SCAB and
we have performed some usual commands as
operator that didn’t include any “Delete File”
command.
This means that our Operational Matrix doesn’t
have this command in the “normal” Network
behavior.
After activating the detection mode, and performing
such command on the network, alert arise and
action is logged with the exact execution time, the
IP source and the information you can see in the
following figure:
Drilling down the violations message tree
we could even find the deleted file name
(IED_CONF.dat):
IEC 61850 abnormal header length
Using the same test environment (DPBI on
MMS activated), we have sent a message
from IP 10.1.1.243 with a 1026 bytes header
to our RTU simulator port on IP 10.1.31.10.
The effect of doing that made
unresponsiveness field devices attached to
that RTU (Denial of Service).
Once done that, SCAB detected the
operation source and destination IP address
9. Continuous Security Monitoring in Industrial Control
System Networks through blueprinting
9
Drilling down the alert we found the exact value
transmitted on the header (1026) that was the cause
of this potentially very serious problem.
Both cases were performed from legitimate IP
addresses by legitimate users, and only using this
technology could be detected, logged and
communicated.
Activity Matrix tests
Network model violations are always detected and
communicated but when these operations have been
performed out of the normal operation timetable,
we have to treat them in a much more important
way.
The most common scenario is detecting anomalies
in non-working hours (Holidays).
Since the holiday dates may vary per country or
region the exact dates can be set using the Holidays
feature.
Single or multiple dates can be added to the list of
holidays.
Using the holiday dates custom Alert Filters
can be added, for example, that show Alerts
that occurred during holiday days.
We have activated that feature on this test to
be warned on any access to field devices on
a holiday date.
Setting the test day as a holiday we got this
result:
All the alarms were detected and escalated
using this filter as all of them happened on a
“Holiday “ day.
10. Continuous Security Monitoring in Industrial Control
System Networks through blueprinting
10
Summary
It seems clear that Critical IC Organizations have
understood the need for Operational Network
continuous monitoring to detect every anomaly that
can compromise the desired performance and
operational goals.
Apart from deploying the appropriate security
procedures within the organization, our
methodology establishes three information sources
to achieve the desired operational security and
availability levels:
Connection Matrix
Operational Matrix
Activity Matrix
There should be a fourth matrix describing
Information Exchange between devices in our ICS
Network: The volumetric matrix.
When transmitting a new configuration file,
updating an SCADA Database or updating field
devices configuration files, we are using a very well
bounded range of values.
It is possible to establish maximum and minimum
values for the information size our ICS Network is
exchanging between different devices through flow-
based solutions.
To maintain and manage all this information we
propose the Security Control Awareness Box
solution as a very useful tool in SCADA Networks
to build the behavioral blueprint and detect any
problem caused by human errors or malicious
attempts to compromise those networks.
This methodology and disruptive sensor solution
combination proposed, let you examine your actual
network configuration & communications (Network
devices, application protocols, message
types/values), analyze network performance, detect
unexpected network communications and
configuration changes or new field devices
deployment errors, making easier to increase your
business ICS Network resilience.
Acknowledgements
I would like to thank Dell Spain and Intel
Spain by their support with the Cyberlab
server platforms.
References
[1] S.M. Bellovin. “Security Problems
in the TCP/IP Protocol Suite”. Computer
Communication Review, Vol. 19, No. 2, pp.
32-48, April 1989.
[2] Suzanne Dawson, Heather Davis,
Richard Lynch and Others. “2013 State of
Cybercrime Survey”. The Software
Engineering Institute CERT Program at
Carnegie Mellon University. pp. 9-12, Jun
2013
[3] Dr. Sandro Etalle, Dr. Cliford
Gregory, Dr. Damiano Bolzoni, Dr.
Emmanuele Zambon, Dr. Daniel
Trivellato,“Monitoring Industrial Control
Systems to improve operations and
security”, Dic 2013.