Dale Peterson of Digital Bond describes the methodology of using security assessment tools on an operational ICS. He also discusses how to best use the features and functions of these tools.

1. Using Cyber Security
Assessment Tools on
Industrial Control Systems (ICS)
Dale Peterson
Digital Bond, Inc.
peterson@digitalbond.com
Twitter: @digitalbond.com

2. ICS Security Assessments
• Digital Bond performed our first ICS security
assessment in 2000 … 15 years ago
• Digital Bond performs assessments on live /
operational / running critical infrastructure ICS
– Power plants, pipelines, water treatment, chemical
manufacturing, transportation
• Digital Bond uses scanning tools
• And we have never caused an unacceptable
impact to operations

3. Assessment Types
• Asset Owner / ICS End User Assessments
– Is the ICS deployed and maintained in a good security
practice configuration?
– Are known vulnerabilities remediated / fixed?
– This presentation covers Asset Owner Assessments
• Assessments for Vendors / New Purchases
– Attempts to find new, 0day vulnerabilities
– Very advanced testing, uses some commercial and
free tools, but also a lot of custom code
– Digital Bond Labs does these, see more tomorrow

4. Asset Owner Assessments
• Architecture Review
• Configuration Inspection
• Physical Inspection
• Policy and Procedure Review and Audit
• Interview (very important for determining risk)
and
• Online Scanning/Testing/Exploits

5. Current State of ICS Security
• Many organizations are just beginning to worry
about ICS security
– They may have a poorly configured firewall
– They may have some anti-virus running
– Little else in the way of ICS cyber security
• ICS protocols and PLC’s are insecure by design
– They lack basic security such as authentication
– Access = compromise
– Impact is limited to engineering and automation skill

6. Efficient Risk Reduction
What should I do next?
Where should you spend your next ¥ or
hour of time on ICS cyber security to
get the maximum risk reduction or
improvement in security posture?
• Assessment should provide a list of actions
prioritized by efficient risk reduction
• Companies have limited ability to add security

7. Prioritization
• Threat
– Very difficult to determine
– Typically look at the accessibility of the device/system
• Vulnerability
– Assessment can clearly identify this
• Impact
– This is the most important factor
– Don’t waste time on small impact risks, eg serial
connected panels
– Talk to the Operations team, what would happen if …

8. Even the most basic, simple,
non-intrusive scan of
a PLC or ICS application can cause
a denial of service condition.
TRUE!

9. Example 1
• Safety PLC
– Simple port scan of a safety PLC caused it to crash,
and it did not recover when rebooted
– Additional scanning found a port that was used to load
new firmware did not have authentication or even
check parameters
– Any activity on the port started a firmware update
process
– PLC needed to be completely reloaded to recover

10. Example 2
• Redundant Pair of Real Time Servers
– Issues read and write commands to PLC’s
– Provides data and forwards commands from HMI /
Operator Stations
• Scan of Standby Server … no problem
• Scan of Hot/Active Server … crash and failover

11. You cannot and should not use security
scanning tools on an operational ICS
because they can cause important
things to crash.
False!

12. How To Scan ICS
• Staging area or lab
– Some sites have non-operational systems to test
• Leverage redundancy
– An ICS should not have a single point of failure
– Many operator stations / HMI
– Hot and standby servers
• Select best testing time
– Many processes have key times weekly or daily were a
computer or device outage is more difficult to handle

13. Questions For Operations: 1. Is it
acceptable if computer x crashes during
the testing window? 2. Can you recover
the system in an acceptable time frame
if it crashes.
Answer: Yes … schedule scan

15. Answer: No … important security finding
• You have a recovery issue
– Don’t touch that because the guy who knew how it
worked is no longer with the company
– What is your Recovery Time Objective (RTO)?
– Do you have a proven ability to meet your RTO?
or
• You have a single point of failure
– Missing redundancy
– We can never reboot or have an outage of a Windows
NT, XP, 2003, 2008, 7 … FRAGILITY

16. Create Your Scan List
• Work with Operations to identify one of each
time of computer or device
• Find a sample that you can scan, assuming it
may go down, without having an unacceptable
impact to Operations
– Always assume it will go down
– Things are much better than 10 years ago

17. Scanning Tool Categories
• Basic Enumeration (what is it?)
• Full featured scan (1000’s of tests)
• Basic, random data fuzz testing
• Secondary application testing
– Web servers, databases
• Exploit proof of concept

18. Basic Enumeration
• Almost all recommend Nmap
– It’s free and fast
– Many claim it is more accurate
– The results are reasonable size and good for reference
• Nmap tells you
– What TCP/UDP ports are open
– What application and version is running on a port
– What operating system is running
• When not to run Nmap

19. Project Redpoint
• Digital Bond research project (free)
– https://github.com/digitalbond/Redpoint
– Also being integrated into Nmap download
• Nmap Scripting Engine (NSE) scripts
– Send legitimate ICS commands to enumerate specific
ICS devices and applications
– Identify ICS on the corporate network
– Great for creating and maintaining inventory
– Digital Bond tries to create new script whenever we
encounter a new ICS computer or device

21. BACnet

23. Broad Based Security Scanner
• Nessus from Tenable Network Security
• Nexpose from Rapid 7
• Retina from Beyond Trust
• DeepDiscovery from Trend Micro
Or
• Scanning as a service, Qualys

24. Example: Nessus
• Credentialed Scanning
• Learn the Product
• Security Audit

25. Broad Based Security Scanner
• New plugins (tests) are created for each
vulnerability or patch
• Nessus has over 75,000 plugins
– Not all will be applicable
– Not all will run in default config

26. Credentialed Scanning
• Inspect system with the same rights as an
Administrator or root user
• More accurate
– Patches: registry check vs. response to packet
• Less intrusive / less likely to crash computer
– Port scan vs netstat
• A lot more information
– Installed software, running services, users, group
policy info, USB usage, …
– Look at the information level results

27. Adding Credentials

28. Security Patching
• ICS scans often identify many missing patches
– Microsoft security patches
– 3rd party / application software security patches
– Security software security patches, eg anti-virus
– Even ICS security patches
Question: What is the security finding?
Answer: Ineffective security patching program

29. Security Patching in ICS
• Good security practice is to apply patches in a
reasonable time after available
– IT / corporate network typically 30 days
– Best in ICS is typically quarterly / 90 days
Question: Can you go from little or no security
patching to applying all patches every 90 days?
Think Efficient Risk Reduction

30. Prioritized Security Patching
• Priority 1 – Computers accessible from corporate
or external network
– Monthly … should be a small number of computers
that are not required for operation
• Priority 2 – Computers accessible from Priority 1
computers
– Quarterly … attackers will compromise Priority 1
computers and pivot
• Priority 3 – Everything else
– Annual … maintain supported system

31. Controversial
• If you can do better, great
– Shorter patching windows are better security, but
– We see many owner/operators fail in patching
• Select some achievable plan, succeed, and then
shorten patching window
• Also … if an attacker can reach a Priority 3
computer he can compromise the ICS even if it is
patched … ICS is insecure by design

32. Know Your Scanner
• These are complex, full feature products
• Default scan configurations will miss a lot of
what you want to know in an assessment
• Take a class from the vendor or skilled teacher

33. Nessus Example 1
• Oracle Default Passwords

34. Nessus Example 2 – USB Usage
• USB Drive Usage

35. Compliance Audit
• Identify an optimal security configuration for OS
and all ICS applications
• Develop an audit file for the scanner
• Use the compliance plugin
• Digital Bond Bandolier Project
– Funded by US Department of Energy

36. Adding the Audit File
• About 200 operating system (OS) audit tests
• Number of ICS application tests vary

37. Audit File Example
• Folder Permissions
• ICS applications install software in one or more
folders
– Read, write and execute permissions for the folders
should be least privilege
– Permissions are often set to Everyone
• Vendor should define optimal security config
– Ideally provide a document and audit file
– Modify as necessary for your policies & environment

38. Random Data Fuzzing
• ICS vendors historically only performed positive
testing
– Does the application or device perform properly when
receiving a legitimate command or packet
• Hackers, scanners, new applications may send
something unexpected
– Will the application/device handle the “error” properly
– Or will it crash
• This is a crude test
– Not intelligent fuzzing that the vendor should perform

39. Secondary Testing
• May not be necessary
– Usually required after an ICS security program has
been running for 2 to 3 years
– An attacker will take the easiest path to success
• Specialized tools and techniques
– Web application testing
– Database testing
– Password cracking
– Man-in-the-middle / ARP spoofing

40. Proof of Concept Exploits
• If assessor is uncertain if vulnerability can be
exploited
– Should be attempted to accurately determine risk
– Denial of service vs. remotely run code
• Prove the danger of missing security patches /
default credentials / other vulnerabilities
– Show the Operator Station on your laptop
– Attack compromise and pivot

42. How Many Assessments?
What if you have 50 or 100 factories or plants?
Should you perform an assessment at each
factory or plant?

43. Recommendation
• Pick 3 to 5 different sites
– Pick a variety of size and types of plants
– Select a representative sample
– Perform assessments on the samples
• Identify the common high priority findings
• Define a common set of required security controls
– Not too much in the first year
• Define how the controls will be audited
• Add additional controls in years 2, 3, …

44. Questions