Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Master Serial Killer - DEF CON 22 - ICS Village

1,999 views

Published on

Updated slides on Master Serial Killer from Adam Crain and Chris Sistrunk's research on ICS Protocol Vulnerabilities called Project Robus, the Aegis Fuzzer, and mitigations of these vulnerabilities.

Published in: Software
  • Be the first to comment

Master Serial Killer - DEF CON 22 - ICS Village

  1. 1. Master Serial Killer Chris Sistrunk PE, Mandiant Adam Crain, Automatak
  2. 2. About Us Chris Sistrunk, PE • Electrical Engineer • SCADA Expert • Loves Security • DNP3 Member • Button Pusher Adam Crain • Software Engineer • OSS Advocate • openDNP3 Author • DNP3 Member • Code Monkey
  3. 3. How I Audit SCADA systems http://securityreactions.tumblr.com/post/30866100673/how-i-audit-scada-systems
  4. 4. ICS/SCADA Security • ICS/SCADA lags IT by 10-15 years • 708 SCADA-related vulns on OSVDB.org since 2011. “Like kicking a puppy” • Positive vs. Negative Testing: The front yard is mowed, but the back yard is overgrown.
  5. 5. Software Testing
  6. 6. When you scan ICS with nmap
  7. 7. SCADA Protocol Vuln Research We chose to focus on popular SCADA protocols Fuzzers did exist, but only tested server side Serial had not been fuzzed before (that we know of) We chose to use Responsible Disclosure • Inform the vendor, then ICS-CERT, DNP3 UG • Worked with the vendor to help them replicate and begin further negative testing
  8. 8. Project Robus • Latin for “bulwark” • Started in April 2013 • 24 advisories / 30 tickets • 22 DNP3, 1 Modbus, 1 Telegyr 8979 www.automatak.com/robus www.automatak.com/aegis
  9. 9. Fuzzing Master Stations • Referenced in Nat’l SCADA Test Bed reports but no data available • Wurldtech & Spirent (Mu Dynamics) don’t fuzz the master side of ICS protocols…………..yet Master Slave
  10. 10. Fuzzing Master Stations DNP3 Application Function Code 0x82 • If the Master Station has Unsol enabled, it must accept messages from its RTUs at any time • Design of System must be fine tuned...or else DNP3 Outstation Unsolicited Response Storm • If the Master parser has problem with one message, you can imagine the problems with many many messages
  11. 11. Serial Fuzzing All the security focus has been on ethernet networks, but many ICS, especially SCADA, still utilize serial networks. • DNP3 is same! (unlike Modbus) • Impact to NERC/CIP v3 & v5 Physical Security (discuss later) • Pole-mounted RTUs • PQ Meters, etc
  12. 12. DNP3 (IEEE 1815-2012) Primer DNP3 is a SCADA protocol used by almost all of the electric utilities and some water in North America, Australia, and the UK. Created in 1990s and turned over to DNP3 UG in 1993. One of the few ICS protocols that has secure auth. SCADA Master RTU with I/O
  13. 13. Breaking Down DNP3 TCP 20000 TCP 19999 (TLS) UDP 20000 Ref from IEEE Std 1815-2012
  14. 14. Courtesy of
  15. 15. Vendor Response Matrix ICS-CERT Adv Company Protocol Bug Fix Days Advisory ICSA-13-161-01 IOServer DNP3 4/24 5/24 30 6/10/2013 ICSA-13-213-03 IOServer DNP3 5/1 7/20 80 8/1/2013 ICSA-13-219-01 SEL DNP3 5/1 5/30 29 8/7/2013 ICSA-13-226-01 Kepware DNP3 4/24 6/18 55 8/14/2013 ICSA-13-234-02 TOP Server DNP3 4/24 6/18 55 8/22/2013 ICSA-13-240-01 TMW DNP3 4/24 6/17 54 8/28/2013 ICSA-13-213-04A Matrikon DNP3 4/24 6/17 54 8/29/2013 ICSA-13-252-01 Subnet DNP3 4/24 8/30 128 9/9/2013 ICSA-13-282-01 Alstom DNP3 4/24 6/4 41 10/21/2013 ICSA-13-297-01 Catapult DNP3 4/24 10/1 160 11/22/2013 ICSA-13-297-02 GE IP DNP3 Self Report 10/1 n/a 11/22/2013 ICSA-13-337-01 Elecsys DNP3 9/12 11/4 53 12/3/2013 ICSA-13-346-02 Cooper OPC DNP3 7/31 None ∞day™ 12/12/2013 ICSA-13-346-01 Cooper/Cybectec DNP3 5/1 12/12 225 12/12/2013 ICSA-13-352-01 Novatech DNP3 5/1 9/5 127 12/18/2013 ICSA-14-014-01 Schneider DNP3 8/6 8/23 17 1/14/2014 ICSA-14-100-01 IOServer Modbus 2/6 3/4 26 4/10/2014 ICSA-14-154-01 COPA-DATA DNP3 Self Report using Aegis! n/a 6/3/2014 ICSA-14-196-01 Subnet TG8979 4/18 6/18 61 7/31/2014
  16. 16. Vendor Response • Most of the vendors were very pleased • A few were not >> head in the sand • Some had never done negative testing • Nearly all devices and hosts with DNP3 were affected, so it was an industry-wide wakeup call.
  17. 17. White Noise Fuzzing #1 random == really “dumb”
  18. 18. Template (mutational) Fuzzing
  19. 19. Generational “Smart” Fuzzing
  20. 20. Multi-field Anomalies
  21. 21. Generational == most vulns!
  22. 22. The Aegis ICS Fuzzing Framework • We decided that we needed to release our fuzzing framework tool as open source. • Open source security tools have a proven track record of raising security (hello MSF!) • We do encourage people to join our efforts to add more protocols to Aegis
  23. 23. Aegis Specifics • Version 0.1.x in Scala www.scala-lang.org • Current version (private release) in C# • Protocol boundary conditions • Abstracts physical layer • Combines aspects of generation and mutation • Repeatable random seeds • ~500,000 test cases with one seed
  24. 24. Test DNP3 Message (DL, TL, or AL) Request Link States Link Status x Num Test Cases Request Response x Num Retry (10) Fuzzer Test Flow
  25. 25. I 0x0564 U... Y U NO 0x0564 ME BAK ?!
  26. 26. Combinatorics val nums = List(1, 3) val colors = List(“red”,”green”) // repeat the reversed string num times def combine(i: Int, s: String) = List.fill(i)(s.reverse).mkString val result = Cartesian.Transform(colors,nums)(combine) What is result?
  27. 27. Lazy Generator // val nums = List(1, 3) // val colors = List(“red”,”green”) > result.foreach(println) der derderder neerg neergneergneerg
  28. 28. { frames } = f (byte,Type) {byte} = f (bool, bool, int) {Type} = f (.....) { true, false } { true, false } { 0, 1, 63 } ........................... Fuzzing is O(2n)
  29. 29. Generators can get large! { test cases } ● Many function codes ● Many objects ● Header types ● Many field values
  30. 30. Types of Vulnerabilities
  31. 31. FA 82 00 00 01 00 02 00 00 00 00 FF FF FF FF Unsolicited Response Group 1 Variation 0 Sizeless?! 4 byte start/stop 0 4294967295 ● infinite loop ● missing data ● integer overflow? ● accepts broadcast Vuln #1
  32. 32. DD 82 00 00 0A 02 01 00 00 FF FF UNSOL Group 10 Variation 2 Binary Output Status 2 byte start/stop 0 65535 ● infinite loop ● missing data ● unexpected data ● integer overflow? Vuln #2
  33. 33. 05 64 06 44 64 00 64 00 FF F2 C0 1D 0A 1 byte payload 100 100 ● transport header only ● unhandled exception unconfirmed user data CRC CRC FIR / FIN SEQ = 0 Vuln #3
  34. 34. Recorded Demos Video 1: a DNP3 outstation -application layer object fuzzing Video 2: a DNP3 master -unsolicited application layer fuzzing
  35. 35. Vuln #4 (TMW integration) DD 82 00 00 0C 01 00 00 01 rnd(11) rnd(11) Unsolicited Response Control Relay Output Block 1 byte start/stop CROB #1 CROB #2 ● buffer overrun ● not malformed! ● unexpected objects ● accepts broadcast
  36. 36. Vuln #5 (TMW integration) FA 82 00 00 02 02 01 01 00 FF FF Unsolicited Response Group 2 Var 2 (event) 2 byte start/stop 1 65535 ● stable infinite loop ● max range - 1 and no data ● accepts broadcast
  37. 37. Using Aegis
  38. 38. So easy…Twitter can do it
  39. 39. Examples Run 10 link layer test cases starting at #123 $ aegis-console -mid dnp3 -pid lfuzz -start 123 -count 10 Unsolicited response fuzzing of a master listening on default port 20000 with master address of 0 and an outstation address of 1 $ aegis-console -mid dnp3 -pid aufuzz -dest 0 -src 1 -master -listen Outstation link layer fuzzing test case #100 only $ aegis-console -mid dnp3 -pid lfuzz -start 100 -count 1 Outstation application object fuzzing against 192.168.1.55:20001 with default addressing $ aegis-console -mid dnp3 -id aofuzz -host 192.168.1.55 -port 20001
  40. 40. Further Aegis Development • In addition to DNP3 protocol, we’ve added Modbus and Telegyr 8979 (serial only) protocol modules to the framework. • Migrated from scala to C#. • Added a GUI • Working with vendors and other trusted researchers.
  41. 41. New Aegis Demo --- module: dnp3 - Test routines for the DNP3 protocol --- Procedure ids: link Fuzzing of the link layer (masters or outstations) transport Fuzzing of the transport function (masters or outstations) requests Fuzzes the application layer with malformed and unexpected requests (outstation) unsol Fuzzes the application layer with malformed and unexpected unsolicited responses (master) octetunsol Reports large numbers of 0-length octet string headers via unsolicited mode (master) octetwrite Writes large numbers of 0-length octet string headers (outstation) randrequest Fuzzes the application layer with semi-random requests (outstation) randunsol Fuzzes the application layer with semi-random unsolicited responses (master) -dest <arg>(1024)[0, 65535] link layer address of the target -src <arg>(1)[0, 65535] link layer address of the fuzzer -master <arg>(False) set the link-layer master bit for master fuzzing -retries <arg>(10)[1, none] Number of link status retries -timeout <arg>(1000)[10, none] Read timeout in milliseconds -health <arg>(LinkStatus) Type of health check to use [linkstatus, resetlink] --- module: modbus - Test routines for the Modbus protocol --- Procedure ids: request Sends malformed or unexpected requests at a Modbus slave
  42. 42. ICS/SCADA Defense
  43. 43. ICS/SCADA Defense Network Security Monitoring (do it now!) • Bro, SNORT, Wireshark can parse DNP3 & Modbus! • Deep packet inspection firewalls • Full packet capture (even serial) – 1TB y’all • Use Security Onion to monitor ICS networks http://www.liquidmatrix.org/blog/2014/07/01/is-there-a-cuckoo-in-your-control-system/ tl;dr ≥1 person, Security Onion, and an ICS Honeypot
  44. 44. ICS/SCADA Defense • Install patches – not quite like IT • Robust device & network configuration • Disable unused ports, protocol function codes • Whitelist apps and even traffic • DNP3 Secure Authentication v5 & TLS • Signed software/firmware
  45. 45. Physical Security 3/8” Mesh ASTM Grade 6 Buys extra time
  46. 46. What’s different about Robus? SCADA Vulns reported for a while now Adam and I aren’t security researchers • He’s a software geek…I’m an engineer • Our skills complemented each other • Both experts in DNP3 protocol, but from different angles
  47. 47. Some theories Why did the industry move instead of ignore? • I was an end user and we really cared! • Not just a wham-bam researcher • Respectful, tactful, responsible • We released our tool …………we weren’t going away
  48. 48. I’m still more worried about…
  49. 49. SHODAN Probably default configs • Many similar responses • Same DNP Addresses python shell >>> " ".join("%02x" % ord(i) for i in "DNP3 paste from shodan”) Unsolicited Response with Binary and Analog Data Class 1/2/3/0 Poll!!! https://ics-radar.shodan.io/ https://maps.shodan.io/
  50. 50. Conclusions • DNP3 is not a special case, other protocols same fate Modbus, IEC 60870, IEC 61850, ICCP, EtherNet/IP… • Early testing both slave/server AND master/client sides of protocols are important! • Compliance != Security, but the culture is important • Don’t have to be a nation/state or large firm to do this • A few good folks can make a difference in the industry
  51. 51. Questions? @jadamcrain @chrissistrunk

×