Rorschach Plots and Network
Performance Analysis
Jim Gilsinn
Kenexis Consulting Corporation

October 19-20, 2013

BSidesDC...
Rorschach?

October 19-20, 2013

BSidesDC 2013

2
“Rorschach” Plots

October 19-20, 2013

BSidesDC 2013

3
“Rorschach” Plots

October 19-20, 2013

BSidesDC 2013

4
“Rorschach” Plots

October 19-20, 2013

BSidesDC 2013

5
ICS Environment

October 19-20, 2013

BSidesDC 2013

6
ICS Environment

October 19-20, 2013

BSidesDC 2013

7
ICS Systems

October 19-20, 2013

BSidesDC 2013

8
What’s This All About?
• I used to work at NIST
• I left about a year ago
• I worked on ICS network performance metrics, t...
Performance Testing Methodology:
Performance Metrics

• Publish/subscribe or peer-to-peer communications
• Main performanc...
Performance Testing Methodology:
Performance Metrics

• Command/response or master/slave communications
• Main performance...
IENetP Test Tool
• Industrial Ethernet Network Performance (IENetP)
• http://sourceforge.net/projects/ienetp/
• Current Ve...
NIST Performance Test Tool
• Industrial Ethernet Network Performance (IENetP) Test Tool

• Factory Equipment Network Testi...
FENT Framework
Universal Client Application
Testing
Testing
Module
Testing
Module
Testing
Module
Module

Analysis
Analysis...
FENT Features
• All Analysis Features From IENetP
• Analyze Wireshark capture files
• Build graphs and reports of results
...
FENT Personality Modules
• Wrapper for Driver
Application
• Implement a TCP-socket
interface for UCA-API
messaging
• Build...
FENT Framework Run-Time
UCA
Testing
Module

Analysis
Engine

1.
Reporting
Engine

2.
PSML File

3.
4.

UCA – API

Wireshar...
FENT UCA-API Schema

October 19-20, 2013

BSidesDC 2013

18
FENT Framework
• Project Home:
• http://sourceforge.net/projects/fent/

• What’s Available:
• SVN repository & schema
• FE...
FENT Framework
• Known Problems & Issues
• Doesn’t work with Wireshark 1.9+
• Tshark argument for getting fields changed

...
FENT Demo

October 19-20, 2013

BSidesDC 2013

21
“Gold Standard” Background Traffic

October 19-20, 2013

BSidesDC 2013

22
“Gold Standard” Background Traffic
• What Is It?
• A set of Wireshark captures, Linux scripts, and analysis results
• Base...
ARP Request Broadcasts

180

Gratuitous ARP Broadcasts

180

DHCP Request Broadcasts

100

ICMP (ping) Request Broadcasts
...
“Gold Standard” Captures
• Built From Individual Traffic Streams
• Each traffic stream generated and captured using NIST I...
Licensing?
• The project is Public Domain!!!
• There are NO LICENSING ISSUES!!!

October 19-20, 2013

BSidesDC 2013

26
What’s Next?
• Contact Me
•
•
•
•
•

Jim Gilsinn
301-706-9985
jim.gilsinn@kenexis.com
Twitter – @JimGilsinn
LinkedIn – htt...
Upcoming SlideShare
Loading in …5
×

Rorschach Plots and Network Performance Analysis

673 views
542 views

Published on

Presented @ BSidesDC 2013
Washington, DC, October 20, 2013

Measuring the performance of network protocols that require determinism can be difficult with the existing set of tools. Tools like Wireshark can give you the details of the protocols themselves and some general statistics about the packet streams, but they don’t easily show the full set of traffic for those streams. Visual tools like Etherape can show you the full set of traffic streams, but don’t give you any idea of the nuances of the performance represented in those traffic streams.

While at the National Institute of Standards and Technology (NIST), I built a tool capable of analyzing and displaying the performance of network protocols. The first generation of the tool was called the Industrial Ethernet Network Performance (IENetP) test tool and the second generation of the tool is called the Factory Equipment Network Testing (FENT) framework. Both these tools are available on SourceForge and are public domain. I have since left NIST and the tools haven’t been picked up by anyone.

The FENT framework is useful for analyzing the performance of any deterministic protocol and reporting certain performance characteristics. It was originally designed for EtherNet/IP (Ethernet / Industrial Protocol), Modbus, Profinet, and other industrial Ethernet-based protocols, but has proven to be useful for many other protocols as well. The most beneficial part of the software has actually been the graphical analysis, which in many cases resemble Rorschach plots due to the subtle performance problems that show up as strange patterns in the data.

My presentation will describe the FENT framework, present the tool in its current state, and display some of the more interesting results. It will also be a plea for someone to take up the open-source development of this project and move it forward. My new position does not leave me with enough time to dedicate to the project, so the project has been dormant for the last few months. I’ve received complements on the project from many industrial partners in the past and they would like to see further development, but that means that someone else has to take on the task.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
673
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Rorschach Plots and Network Performance Analysis

  1. 1. Rorschach Plots and Network Performance Analysis Jim Gilsinn Kenexis Consulting Corporation October 19-20, 2013 BSidesDC 2013 1
  2. 2. Rorschach? October 19-20, 2013 BSidesDC 2013 2
  3. 3. “Rorschach” Plots October 19-20, 2013 BSidesDC 2013 3
  4. 4. “Rorschach” Plots October 19-20, 2013 BSidesDC 2013 4
  5. 5. “Rorschach” Plots October 19-20, 2013 BSidesDC 2013 5
  6. 6. ICS Environment October 19-20, 2013 BSidesDC 2013 6
  7. 7. ICS Environment October 19-20, 2013 BSidesDC 2013 7
  8. 8. ICS Systems October 19-20, 2013 BSidesDC 2013 8
  9. 9. What’s This All About? • I used to work at NIST • I left about a year ago • I worked on ICS network performance metrics, tests, and tools • The test tools I developed have been dormant since leaving • The vendors I worked with while at NIST want to tool • My new employer won’t support open-source development • I’m here to beg for help! October 19-20, 2013 BSidesDC 2013 9
  10. 10. Performance Testing Methodology: Performance Metrics • Publish/subscribe or peer-to-peer communications • Main performance metric: Cyclic frequency variability/jitter • Real-time EtherNet/IP uses publish/subscribe • Requested/Accepted Packet Interval (RPI/API) • Measured Packet Interval (MPI) October 19-20, 2013 BSidesDC 2013 10
  11. 11. Performance Testing Methodology: Performance Metrics • Command/response or master/slave communications • Main performance metric: Latency • Large numbers of protocols use this • Most (All?) PC-based server/client protocols – HTTP(S), (S)FTP, etc. • Most industrial protocols – Modbus/TCP, Profinet, Ethercat, etc. October 19-20, 2013 BSidesDC 2013 11
  12. 12. IENetP Test Tool • Industrial Ethernet Network Performance (IENetP) • http://sourceforge.net/projects/ienetp/ • Current Version = 1.1.2, Released 2011-02-11 • Software Features • • • • • • • Analyze existing Wireshark captures Allows user to override default EtherNet/IP filter Isolates individual traffic streams Determine cyclic jitter of those streams Generates HTML report Generates time-space & histogram graphs Graphs allow zooming October 19-20, 2013 BSidesDC 2013 12
  13. 13. NIST Performance Test Tool • Industrial Ethernet Network Performance (IENetP) Test Tool • Factory Equipment Network Testing (FENT) Framework
  14. 14. FENT Framework Universal Client Application Testing Testing Module Testing Module Testing Module Module Analysis Analysis Engine Engine Reporting Engine Universal Client Application API Personality Module Personality Module Personality Module Ethernet Sensor Gateway Internet Fieldbus October 19-20, 2013 BSidesDC 2013 14
  15. 15. FENT Features • All Analysis Features From IENetP • Analyze Wireshark capture files • Build graphs and reports of results • Added Features • True multi-protocol support • Real-time testing capability • Extensible framework October 19-20, 2013 BSidesDC 2013 15
  16. 16. FENT Personality Modules • Wrapper for Driver Application • Implement a TCP-socket interface for UCA-API messaging • Build a simple XML-based PM Descriptor file UCA – API Protocol PM PM Descriptor • Features • Describes Wireshark parameters • Allows any protocol to be used • Can be built/loaded at run-time October 19-20, 2013 BSidesDC 2013 Driver App 16
  17. 17. FENT Framework Run-Time UCA Testing Module Analysis Engine 1. Reporting Engine 2. PSML File 3. 4. UCA – API Wireshark Protocol PM PM Descriptor 5. Driver App 6. Testing Module  Protocol PM – Grab protocol-specific Wireshark parameters via UCA-API Testing Module  Wireshark – Start capturing traffic Testing Module  Protocol PM – Command driver app to communicate with DUT Testing Module  Wireshark – Stop Capturing traffic, process capture file using desired protocol and user parameters, generate PSML file Analysis Engine – Read PSML file, analyze packets for desired metrics Reporting Engine – Report data to user DUT October 19-20, 2013 BSidesDC 2013 17
  18. 18. FENT UCA-API Schema October 19-20, 2013 BSidesDC 2013 18
  19. 19. FENT Framework • Project Home: • http://sourceforge.net/projects/fent/ • What’s Available: • SVN repository & schema • FENT software • Conduct real-time testing • Analyze results • Build graphs on-screen • NIST SensorSim PM, IEEE 1451 PM • EtherNet/IP PlugFest “Gold Standard” Background Traffic October 19-20, 2013 BSidesDC 2013 19
  20. 20. FENT Framework • Known Problems & Issues • Doesn’t work with Wireshark 1.9+ • Tshark argument for getting fields changed • • • • Logic problems with using multi-protocol Wireshark headers Software doesn’t use true database Testing automation not integrated No installer October 19-20, 2013 BSidesDC 2013 20
  21. 21. FENT Demo October 19-20, 2013 BSidesDC 2013 21
  22. 22. “Gold Standard” Background Traffic October 19-20, 2013 BSidesDC 2013 22
  23. 23. “Gold Standard” Background Traffic • What Is It? • A set of Wireshark captures, Linux scripts, and analysis results • Based on EtherNet/IP PlugFest performance testing requirements • High precision and accuracy Wireshark captures of PlugFest performance background traffic • Linux scripts designed for use in BackTrack Linux (http://www.backtrack-linux.org/) • Analysis results show validation for use in PlugFest performance testing • Where Can You Get It? • http://ienetp.sourceforge.net/EtherNet-IP_Testing.zip or • FENT SVN in Background_Traffic folder October 19-20, 2013 BSidesDC 2013 23
  24. 24. ARP Request Broadcasts 180 Gratuitous ARP Broadcasts 180 DHCP Request Broadcasts 100 ICMP (ping) Request Broadcasts 100 NTP Multicasts 10 EtherNet/IP ListIdentity Request 10 EtherNet/IP Class 1 1800 ARP Burst Requests 240 pkts @ 4k Hz Burst Unmanaged Burst Managed Steady-State Unmanaged Rate (pps) Steady-State Managed Traffic Type Baseline PlugFest Performance Traffic
  25. 25. “Gold Standard” Captures • Built From Individual Traffic Streams • Each traffic stream generated and captured using NIST Ixia system (a few microseconds jitter) • Assembled using editcap and mergecap scripts • Final captures are 60-seconds long • Can’t just loop continuously • Longer test captures require rebuilding (not hard) • Analyzed Using IENetP • Analysis results are included in package • Well within spec for PlugFest performance testing needs (<25% of desired packet intervals) October 19-20, 2013 BSidesDC 2013 25
  26. 26. Licensing? • The project is Public Domain!!! • There are NO LICENSING ISSUES!!! October 19-20, 2013 BSidesDC 2013 26
  27. 27. What’s Next? • Contact Me • • • • • Jim Gilsinn 301-706-9985 jim.gilsinn@kenexis.com Twitter – @JimGilsinn LinkedIn – http://www.linkedin.com/in/jimgilsinn/ • Review the FENT SourceForge Project • http://sourceforge.net/projects/fent/ • Fork the Project October 19-20, 2013 BSidesDC 2013 27

×