Successfully reported this slideshow.

Test Tool for Industrial Ethernet Network Performance (June 2009)


Published on

Presented @ 55th International Instrumentation Symposium
League City, Texas, 1–5 June 2009

Ethernet is being used by a wider variety of industrial devices and applications. Industrial applications and systems require deterministic operations that traditional Ethernet and Transport Control Protocol / Internet Protocol (TCP/IP) suites were not originally designed to support. A standardized way to describe and test industrial devices is needed in order to aid users to characterize the performance of their software and hardware applications.

The Manufacturing Engineering Laboratory (MEL) of the National Institute of Standards & Technology (NIST) has been working to develop a set of standardized network performance metrics, tests, and tools since 2002. NIST has cooperated with standards organizations and other groups during that time.

NIST is presently working on developing an open-source test tool, called Industrial Ethernet Network Performance (IENetP), to aid vendors in characterizing the performance of their devices. The IENetP test tool will be capable of conducting a full series of performance tests and reporting the results to the user. The current version of the software is capable of analyzing network traffic and producing statistics and graphs showing the network performance of a device.

Published in: Technology
  • Be the first to comment

Test Tool for Industrial Ethernet Network Performance (June 2009)

  1. 1. Test Tool for Industrial Ethernet Network Performance Jim Gilsinn & Freemon Johnson Standards Certification Education & Training Publishing Conferences & Exhibits National Institute of Standards & Technology (NIST) Manufacturing Engineering Laboratory (MEL)
  2. 2. Author Bio: Jim Gilsinn • Over 16 years of experience at NIST • Variety of manufacturing projects, including intelligent unmanned vehicles, welding standards, smart sensors, and nano-manufacturing • Current research focus is industrial network reliability, including performance and security • Masters in Electrical Engineering from Johns Hopkins University, specialty in Control Systems; Bachelors in Electrical Engineering from Drexel University, specialties in Robotics and Control Systems 2
  3. 3. Author Bio: Freemon Johnson • Over 10 years of experience; previously worked at NASA Goddard Space Flight Center and Motorola • Expertise in communication protocol security, architecture, and embedded real-time systems • Masters in Computer Science from Johns Hopkins University; Masters in Communication and Information Studies from Rutgers University; Bachelors in Electrical Engineering from New Jersey Institute of Technology 3
  4. 4. Overview • • • • • Background Performance Testing Methodology Data Analysis Test Tool Future Plans 4
  5. 5. Background • The Problem – Ethernet & TCP/IP inherently non-deterministic – Users have many questions about industrial Ethernet devices – Automobile Example • Why NIST? – Aid U.S. businesses and industries, specifically manufacturing – Many companies are global – Industrial Ethernet standards are available worldwide 5
  6. 6. Performance Testing Methodology: Performance Metrics • Publish/subscribe or peer-to-peer communications • Main performance metric: Cyclic frequency variability/jitter • Real-time EtherNet/IP uses publish/subscribe – Requested/Accepted Packet Interval (RPI/API) – Measured Packet Interval (MPI) 6
  7. 7. Performance Testing Methodology: Performance Metrics • Command/response or master/slave communications • Main performance metric: Latency 7
  8. 8. Performance Testing Methodology: Testing Methodology • Overall methodology for testing performance: 1. Begin recording network traffic 2. Establish a connection with the device under test (DUT) 3. Begin transmitting background network traffic, based on the particular test conditions 4. Wait for a given amount of time 5. Stop transmitting background network traffic 6. Close the connection with the DUT 7. Stop recording network traffic 8. Analyze the network traffic capture and report the results • Current test tool only for data analysis & reporting 8
  9. 9. Performance Testing Methodology: Test System 9
  10. 10. Data Analysis: Overview • Current test tool focuses on cyclic frequency variability/jitter of the MPI • Capable of analyzing multiple network packet streams from different devices during a single test run • Conduct a distribution analysis on the MPI results – Determine minimum, maximum, mean, standard deviation, skewness, and kurtosis of data set • Produce graphs of MPI results 10
  11. 11. Data Analysis: Example Jitter Graph Measured Packet Interval (s) This Example Test Duration ≈ 61 s Mean ≈ 2 ms Minimum ≈ 1.2 ms Maximum ≈ 2.9 ms Test Time (s) 11
  12. 12. Data Analysis: Observations 12
  13. 13. Data Analysis: Observations 13
  14. 14. Test Tool: Design • Geared towards vendors & users – Vendors: develop better products during design process – Users: determine how well devices operate in a real system • Multi-phase design approach – Analysis → Testing • Meaningful results – Mathematical foundation – User selectable complexity for results 14
  15. 15. Test Tool: Technical Details • Free software package – Microsoft Windows-based (Windows 2000 or later) • Public domain source-code – Visual C# modular design – Commercial graphing package • Open-source installed package – No special software or hardware needed to run – Integrated with Wireshark for packet analysis 15
  16. 16. Future Plans • Planned versions of IENetP Test Tool – Data Analysis Tool – Version 1.x – Basic functionality & test engine – Version 2.x – Additional data analysis capability & metrics – Version 3.x – Additional industrial networks – Active Testing Tool – Version 4.x – PC-based test tool – Version ?.? – Specialized hardware test tool 16
  17. 17. Summary • NIST is working with industry partners to develop industrial Ethernet performance metrics, tests, and tools – Determined metrics – Developed basic methodology • NIST is currently working on a test tool – Designed as a development tool for industrial Ethernet vendors – Focused on data analysis presently • Some patterns have emerged while analyzing data • Additional functionalities are planned for the test tool 17
  18. 18. Contact Information • Jim Gilsinn, 301-975-3865, • Freemon Johnson, 301-975-3429, • Mailing Address – National Institute of Standards & Technology (NIST) Manufacturing Engineering Laboratory (MEL) 100 Bureau Drive, Mailstop 8230 Gaithersburg, MD 20899-8230 • Websites – Project: – Division: 18