Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Developing the Web100 Based  Network Diagnostic Tool (NDT)  Internet2 piPEs Tutorial Rich Carlson [email_address]
Demo <ul><li>http://ndt-newyork.abilene.ucaid.edu:7123 </li></ul>
Normal operation in campus
Duplex Mismatch Detected
Low throughput from remote host
Increase TCP buffer size
Motivation for work <ul><li>Measure performance to users desktop </li></ul><ul><li>Develop “single shot” diagnostic tool t...
Web100 Project <ul><li>Joint PSC/NCAR project funded by NSF </li></ul><ul><li>‘First step’ to gather TCP data </li></ul><u...
Web Based Performance tool <ul><li>Operates on Any client with a Java enabled Web browser </li></ul><ul><li>What it can do...
Web base Performance tool <ul><li>What it can’t do </li></ul><ul><ul><li>Tell you where in the network the problem is </li...
Internet2 piPEs Project <ul><li>Develop E2E measurement infrastructure capable of finding network problems </li></ul><ul><...
piPEs Integration
Bottleneck Link Detection <ul><li>What is the slowest link in the end-2-end path? </li></ul><ul><ul><li>Monitors packet ar...
Duplex Mismatch Detection <ul><li>Developed analytical model to describe how Ethernet responds (no prior art?) </li></ul><...
Future enhancements <ul><li>WiFi detection </li></ul><ul><li>Faulty Hardware detection </li></ul><ul><li>Congestion modifi...
Additional Functions and Features <ul><li>Provide basic tuning information </li></ul><ul><li>Basic Features  </li></ul><ul...
Availability <ul><li>Open Source Development project </li></ul><ul><ul><li>http://www.sourceforge.net/projects/ndt </li></...
NDT Flow Chart Client Web Browser Java Applet NDT - Server Web Server Testing  Engine Child Test Engine Spawn child Well K...
NDT servers
Results and Observations <ul><li>Changing desktop effects performance </li></ul><ul><li>Faulty Hardware identification </l...
<ul><li>10 Mbps NIC </li></ul><ul><ul><li>Throughput 6.8/6.7 Mbps send/receive  </li></ul></ul><ul><ul><li>RTT 20 ms </li>...
<ul><li>100 Mbps FD </li></ul><ul><li>Ave Rtt %loss </li></ul><ul><ul><li>5.41 0.00 </li></ul></ul><ul><ul><li>1.38 0.78 <...
<ul><li>100 Mbps FD </li></ul><ul><li>Ave Rtt %loss  loss/sec </li></ul><ul><ul><li>5.41 0.00 0.03 </li></ul></ul><ul><ul>...
Mathis et.al Formula fails <ul><li>Estimate = (K * MSS) / (RTT * sqrt(loss)) </li></ul><ul><ul><li>old-loss = (Retrans - F...
NDT Hardware Requirements <ul><li>Minimum requirements </li></ul><ul><ul><li>500 MHz Intel or AMD CPU </li></ul></ul><ul><...
NDT Software Requirements <ul><li>Web100 enhancements </li></ul><ul><ul><li>Linux kernel </li></ul></ul><ul><ul><li>User l...
Recommended Settings <ul><li>There are no settings or options for the Web based java applet.  </li></ul><ul><ul><li>It all...
Potential Risks <ul><li>Non-standard kernel required </li></ul><ul><ul><li>GUI tools can be used to monitor other ports </...
Possible Alternatives <ul><li>Other tools that can perform client testing </li></ul><ul><ul><li>Several web sites offer th...
<ul><li>Supplemental information </li></ul>
NDT’s Web100 Based Approach <ul><li>Simple bi-directional test to gather E2E data </li></ul><ul><li>Gather multiple data v...
NDT Benefits <ul><li>End-user based view of network </li></ul><ul><li>Can identify configuration problems </li></ul><ul><l...
NDT methodology <ul><li>Identify specific problem(s) that affect end users </li></ul><ul><li>Analyze problem to determine ...
IEEE 802.11 (WiFi) Detection <ul><li>Detect when host is connected via wireless (wifi) link </li></ul><ul><ul><li>Radio si...
Faulty Hardware/Link Detection <ul><li>Detect non-congestive loss due to </li></ul><ul><ul><li>Faulty NIC/switch interface...
Full/Half Link Duplex setting <ul><li>Detect half-duplex link in E2E path </li></ul><ul><ul><li>Identify when throughput i...
Normal congestion detection <ul><li>Shared network infrastructures will cause periodic congestion episodes </li></ul><ul><...
Upcoming SlideShare
Loading in …5
×

A Web Based Network Monitoring Tool

913 views

Published on

  • Be the first to comment

  • Be the first to like this

A Web Based Network Monitoring Tool

  1. 1. Developing the Web100 Based Network Diagnostic Tool (NDT) Internet2 piPEs Tutorial Rich Carlson [email_address]
  2. 2. Demo <ul><li>http://ndt-newyork.abilene.ucaid.edu:7123 </li></ul>
  3. 3. Normal operation in campus
  4. 4. Duplex Mismatch Detected
  5. 5. Low throughput from remote host
  6. 6. Increase TCP buffer size
  7. 7. Motivation for work <ul><li>Measure performance to users desktop </li></ul><ul><li>Develop “single shot” diagnostic tool that doesn’t use historical data </li></ul><ul><li>Combine numerous Web100 variables to analyze connection </li></ul><ul><li>Develop network signatures for ‘typical’ network problems </li></ul>
  8. 8. Web100 Project <ul><li>Joint PSC/NCAR project funded by NSF </li></ul><ul><li>‘First step’ to gather TCP data </li></ul><ul><ul><li>Kernel Instrument Set (KIS) </li></ul></ul><ul><li>Requires patched Linux kernel </li></ul><ul><li>Geared toward wide area network performance </li></ul><ul><li>Future steps will automate tuning to improve application performance </li></ul>
  9. 9. Web Based Performance tool <ul><li>Operates on Any client with a Java enabled Web browser </li></ul><ul><li>What it can do </li></ul><ul><ul><li>Positively state if Sender, Receiver, or Network is operating properly </li></ul></ul><ul><ul><li>Provide accurate application tuning info </li></ul></ul><ul><ul><li>Suggest changes to improve performance </li></ul></ul>
  10. 10. Web base Performance tool <ul><li>What it can’t do </li></ul><ul><ul><li>Tell you where in the network the problem is </li></ul></ul><ul><ul><li>Tell you how other servers perform </li></ul></ul><ul><ul><li>Tell you how other clients will perform </li></ul></ul>
  11. 11. Internet2 piPEs Project <ul><li>Develop E2E measurement infrastructure capable of finding network problems </li></ul><ul><li>Tools include </li></ul><ul><ul><li>BWCTL: Bandwidth Control wrapper for NLANR Iperf </li></ul></ul><ul><ul><li>OWAMP: One-Way Active Measurement </li></ul></ul><ul><ul><li>NDT: Network Diagnostic Tool </li></ul></ul>
  12. 12. piPEs Integration
  13. 13. Bottleneck Link Detection <ul><li>What is the slowest link in the end-2-end path? </li></ul><ul><ul><li>Monitors packet arrival times using libpcap routine </li></ul></ul><ul><ul><li>Use TCP dynamics to create packet pairs </li></ul></ul><ul><ul><li>Quantize results into link type bins (no fractional or bonded links) </li></ul></ul><ul><li>Cisco URP grant work </li></ul>
  14. 14. Duplex Mismatch Detection <ul><li>Developed analytical model to describe how Ethernet responds (no prior art?) </li></ul><ul><li>Expanding model to describe UDP and TCP flows </li></ul><ul><li>Develop practical detection algorithm </li></ul><ul><li>Test models in LAN, MAN, and WAN environments </li></ul><ul><li>NIH/NLM grant funding </li></ul>
  15. 15. Future enhancements <ul><li>WiFi detection </li></ul><ul><li>Faulty Hardware detection </li></ul><ul><li>Congestion modification </li></ul><ul><li>Full/Half duplex detection </li></ul>
  16. 16. Additional Functions and Features <ul><li>Provide basic tuning information </li></ul><ul><li>Basic Features </li></ul><ul><ul><li>Basic configuration file </li></ul></ul><ul><ul><li>FIFO scheduling of tests </li></ul></ul><ul><ul><li>Simple server discovery protocol </li></ul></ul><ul><ul><li>Federation mode support </li></ul></ul><ul><ul><li>Command line client support </li></ul></ul><ul><li>Created sourceforge.net project page </li></ul>
  17. 17. Availability <ul><li>Open Source Development project </li></ul><ul><ul><li>http://www.sourceforge.net/projects/ndt </li></ul></ul><ul><li>Tools available via from </li></ul><ul><ul><li>http://e2epi.internet2.edu/ndt/download.html </li></ul></ul><ul><ul><li>Contains source code </li></ul></ul><ul><li>Email discussion list [email_address] </li></ul><ul><ul><li>Goto http://e2epi.internet2.edu/ndt web site and click </li></ul></ul><ul><ul><ul><li>ndt-users – General discussion on NDT tool </li></ul></ul></ul><ul><ul><ul><li>ndt-announce – Announcements on new features </li></ul></ul></ul>
  18. 18. NDT Flow Chart Client Web Browser Java Applet NDT - Server Web Server Testing Engine Child Test Engine Spawn child Well Known NDT Server Web Request Redirect msg Web Page Request Web page response Test Request Control Channel Specific test channels
  19. 19. NDT servers
  20. 20. Results and Observations <ul><li>Changing desktop effects performance </li></ul><ul><li>Faulty Hardware identification </li></ul><ul><li>Mathis et.al formula fails </li></ul>
  21. 21. <ul><li>10 Mbps NIC </li></ul><ul><ul><li>Throughput 6.8/6.7 Mbps send/receive </li></ul></ul><ul><ul><li>RTT 20 ms </li></ul></ul><ul><ul><li>Retransmission/Timeouts 25/3 </li></ul></ul><ul><li>100 Mbps NIC </li></ul><ul><ul><li>Throughput 84/86 Mbps send/receive </li></ul></ul><ul><ul><li>RTT 10 ms </li></ul></ul><ul><ul><li>Retransmission/Timeouts 0/0 </li></ul></ul>Different Host, Same Switch Port
  22. 22. <ul><li>100 Mbps FD </li></ul><ul><li>Ave Rtt %loss </li></ul><ul><ul><li>5.41 0.00 </li></ul></ul><ul><ul><li>1.38 0.78 </li></ul></ul><ul><ul><li>6.16 0.00 </li></ul></ul><ul><ul><li>14.82 0.00 </li></ul></ul><ul><li>10 Mbps </li></ul><ul><ul><li>72.80 0.01 </li></ul></ul><ul><ul><li>8.84 0.75 </li></ul></ul><ul><li>Speed </li></ul><ul><ul><li>94.09 </li></ul></ul><ul><ul><li>22.50 </li></ul></ul><ul><ul><li>82.66 </li></ul></ul><ul><ul><li>33.61 </li></ul></ul><ul><ul><li>6.99 </li></ul></ul><ul><ul><li>7.15 </li></ul></ul>LAN Testing Results
  23. 23. <ul><li>100 Mbps FD </li></ul><ul><li>Ave Rtt %loss loss/sec </li></ul><ul><ul><li>5.41 0.00 0.03 </li></ul></ul><ul><ul><li>1.38 0.78 15.11 </li></ul></ul><ul><ul><li>6.16 0.00 0.03 </li></ul></ul><ul><ul><li>14.82 0.00 0.10 </li></ul></ul><ul><li>10 Mbps </li></ul><ul><ul><li>72.80 0.01 0.03 </li></ul></ul><ul><ul><li>8.84 0.75 4.65 </li></ul></ul><ul><li>Speed </li></ul><ul><ul><li>94.09 Good </li></ul></ul><ul><ul><li>22.50 Bad NIC </li></ul></ul><ul><ul><li>82.66 Bad reverse </li></ul></ul><ul><ul><li>33.61 Congestion </li></ul></ul><ul><ul><li>6.99 Good </li></ul></ul><ul><ul><li>7.15 Bad NIC </li></ul></ul>LAN Testing Results
  24. 24. Mathis et.al Formula fails <ul><li>Estimate = (K * MSS) / (RTT * sqrt(loss)) </li></ul><ul><ul><li>old-loss = (Retrans - FastRetran) / (DataPktsOut - AckPktsOut) </li></ul></ul><ul><ul><li>new-loss = CongestionSignals / PktsOut </li></ul></ul><ul><li>Estimate < Measured (K = 1) </li></ul><ul><ul><li>old-loss 91/443 (20.54%) </li></ul></ul><ul><ul><li>new-loss 35/443 (7.90%) </li></ul></ul>
  25. 25. NDT Hardware Requirements <ul><li>Minimum requirements </li></ul><ul><ul><li>500 MHz Intel or AMD CPU </li></ul></ul><ul><ul><li>64 MB of RAM </li></ul></ul><ul><ul><li>Fast Ethernet </li></ul></ul><ul><li>Buying something now </li></ul><ul><ul><li>2 GHz or better processor </li></ul></ul><ul><ul><li>256 MB of RAM </li></ul></ul><ul><ul><li>Gigabit Ethernet </li></ul></ul><ul><li>Disk space for executables and log files </li></ul><ul><ul><li>No disk I/O involved during test </li></ul></ul>
  26. 26. NDT Software Requirements <ul><li>Web100 enhancements </li></ul><ul><ul><li>Linux kernel </li></ul></ul><ul><ul><li>User library </li></ul></ul><ul><li>Other 3 rd party SW needed to compile source </li></ul><ul><ul><li>Java SDK </li></ul></ul><ul><ul><li>pcap library </li></ul></ul><ul><ul><li>Client uses Java JRE (beware of version mismatch) </li></ul></ul><ul><li>NDT source file </li></ul><ul><ul><li>Test engine (web100srv) requires root authority </li></ul></ul>
  27. 27. Recommended Settings <ul><li>There are no settings or options for the Web based java applet. </li></ul><ul><ul><li>It allows the user to run a fixed set of tests for a limited time period </li></ul></ul><ul><li>Test engine settings </li></ul><ul><ul><li>Turn on admin view ( -a option) </li></ul></ul><ul><ul><li>If multiple network interfaces exist use –i option to specify correct interface to monitor ( ethx ) </li></ul></ul><ul><li>Simple Web server (fakewww) </li></ul><ul><ul><li>Use –l fn option to create log file </li></ul></ul>
  28. 28. Potential Risks <ul><li>Non-standard kernel required </li></ul><ul><ul><li>GUI tools can be used to monitor other ports </li></ul></ul><ul><li>Public servers generate trouble reports from remote users </li></ul><ul><ul><li>Respond or ignore emails </li></ul></ul><ul><li>Test streams can trigger IDS alarms </li></ul><ul><ul><li>Configure IDS to ignore NDT server </li></ul></ul>
  29. 29. Possible Alternatives <ul><li>Other tools that can perform client testing </li></ul><ul><ul><li>Several web sites offer the ability for a user to check PC upload/download speed. </li></ul></ul><ul><ul><li>Internet2/Surfnet Detective </li></ul></ul><ul><ul><li>NCSA Advisor </li></ul></ul>
  30. 30. <ul><li>Supplemental information </li></ul>
  31. 31. NDT’s Web100 Based Approach <ul><li>Simple bi-directional test to gather E2E data </li></ul><ul><li>Gather multiple data variables from server </li></ul><ul><li>Compare measured performance to analytical values </li></ul><ul><li>Translate network values into plain text messages </li></ul><ul><li>Geared toward campus area network </li></ul>
  32. 32. NDT Benefits <ul><li>End-user based view of network </li></ul><ul><li>Can identify configuration problems </li></ul><ul><li>Can identify performance bottlenecks </li></ul><ul><li>Provides some ‘hard evidence’ to users and network administrators to reduce finger pointing </li></ul><ul><li>Doesn’t rely on historical data </li></ul>
  33. 33. NDT methodology <ul><li>Identify specific problem(s) that affect end users </li></ul><ul><li>Analyze problem to determine ‘Network Signature’ for this problem </li></ul><ul><li>Provide testing tool to automate detection process </li></ul>
  34. 34. IEEE 802.11 (WiFi) Detection <ul><li>Detect when host is connected via wireless (wifi) link </li></ul><ul><ul><li>Radio signal changes strength </li></ul></ul><ul><ul><li>NICs implement power saving features </li></ul></ul><ul><ul><li>Multiple standards (a/b/g/n) </li></ul></ul><ul><li>Some data has been collected </li></ul>
  35. 35. Faulty Hardware/Link Detection <ul><li>Detect non-congestive loss due to </li></ul><ul><ul><li>Faulty NIC/switch interface </li></ul></ul><ul><ul><li>Bad Cat-5 cable </li></ul></ul><ul><ul><li>Dirty optical connector </li></ul></ul><ul><li>Preliminary works shows that it is possible to distinguish between congestive and non-congestive loss </li></ul>
  36. 36. Full/Half Link Duplex setting <ul><li>Detect half-duplex link in E2E path </li></ul><ul><ul><li>Identify when throughput is limited by half-duplex operations </li></ul></ul><ul><li>Preliminary work shows detection possible when link transitions between blocking states </li></ul>
  37. 37. Normal congestion detection <ul><li>Shared network infrastructures will cause periodic congestion episodes </li></ul><ul><ul><li>Detect/report when TCP throughput is limited by cross traffic </li></ul></ul><ul><ul><li>Detect/report when TCP throughput is limited by own traffic </li></ul></ul>

×