Linaro
Connect,
Hong Kong
March 2013
July 2013
LCE13
LNG Testing and Benchmarking
connect.linaro.org
Agenda
● Discussion on how to enhance LAVA framework for
networking applications.
● Discussion on LNG t...
connect.linaro.org
● Why: Make sure we don't have regression in functionality and
performance. Regression could be introdu...
connect.linaro.org
LNG Focus Areas
● Big Endian
● Foundation
○ LNG Kernel (LSK + Hugepage + NOHZ_FULL + RT...)
● Data Plan...
connect.linaro.org
LNG Test Cases
LNG Tests/Benchmarks Tracking Sheet
● Functional Testing
○ Additional patches carried in...
connect.linaro.org
LNG Test Cases
Measure:
● Throughput
● Number of dropped packets
● Latency
● CPU utilization
● Power
Ot...
connect.linaro.org
LNG Test Bench (Network Traffic)
Tests with network traffic,
especially for data plane
applications.
DU...
connect.linaro.org
LNG Test Bench (Network Traffic)
Some ports directly
connected between
Traffic Generator and
DUT.
Some ...
connect.linaro.org
LNG Test Bench (Network Traffic)
● Resource Management
○ Isolated data path between
traffic generator a...
connect.linaro.org
LNG Test Bench (w/o Network Traffic)
Tests without network
traffic, primarily for kernel
regression tes...
connect.linaro.org
● What are the target applications? Example code?
● Recommendations of how we should categorize
benchma...
Linaro
Connect,
Hong Kong
March 2013
Questions?
More about Linaro: http://www.linaro.org/about/
More about Linaro engineering: http://www.linaro.org/engineering/
How to j...
Upcoming SlideShare
Loading in...5
×

LCE13: LNG Testing, benchmarking, etc

972
-1

Published on

Resource: LCE13
Name: LNG Testing, benchmarking, etc
Date: 12-07-2013
Speaker: Zi Shen LimDate: Friday July 12, 2013
Video: https://www.youtube.com/watch?v=8eY08UKQ1qk

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
972
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

LCE13: LNG Testing, benchmarking, etc

  1. 1. Linaro Connect, Hong Kong March 2013 July 2013 LCE13 LNG Testing and Benchmarking
  2. 2. connect.linaro.org Agenda ● Discussion on how to enhance LAVA framework for networking applications. ● Discussion on LNG testing strategy and goals. https://lce-13.zerista.com/event/member/79674 http://pad.linaro.org/p/LCE13_LNG-Benchmarking
  3. 3. connect.linaro.org ● Why: Make sure we don't have regression in functionality and performance. Regression could be introduced by upstream or by us. ● What: Baselines and regression testing. ● How: LAVA. For example: compare LNG kernel with and without RT patchset. LNG Testing/Benchmarking
  4. 4. connect.linaro.org LNG Focus Areas ● Big Endian ● Foundation ○ LNG Kernel (LSK + Hugepage + NOHZ_FULL + RT...) ● Data Plane ○ RTE + SOC APIs What are the Key Performance Indicators?
  5. 5. connect.linaro.org LNG Test Cases LNG Tests/Benchmarks Tracking Sheet ● Functional Testing ○ Additional patches carried in LNG kernel trees. Some have wider impact (e.g. RT), some may be backported features (e.g. hugepage, nohz_full). ○ Need coverage for additional features, as well as kernel in general post integration. ● Performance Benchmarking ○ Some features improve certain performance characteristics (e.g. hugepage, nohz_full). ○ Need to characterize those and make sure they continue to work well across releases.
  6. 6. connect.linaro.org LNG Test Cases Measure: ● Throughput ● Number of dropped packets ● Latency ● CPU utilization ● Power Other Variables: ● Number of cores ● Core frequency ● Number of ports Test Variations: ● Accelerated using Dataplane APIs. ● Inside KVM guest. ○ Guest-network. ○ Guest-guest. ● Kernel with different patchsets (nohz_full, rt, etc.). Example: network packet forwarding. Compare against x86.
  7. 7. connect.linaro.org LNG Test Bench (Network Traffic) Tests with network traffic, especially for data plane applications. DUT: member platforms, multi-10G, multi-1G ports. Using x86 machines as traffic generator. Could also be DUT for comparative benchmarking.
  8. 8. connect.linaro.org LNG Test Bench (Network Traffic) Some ports directly connected between Traffic Generator and DUT. Some ports connected through switch. Can even repurpose some DUT as Traffic Generator.
  9. 9. connect.linaro.org LNG Test Bench (Network Traffic) ● Resource Management ○ Isolated data path between traffic generator and DUT. ○ Automated / programmatic way to access traffic generator, switch, etc. ○ Efficient resource scheduling. ● Test & Resource Coordination ○ Start app on DUT, start traffic, stop traffic, stop app on DUT. ● Test Report & Database ○ Performance trends, passing criteria, statistics ○ Visualization, alerts.
  10. 10. connect.linaro.org LNG Test Bench (w/o Network Traffic) Tests without network traffic, primarily for kernel regression testing. ● Resource Management ○ Automated / programmatic way to access traffic generator, switch, etc. ○ Efficient resource scheduling. ● Test Report & Database ○ Performance trends, passing criteria, statistics ○ Visualization, alerts.
  11. 11. connect.linaro.org ● What are the target applications? Example code? ● Recommendations of how we should categorize benchmarks? ○ By verticals? e.g. wireless, security? ○ L1-L7? ● Any additional hardware platforms to consider? ● Wrt comparative benchmarking, which platforms do we care about? ● How much do we care about industry benchmarks such as SPEC, EEMBC? ● How do we characterize success of our data plane APIs? ● In case of data plane, there are different solutions: bare metal, Linux userspace RTE, etc. How do we showcase LNG solution? Questions
  12. 12. Linaro Connect, Hong Kong March 2013 Questions?
  13. 13. More about Linaro: http://www.linaro.org/about/ More about Linaro engineering: http://www.linaro.org/engineering/ How to join: http://www.linaro.org/about/how-to-join Linaro members: www.linaro.org/members
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×