Improving Network Application Performance using Load Aware Libeventdev
Report
Share
•2 likes•1,103 views
Improving Network Application Performance using Load Aware Libeventdev
•2 likes•1,103 views
Report
Share
Download to read offline
Compared to load unaware packet distribution mechanisms often used in the run to completion model, an event scheduler improves core utilization and better handles dynamic traffic mixes by scheduling packets to cores according to their load. It simultaneously provides both atomicity and packet ordering. Hardware-based event schedulers can also provide low-latency inter-core communication. The libeventdev library from Data Plane Development Kit (DPDK) helps developers leverage the event scheduler model. About the presenter: Sundar Vedantham, Intel, is a Senior Technical Manager working in the Data Center Group in Allentown, PA. His research interests include network traffic and congestion management, high-speed networking, and theoretical computer models, areas in which he holds patents and has published papers, book chapter & articles. He received his Ph.D. in Computer Science in 1997 from Louisiana State University. He enjoys writing articles in English and Tamil to help improve public understanding of technical details behind the fields he has worked on and to attract young students to get into STEM fields.
Recommended
More Related Content
What's hot
What's hot(20)
Similar to Improving Network Application Performance using Load Aware Libeventdev
Similar to Improving Network Application Performance using Load Aware Libeventdev(20)
More from Michelle Holley
More from Michelle Holley(20)
Recently uploaded
Recently uploaded(20)
Improving Network Application Performance using Load Aware Libeventdev
- 1. Intel Confidential – Internal Use Only Improving Network Application performance using Load Aware Libeventdev Sundar Vedantham - June 13, 2017
- 2. 2 Agenda • Multicore Applications • Why do we need Eventdev? • Eventdev Theory • Eventdev API & Application • Code Availability – Take Away
- 3. 3 Multicore Applications • Networking applications use one of two models: • Run-to-completion • Packet ordering for pinned flows • Flow atomicity is supported • Cache locality advantage • Load imbalance could be a con.
- 4. 4 Multicore Applications • Pipeline processing • Ensures packet ordering always • Flow atomicity is supported • Cache locality advantage • Core under utilization is a con • May not be flexible to handle different traffic types
- 5. 5 Why Eventdev? • Event Scheduler addresses the shortcomings of both models. • Improves core utilization and better handles dynamic traffic mixes by scheduling packets to cores according to their load. • Simultaneously provides both atomicity and packet ordering • Hardware-based event schedulers can provide low-latency inter- core communication.
- 6. Why Eventdev? Eventdev Scheduler Rx Queue Tx Queue Core i Packet Rx Core m Packet Tx Core j Core k Firewall/Flow Classifier/Metering/ Routing Firewall/Flow Classifier/Metering/ Routing
- 7. 7 Current Status • It is a DPDK Library for scheduling events • Initially proposed by Jerin Jacob (Cavium) • Worked with Intel + NXP to evolve API to current state • Easy setup of pipeline processing • Eventdev SQM PMDs does scheduling work • Development ongoing since Nov 2016 • http://dpdk.org/browse/next/dpdk-next-eventdev/ • Community feedback / patches included in the DPDK 17.05 release
- 8. 8 Eventdev Theory • Pipelines • Components • Device • Ports • Queues • Concepts • Scheduling types • Linking queues and ports
- 9. 9 Eventdev Theory - Pipelines • Pipeline made up of “stages” • Each “stage” is a stepping-stone to the next • One action performed at each stage • Eg: Flow Classification, Firewall, ACL • Load balancing • Worker logical cores (lcore) perform all stages of work • Ensures all lcores have equal load • No bottleneck due to single worker-lcore
- 10. 10 Eventdev Theory - Pipelines RX W1 TXAtomic #1 W.. WN W1 W.. WN Atomic #2 Single Link Wi - Worker core i
- 11. 11 Eventdev Theory - Components • Device • Instance of an eventdev PMD • Can be SW or HW • SW PMD is available • Ports and Queues are part of the device
- 12. 12 Eventdev Theory - Components NIC
- 13. 13 Eventdev Theory – Components – Event Port • Port • Event ports are logical ports associated with cores • Where CPU core and eventdev interact • Enqueue / Dequeue of events • 1 port used by 1 lcore • Events dequeued from port must be returned to it again • Eventdev tracks inflight events per port
- 14. 14 Eventdev Theory – Components – Event Port NIC
- 15. 15 Eventdev Theory – Components - Queue • Queue • Each queue is a stage of a pipeline • Destination for events • Has a Scheduling type • Atomic, Ordered, Parallel • Holds events until space available in linked port
- 16. 16 Eventdev Theory – Components - Queue NIC
- 17. 17 Eventdev Theory - Concepts • Scheduling Modes • Atomic - Flow ID used to ensure flow is not on > 1 lcore • Ordered - Events reordered into ingress order • Parallel - Events allowed to continue out-of-order
- 18. 18 Eventdev Theory - Concepts • Linking queues and ports • Ports need to be mapped to queues • Linking port to queue results in events from queue arriving at the port • Ports can be linked to multiple queues • Load balancing use-case
- 19. 19 Eventdev Theory – Concepts - Linking Link: - From Port #2 - To Queue #1 NIC
- 20. 20 Eventdev Theory - Concepts • Workers • Dequeue burst of events from port • For ( events ) • Read QUEUE_ID to get last stage • Process() • Write QUEUE_ID to set next stage • Enqueue burst back to port
- 21. 21 Eventdev API – rte_event • The rte_event struct • The struct that is used in eventdev • DPDK packets are struct rte_mbuf * • Eventdev events are struct rte_event (note no *) • Size is 16 bytes • 8 bytes metadata • 8 bytes payload Event Metadata Event Pointer
- 22. 22 Eventdev API – rte_event • The rte_event struct Metadata • Enough info for PMD to schedule event to Queue ID • Uses Flow ID, Operation type, Priority Flow ID Sub event type Even t type Sched type Queue ID Priority OP type
- 23. 23 Eventdev API - Configuration • API calling order to setup a Device • Device Configuration • Queue Setup • Port Setup • Port Link • Device Start
- 24. 24 Eventdev API – Runtime usage • RX lcore • NEW event ingress • Worker lcores • FORWARD existing event • TX lcore • RELEASE event on egress • Eventdev Pipeline Sample App Demonstrates usage
- 25. 25 Eventdev SW PMD - Overview • Software PMD for doing scheduling work • Implements the eventdev API • Merged into eventdev tree • Included in the DPDK 17.05 release
- 27. 27 Eventdev SW PMD - Performance • Best Known Configuration (BKC) depends on • Number of stages • Cycles needed for work per stage • Number of workers • Adding cores is NOT always going to go faster! Performance comes from balance!
- 28. 28 Eventdev SW PMD – Performance - Prototyping • Use the eventdev_pipeline sample app • Configure to requirements • Number of stages • Cycles needed for work per stage • Number of Flows as expected • Choose realistic lcore count • Leave defaults for remaining values • Measure baseline • Use Monitoring tools • Cycles/pkt per port • Port burst distribution • “What % of bursts were 1-4 pkts?”
- 29. 29 Eventdev API – Resources • DPDK Eventdev Docs + Sample App • http://dpdk.org/doc/api/ • Download DPDK-master • Run make doc-guides-html • View Eventdev docs at • build/doc/html/guides/prog_guide/eventdev.html • Eventdev header file: lib/librte_eventdev/rte_eventdev.h • http://dpdk.org/doc/api/rte__eventdev_8h.html
- 30. 30 Eventdev Pipeline Documentation • Two docs patches • Sample App User Guide wget -O docs_sample_app.patch http://dpdk.org/dev/patchwork/patch/23800/raw/ • Eventdev Programmer Guide wget -O docs_eventdev_prog_guide.patch http://dpdk.org/dev/patchwork/patch/23801/raw/ • Build documentation • make doc-guides-html • Output files in .html format <dpdk>/build/doc/html/guides/ • http://dpdk.org/doc/guides/contributing/documentation.html#building-the-documentation