Arkville is an FPGA-based DPDK packet conduit developed by Atomic Rules LLC that provides high-speed data transfer between DPDK applications running on a GPP and FPGA-based accelerators. It consists of an open-source DPDK Poll Mode Driver software and a proprietary FPGA RTL IP core. Arkville delivers over 150Gbps throughput and 1.25us latency while eliminating CPU overhead through zero-copy of packets between the NIC and application memory. It is supported on major FPGA boards and can be used to accelerate DPDK workloads in smart NICs, network appliances, and other solutions requiring efficient GPP-FPGA communication.

1. Arkville
DPDK Acceleration
Shep Siegel, CTO
Atomic Rules LLC
1©2017 Atomic Rules LLC
Shepard.Siegel@atomicrules.com
Arkville DPDK Acceleration
2017-10-08

2. AR Background
• Providing FPGA Design Services since 2008
– Contributions to numerous Open-Source Projects
– Corporate Member of The Linux Foundation
• Began augmenting services with IP Core products
– UDP Offload Engine (2014)
– Arkville DPDK-Aware Data Mover (2016)
– TimeServo FPGA System Timer (2017)
• Differentiation with Tools and Methodology
• Broad and Agnostic set of Partners
• Growing list of Recurring Customers and Clients
2©2017 Atomic Rules LLC

3. Arkville: A DPDK Packet Conduit
3©2017 Atomic Rules LLC

4. Arkville: Why?
• DPDK First
– Not an Afterthought or Adaptation
– Both Higher Throughput and Lower Latency
• DPDK Aware
– Push GPP cycles to specialized FPGA RTL gates
– Zero-Copy GPP Driver = Lowest Possible Overhead
• DPDK Acceleration
– Certain workloads are better in RTL gateware
– Arkville enables a Software-First approach
4©2017 Atomic Rules LLC

5. Arkville: Where?
• Arkville is used as a building-block component
in products and solutions such as
– Smart-NIC Devices
– Network Appliances
– DPDK Accelerators
• Anywhere there is the need to efficiently
exchange data between DPDK-mbufs and AXI
FPGA gates
5©2017 Atomic Rules LLC

6. Smart-NIC
6©2017 Atomic Rules LLC 6
Any
Rate
ENET
MAC
NIC
H/W Offload
RTL IP
Arkville
AXI HW
Arkville
DPDK PMD
SW
DPDK Applications
GPP SW
PCIe
FPGA
GPP
AR Arkville HW/SW
DPDK
SW API
AXI
HW API

7. Network Appliance
7©2017 Atomic Rules LLC 7
Any
Rate
ENET
MAC
Appliance
RTL IP
Arkville
AXI HW
Arkville
DPDK PMD
SW
Appliance
GPP SW
PCIe
FPGA
GPP
AR Arkville HW/SW
DPDK
SW API
AXI
HW API

8. DPDK Accelerator
8©2017 Atomic Rules LLC 8
AXI Accelerator RTL IP
from HLS, OpenCL, BSV
Arkville
AXI HW
Arkville
DPDK PMD
SW
DPDK Applications or
fd.io/VPP Nodes
GPP SW
PCIe
FPGA
GPP
AR Arkville HW/SW
DPDK
SW API
AXI
HW API

9. Arkville: How?
• A Software and Gateware Combination:
– Arkville 17.08 DPDK Poll Mode Driver (DPDK PMD)
• Open-Source BSD from The Linux Foundation
• http://dpdk.org/doc/guides/nics/ark.html
– Arkville 17.08 FPGA RTL IP Core (AXI RTL IP)
• Atomic Rules sells named-project and site-based licenses
• http://www.atomicrules.com/arkville
• Arkville performs the data-mover work for
products and solutions that require FPGA/GPP
communication
9©2017 Atomic Rules LLC

10. High-Level Block Diagram
10©2017 Atomic Rules LLC

11. Over 150 Gbps Throughput
11©2017 Atomic Rules LLC

12. 1.25 us RTT/2 Latency
12©2017 Atomic Rules LLC

13. About Arkville Latency
• By Itself, Arkville contributes just 100ns to the
ingress/egress latency budget
• Software: Arkville DPDK PMD:
– Constant Time 20ns per packet (e5-2630v4)
– Zero-Copy – No GPP Cycles moving packet data!
• Hardware: Arkville AXI RTL IP:
– 20 Cycles at 250 MHz = 80ns + 4ns/64B
– Deterministic, store-and-forward
13©2017 Atomic Rules LLC

14. No memcpy()
• Arkville is unique among DPDK Poll Mode
Drivers in that packet data is landed zero-copy
exactly in user-land memory
– Zero CPU cycles are used by net/ark PMD for
packet data movement
– Result is a constant O(1) time for all packets that
fit in a single mbuf (See next slide)
• More CPU cycles for your application
• Fewer CPU cores needed for same amount of work
14©2017 Atomic Rules LLC

15. Zero-Copy DPDK Driver
15©2017 Atomic Rules LLC

16. Zero Packets Dropped
16©2017 Atomic Rules LLC

17. Questions / Feedback
• Is FPGA acceleration of packet-based DPDK
workloads of interest?
• Are there specific challenges you are facing
that perhaps Arkville will help solve?
• How do you measure success?
• Do you have a target platform in mind?
• How would you like to get started?
17©2017 Atomic Rules LLC

18. Summary and Thank You!
18©2017 Atomic Rules LLC
• Arkville is a GPP/FPGA DPDK-Aware Conduit
– Software is an Open-Source DPDK Poll Mode Driver
– RTL IP Core Gateware is sold by Atomic Rules
• Named-Project and Site-Based Licenses are Available
• Arkville is Agnostic
– Support for Contemporary FPGA Devices
– Support for COTS and Custom FPGA Boards
• Arkville is Supported
– Ongoing developments to the DPDK/LF Community
– AR offers both HW and SW services and support

19. Roadmap: 17.08
19©2017 Atomic Rules LLC
• Shipped: August 2017
• Support for DPDK 17.08
• Added Support for Vivado 2017.2
• Examples include Atomic Rules TimeServo[1]
– Timestamps from MAC to DPDK metadata
[1] TimeServo is a separate IP product that may be purchased
from Atomic Rules.

20. Roadmap: 17.11
20©2017 Atomic Rules LLC
• Scheduled: November 2017
• Support for DPDK 17.11
• Support for Vivado 2017.3
• Selectable 4/8/16 TX-RX Queue-Pairs
– Next-Generation MBUF Prefetcher
– Reduced FPGA Logic Area
– Still Zero-Copy and Low-microsecond Twire-to-user
• Expanded DPDK examples

21. Backup Material
21©2017 Atomic Rules LLC

22. Core Beliefs and Axioms
22©2017 Atomic Rules LLC
• Our Customer’s Success is Key
• Separation of Concerns
• Divide and Conquer
• Automate or Die
• Write Things Once
• Interface Before Implementation
• Functional Correctness First
• Performance Correctness Improved Iteratively
• Components Must Compose
• Components Must Work as Expected
• IP Should be Portable, Vendor-Agnostic if possible

23. Partner Roster
23©2017 Atomic Rules LLC
• 25G / 50G Ethernet Consortium
• 25-50-100 Ethernet Alliance
• Accellera/OCP-IP Community Member
• Amazon F1 Instance Partner
• ARM Connected Community Member
• BittWare Solution Partner
• Bluespec Technology Partner
• DPDK Project Corporate Member
• FPGA, FCCM and FPL ‘F’ Conference Sponsors
• Intel Network Builders
• Linux Foundation Corporate Member
• MathWorks Connections Partner
• NetFPGA Infrastructure Developer
• OpenCPI Infrastructure Developer
• P4 Language Consortium Member
• PCI-SIG Corporate Member
• VITA Trade Association Member
• Xilinx Alliance Member Partner

24. 100 GbE DPDK Offload
24©2017 Atomic Rules LLC

25. Arkville IPI Component
25©2017 Atomic Rules LLC

26. Arkville Evaluation
26©2017 Atomic Rules LLC
• New Performance Evaluation from Atomic Rules…
• Evaluate Arkville on-site with your own system components
– Measure your own actual system-level performance
• Atomic Rules provides pre-compiled bitstreams for your
FPGA platform
– 100 GbE (x1) and 10 GbE (x4) example designs
• Download trusted, unmodified DPDK from dpdk.org
• Run DPDK “testpmd” unmodified to observe
– Packet Throughput, Latency, Packet Loss
– Compare results to DPDK Labs reports
• Run supplied AR DPDK examples for deeper insights
• Run your own DPDK programs on the example designs