CouchConf-SF-Accelerate-User-Access-Without-Rewriting-Applications

Agenda • Architectural Trends • Tuning Guidelines • Kernel networking directions • User space networking for performance – OpenOnload Architecture and Implementation© 2011 Solarflare Communications, Inc. 2

CPU Architectural TrendsTransistors Clock Speed # CPU Cores # Other Stuff Other stuff: Memory Controllers / Graphics / IO Bus Interface … Aim is to increase performance using Fabric I/O parallel threads (cores) of execution System Memory CPU and IO interconnects are now Fabrics X86 Core with (NUMA) L1/2 Cache Shared L3 Cache © 2011 Solarflare Communications, Inc. 3

History Revisionism (IO Perspective) 100000 10000 Bandwidth Mb/s 1000 BASE-T BASE-X 100 Server IO 10 1 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008 2010 2012 Year We Need TCP We Need Offload Engines! Core Scaling! TSO/LRO and RSS, not TOE address the IO architectural implications of multi-core•2010: Bi-directional 10GE line rate with ~20% CPU Utilization @1500 MTU © 2011 Solarflare Communications, Inc. 4

Go with the flow and ...Tune for IO / NUMA / Affinity / Parallelisation Can add up to big system performance benefits System Memory (NUMA) CPU Interconnect X86 Core Core Directed Interrupts Data transfer with NUMA coherency hints I/O Device Data (Ethernet Frame) containing Information (IP or MAC headers) used by I/O device to determine the core which has been scheduled to process the data© 2011 Solarflare Communications, Inc. 5

Some Guidelines • Think about a future of 48+ cores • Remember Amdahl’s Law • Avoid dispatcher threads / unnecessary serialization – Super fast way to max out a core – Think about hardware assisted de-multiplexing • NUMA/Cache locality matters – Think about affinity of application threads AND the ISR that feeds them • Classic protocols are doing just fine in the data center – NewReno TCP/UDP great for short-fat-pipes – A whole other set of issues for long-fat-pipes (aka content delivery)© 2011 Solarflare Communications, Inc. 6

Really? – Show me some numbers2xX5677 3.4 Ghz Westmere CPU / 6GB RAM2xMemcached instances (one per socket) all threads pinned to cores on same socket2xCores on each socket dedicated to processing interrupts (shielded from app cores)NIC interrupt moderation disabled •>1 M TPS ~220us Average Response Time User Mode ~800K TPS ~330us Average Response Time Flow Affinity ~470K TPS 10Gb/s ~540us Average Response Time © 2011 Solarflare Communications, Inc. 7

Your default configuration • RSS spreads flows randomly over cores. Packets within a flow go to same core NIC receive-side scaling VNIC VNIC VNIC VNIC • Works well when – Connections are long lived – One thread per connection ISR ISR ISR ISR – And the thread happens to run on the right core – Great for network benchmarks • But mapping of flows onto cores App App App App is not deterministic. Cache locality is poor when threads core0 core1 core2 core3 each handle multiple connections – Which tends to be true for high performance server applications© 2011 Solarflare Communications, Inc. 9

Performance improved using flow affinity • Hardware deterministically directs flows to the core that the NIC receive-side scaling application thread handling the VNIC VNIC VNIC VNIC flows is running on • Solarflare drivers already provide ISR ISR ISR ISR support – manual / self-learning • Accelerate RFS with multiqueue hardware (Solarflare 2.6.39 App App App App patch) core0 core1 core2 core3 – OS can now also deterministically manage flow affinities • Follow XPS mapping work for TX priority queue mappings© 2011 Solarflare Communications, Inc. 10

Performance Improved by OpenOnload User-Space Protocol Processing • OpenOnload intercepts networking system calls and handles them at user-level • The OpenOnload library has direct access to hardware for sending and receiving packets – Eliminates system calls and interrupts from the fast path • Further reduction in overheads from – tight integration – optimised locking techniques – batching techniques© 2011 Solarflare Communications, Inc. 11

A user-level architecture?• Network hardware provides a user-safe interface which can route Ethernet packets to an application context based on flow information contained within headers © 2011 Solarflare Communications, Inc. 13

OpenOnload architecture• Stacks live in the kernel• and are mapped into user-space• Protocol processing can take place BOTH in the kernel and user-space context © 2011 Solarflare Communications, Inc. 14

Challenges for User Space Networking• Issues of handling protocol state in the application context – may persist longer than the application (send; exit ) – may require asynchronous advancement (TCP_ACK within TMO) – may be destroyed (exec) – may be shared (fork; exec ) (SOL_SOCKET/SCM_RIGHT ) – may need to be synchronized (ARP, IP ROUTE) – may be modified by a corrupt / malicious application© 2011 Solarflare Communications, Inc. 16

O/S socket is used to resolve parallel stack issues• We don’t initially know which interface a socket will use until it is bound or connected• Listening TCP sockets must be able to accept connections from any and all interfaces• UDP sockets need to send and receive over multiple interfaces © 2011 Solarflare Communications, Inc. 27

Control plane• ARP table, routing table and interface table • single copy shared by all stacks • mapped read-only into user-space • TCP sockets cache their route and layer-2 headers • validation is just a single compare of a generation counter in the ARP table (no bus-locked instructions) • any updates cause generation counters of potentially affected entries in the ARP table to be bumped • which pushes potentially affected parties to do a full lookup • synchronised with Linux via netlink sockets© 2011 Solarflare Communications, Inc. 29

Handling network events in OpenOnload Lazy Receive – improves cache locality • In the traditional architecture, an interrupt is raised which invokes the device driver, which processes notifications and passes received packets to the stack • With OpenOnload it is normal to not enable interrupts – Notifications are delivered to the notification queue, and ignored until the stack is polled – stack_poll() is invoked at user-level as needed to satisfy application operations on sockets, e.g from send(), recv(), poll(), epoll() – If a thread blocks on a socket, interrupts are enabled – When an interrupt is raised the ISR invokes stack_poll() • The stack processes network events at exactly the right time: just before (recv) or after (send) the application has touched the associated data – The stack executes on the same processor core as the application thread, minimising cache line bouncing • Timers are used to ensure TCP protocol is adhered to and that the best balance between asynchronous and application driven processing is maintained© 2011 Solarflare Communications, Inc. 30

Solarflare - OpenOnload – Vendor Plug Solarflare consistently delivers Record Breaking Latency / Message Rates – Latency: <2us TCP and UDP – Bandwidth: 40 Gbit/sec – Millions of messages per second per coreEasy to use• Comprehensive, mature OS bypass• Standard protocol acceleration:• TCP, UDP, Multicast, VLANs, Teaming• Standard APIs (POSIX sockets)• No app modifications• fork/exec/signals/options/poll/select/epoll… Standard deviation of latency was• GPLv2 (www.openonload.org) measured at 0 microseconds• Extensively used in financial trading environment At the highest tested rate of 2.3 million messages / second, the mean latency of • Increasingly used in W2.0 and HPC applications the solution was just 14 microseconds while the standard deviation of latency was measured at 2 microseconds.© 2011 Solarflare Communications, Inc. 34

CouchConf-SF-Accelerate-User-Access-Without-Rewriting-Applications

by Couchbase on Aug 04, 2011

Accessibility

Categories

Upload Details

Usage Rights

Statistics

CouchConf-SF-Accelerate-User-Access-Without-Rewriting-Applications Presentation Transcript