Dror goldenberg

1. Dror Goldenberg | VP Software Architecture
DevelopEX 2015
Software at the Speed of Light

2. 2November 11, 2015
We Live in a World of Data
Data Needs to be Accessible Always and in Real-Time
Data is Growing Exponentially
More DataMore ApplicationsMore Devices

3. 3November 11, 2015
The Challenge - Efficient Data
Processing
StoreAnalyze
Enabling the Use of Data

4. 4November 11, 2015 Figure: https://frankaxtell.bandcamp.com/track/the-speed-
of-light
100 Gigabit Ethernet
64B
Packe
t
1518B Packet
• 8M packets per second
• 123ns/packet
• 149M packets per
second
• 6.7ns/packet

5. 5November 11, 2015 Figure: https://frankaxtell.bandcamp.com/track/the-speed-
of-light
100 Gigabit Ethernet
64B
Packe
t
1518B Packet
10.8
7.6
2.9
1.7
0
2
4
6
8
10
12
OPS/PACKET
Single 1518B Packet
Software Budget
0.59
0.42
0.16
0.1
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
OPS/PACKET
Single 64B Packet
Software Budget

6. 6November 11, 2015 Figure: https://frankaxtell.bandcamp.com/track/the-speed-
of-light
Storage Trends
10
100
1,000
10,000
100,000
1,000,000
10,000,000
100,000,000
Storage Access
Latency (ns)
Storage at 100GE
• 3M I/O per second (4KB
I/Os)
• 333us per I/O
28.9
20.35
7.8
4.7
0
5
10
15
20
25
30
35
OPS/PACKET
Storage I/O Software
Budget

7. 7November 11, 2015
Addressing The Speed Challenges
Hardware Offloads
New Protocols
New APIs
Optimizations

8. 8November 11, 2015
Hardware Offloads
•Scalability to multi CPU cores
•Reduce CPU cycles per operation
•Improve CPU efficiency (NUMA,
affinity, etc)
Single Root I/O Virtualization (SRIOV)
Legacy NIC
Hypervisor
vSwitch
Trusted
VM
V
M
V
M
V
M
PF
SW VNICSW VNICSW VNICSW VNICNIC
SRIOV I/O NIC
Hypervisor
eSwitch
Trusted
VM
V
M
V
M
V
M
PF VF VF VF
NICNICNICNIC
•Direct assignment of devices to VMs
•Improve VM I/O efficiency
•Avoid hypercalls
•Virtual switch offloads
•Enable advanced protocols
Stateless offloads
Kernel bypass
RDMA
T10/DIF Offload
Erasure Code
Offload
SRIOV
Examples
Collective Offloads
Overlay Network
Offloads

9. 9November 11, 2015
New Protocols
•Reduce protocol stack overheads
•Add new functionality and
operations
NVMe Over Fabrics Block Storage Protocol
•Lightweight protocol stack
•Bypass storage stack layers
•Native mapping into storage devices
•Native multi queueing
•Low latency using RDMA (<10us
additional latency vs native)
•More HW offloads
App
Virtual
Filesystem
Block Layer
SCSI Mid Layer
iSCSI
RDMA Verbs
iSER
App
Virtual
Filesystem
Block Layer
NVMe over Fabrics
RDMA Verbs
RDMA over
Converged Ethernet
(RoCE)
Examples
NVMe over Fabrics
Overlay Networks

10. 10November 11, 2015
New APIs
•Reduce protocol stack overheads
•Enable usermode protocol stacks
•Add new functionality and
operations
•Enable offloading
UCX
DPDK
RDMA
Examples
RDMA (RoCE and InfiniBand)
•Kernel bypass
•Message semantics
•Zero copy
•Protocol offload
•Batching operations
•Polling and interrupts
App
ULP/Middleware
RDMA Verbs Provider
RDMA I/F & Infrastructure
RDMA
Aware
App

11. 11November 11, 2015
Software Optimizations
•Optimize overall system
performance
•Leverage offloads
•Amortize overheads
Batching
Polling/interrupt
Usermode Access
Avoid False Cache
Line Sharing
Data/Code
Compaction
Lock Avoidance
Examples
Scaling to multi
core
Affinitizing Flows
Polling vs Interrupt
Latency:
•Polling <1us
•Interrupt
•Bare Metal 5us
•VM 18us
X18 latency
False Cache Line Sharing
Transaction Per Second
& Shared counter on data path
•False sharing 2MTps
•Per core counter 3MTps
X1.5 throughput
Affinitizing Flows
Serving interrupts on the
transaction CPU core
•Before jittery ~50us/operation
•After ~5us/operation
X10 improvement

12. 12November 11, 2015
Release N
Release N+1
Performance Bugs
Improved Performance
Better bandwidth for midsize
packets
Performance Bug
Reduced bandwidth on
large packets
Bandwidth(Gb/s)
Improving performance on
one workload can result in
hurting performance for
other workloads

13. 13November 11, 2015
No Software Without Testing
•Performance benchmarks are critical
–Home grown
–Industry standard
•Performance regression
–Identify commits as soon as they break
performance
•Performance diagnostics
–Performance counters
•System performance tuning

14. 14November 11, 2015 Figure: https://frankaxtell.bandcamp.com/track/the-speed-
of-light
Thank You!