Optimizing Network Performance for Amazon EC2 Instances (CMP308-R1) - AWS re:Invent 2018

© 2018, Amazon Web Services, Inc. or its affiliates. All rights reserved.
Optimizing Network Performance
for Amazon EC2 Instances
C M P 3 0 8
Vishvesh Sahasrabudhe
Product Manager, EC2 Networking
Nick Matthews
Principal Solutions Architect
nickpowpow

© 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
Network performance
Packets in Packets out
?

Network performance: Let’s look
Packets in Packets out
?

What to expect
Performance concepts
Networking performance features
• Nitro
• Elastic Network Adapter
Architectural performance
• Bandwidths and limits
Testing

Basic concepts
Bandwidth Latency Throughput Jitter

Bandwidth delay product
10 Gbps
500 milliseconds
Data

10 Gbps
500 milliseconds
1 second RTT
ACK
(To make math easy)

Data Data DataData
10 Gbps
1 second RTT
Bandwidth per second = How much
data is sent in one round trip
1. Larger packets (MTU)
2. Send more packets (TCP window size)

TCP and latency
Latency is a major factor for:
• Cross-Region connectivity
• Instance-to-instance for clustering (HPC)
• Amazon ElastiCache, memcached
Tune it:
• Set larger MTUs
• TCP tuning
• Increase network efficiency
Make friends with physics:
• AWS Region choice: 10’s of ms
• Availability Zone affinity: milliseconds
• Placement groups: <1 ms
Options

Packets per second (PPS) and
Maximum transmission unit (MTU)
Each packet has processing overhead
• Small packets such as real time systems or transactions
Large packets increase the overall performance
• Jumbo MTU of 9001 available within VPC or VPC peers
1448 B
Payload
8949 B Payload
Jumbo MTUs increase the usable data per packet

0
20
40
60
80
100
0% 2% 4% 6% 8% 10%
Loss Rate
Impact of loss on TCP throughput

Loss is visible as TCP retransmissions
$ netstat -s | grep retransmit
58496 segments retransmitted
52788 fast retransmits
135 forward retransmits
3659 retransmits in slow start
392 SACK retransmits failed

Example application
High transaction rate HTTP service
Jack Jill

Test setup
• 1 test server instance, 1 test client instance
• 80 ms RTT
• HTTP, not HTTPS
• 1500 MTU
• 200k requests for a 10k object
$ ab -n 200000 -c 200 http://server/10k
Goal: Minimize latency, use TCP tuning to speed it up

Results – HTTP application with 200k requests
Test P50 latency Avg BW
Initial congestion window—3 packets 321 ms 12.550 Mbps
79%
Increase!

Examples of TCP tuning
Initial congestion window:
ip route change 10.16.16.0/24 dev eth0
proto kernel scope link initcwnd 16
TCP receive window:
sysctl net.ipv4.tcp_rmem
TCP congestion control algorithms:
net.ipv4.tcp_available_congestion_control
TCP retransmission timer:
ip route change 10.16.16.0/24 dev eth0 proto kernel
scope link rto_min 50ms

Nitro system
• Lightweight hypervisor
• Nitro card
• Nitro security chip
Benefits of Nitro – Enhanced networking
• Over 20x increase in PPS performance
• Reduces instance-to-instance latencies
• Support SR-IOV with PCIe passthrough, to get the hypervisor out of the way and for consistent
performance
• Intel ixgbvef and Amazon Elastic Compute Cloud (Amazon EC2) Elastic Network Adapter (ENA)
Networking on AWS

Benefits of Enhanced Networking
(Latency)
0
50
100
150
200
250
300
1 2
RoundTripLatency(µs)
Series1 Series2 Series3 Series4

Evolution of Nitro
CC2 instance (pre-Nitro)
Xen hypervisor
EC2
Instance
EC2
Instance
EC2
instanceXen
Dom0

Evolution of Nitro
C3 instance (2013)
Xen hypervisor
EC2
Instance
EC2
Instance
EC2
instanceXen
Dom0 ixgbevf
Intel 82599

Evolution of Nitro
C4 instance (2015)
Xen hypervisor
EC2
Instance
EC2
Instance
EC2
instanceXen
Dom0 ixgbevf
Intel 82599
EBS Optimized

Evolution of Nitro
C5 instance (2017)
Nitro hypervisor
EC2
Instance
EC2
Instance
EC2
instance
ENA

Evolution of Nitro
bare metal instances (2018)
EC2
instance

ENA driver capabilities
• Future proof: support for up to 400 Gbps
• Increased queues per device
• Lower latency and jitter
• Broad operating system AMI support
• Linux—Amazon Linux, Amazon Linux 2, RHEL 7.4+, SLES 12+, Ubuntu 14.04, 16.04, 18.04,
CentOS 7 (1711), and in the Linux kernel since 4.9
• Windows Server 2008 R2, 2012, 2012 R2, 2016, 2019
• FreeBSD 11
• DPDK support starting in 16.04, PMD support
• Open Source: https://github.com/amzn/amzn-drivers
ENA v2.0 – Latest release in upstream kernel
• Low Latency Queues – Improved average and tail latencies
• Rx Checksum Offload – Reduces host CPU usage

Benefits of Nitro enhanced networking
(latency)
0
50
100
150
200
250
300
1 2
Series1 Series2 Series3 Series4 Series5

(Network and Storage bandwidth)
0
20
40
60
80
100
120
1 2 3 4
Gbps
Series1 Series2

(instance bandwidth)
T3 instances – Up to 5 Gbps network
performance
Smaller sizes of C5, M5, R5 - up to 10
Gbps network performance
Larger instance sizes have sustained
10 or 25 Gbps
NEW – Smaller sizes of C5n – up to 25
Gbps network performance
NEW – C5n instances have sustained
50 or 100 Gbps
Model vCPU
Mem
(GiB)
Network
Performance
(Gbps)
c5.large 2 4 Up to 10
c5.xlarge 4 8 Up to 10
c5.2xlarge 8 16 Up to 10
c5.4xlarge 16 32 Up to 10
c5.9xlarge 36 72 10
c5.18xlarge 72 144 25
Model vCPU
Mem
(GiB)
Network
Performance
(Gbps)
t3.nano 2 6Up to 5
t3.micro 2 12Up to 5
t3.small 2 24Up to 5
t3.medium 2 24Up to 5
t3.large 2 36Up to 5
t3.xlarge 4 96Up to 5
t3.2xlarge 8 192Up to 5
Model vCPU
Mem
(GiB)
Network
Performance
(Gbps)
c5n.large 2 4 Up to 25
c5n.xlarge 4 8 Up to 25
c5n.2xlarge 8 16 Up to 25
c5n.4xlarge 16 32 Up to 25
c5n.9xlarge 36 72 50
c5n.18xlarge 72 144 100

• Microservices architecture – 100s of microservice
• Request cache – persist data for request duration
• 3 zones/region – read local, write regional
• Millions of requests per second requiring high packet rate performance at
cache layer
• Average read latency < 250 µS
High performance caching

How did Netflix benefit?
Migration to Nitro instances
• M4 to C5
Improved PPS lead to reduction in instance count
• 10 m4.large 1 c5.xlarge
Improved latency performance while reducing costs

AMI Check
• ixgbevf enhanced networking:
aws ec2 describe-image-attribute --image-id ami_id
--attribute sriovNetSupport
• ENA enhanced networking:
aws ec2 describe-image-attribute --image-id ami_id
--attribute enaSupport
ENA support is required for AMIs launching on Nitro based instance
types (C5, M5, R5, T3…)

Beyond enhanced networking with DPDK
Intel Data Plane Development Kit (DPDK) is a set of C libraries and
drivers for fast packet processing
DPDK provides:
• Lower latency due to kernel and TCP stack bypass
• More control of packet processing
• Lower CPU overhead

Non-DPDK application
Nitro hypervisor
EC2
Instance
EC2
Instance
Instance
Kernel
App

DPDK application
Nitro hypervisor
EC2
Instance
EC2
Instance
Instance
Kernel
DPDK app

DPDK on AWS
Supported on enhanced networking instances:
• Intel ixgbevf and AWS Elastic Network Adapter (ENA)
DPDK can reduce latency while increasing control of packet
processing
• Kernel and TCP stack bypass, and PMD instead of interrupts reduce
latency
• DPDK does not change any underlying limits or maximums

Network performance scenarios
The sources and destinations matter:
• Within a placement group
• Over a VPN connection
• Using AWS Direct Connect
• Using Elastic Load Balancing
• Accessing AWS services such as Amazon Simple Storage
Service (Amazon S3)

VPC Performance
No limits per VPC
Availability Zone Availability Zone
No limits for any
Availability Zone or subnet
IGW
No limits for any Internet
Gateway
Bandwidth Limits: NAT gateway
Each NAT Gateway can
sustain 10 Gbps, and
scales horizontally
beyond 10 Gbps

VPC Peering Performance
No limits for VPC peering within a Region
Internet gateway
Bandwidth Limits:
Internet gateway
VPC peering

Placement group
Instance Bandwidth Limits
Availability Zone B
Availability Zone A
Amazon S3
Placement Group – 10 Gbps
flows
Placement group:
5 Gbps aggregate per instance
with 5 Gbps flows
25 Gbps
25Gbps
Everything else:
Between instances:
25 Gbps within the VPC or peered
VPC
To Amazon S3:
25 Gbps to VPC Endpoints and
public IPs in the same Region

Virtual private gateway performance
~1.25 Gbps aggregate per VPN tunnel
Availability Zone
Bandwidth limits:
Virtual private
gateway
Virtual private network (VPN)
AWS Direct Connect
AWS Direct Connect is only limited
by physical port speeds

Virtual private gateway performance
5 Gbps limit for egress traffic
per instance
Availability Zone
Bandwidth limits:
Virtual private
gateway
Virtual private network (VPN)
AWS Direct Connect

Aviatrix Insane Mode
25G Forwarder
instance
VPN
ENI Route
Up to 5 Gbps Up to 5 Gbps
Cumulatively
up to 25 Gbps
High-speed VPN
C5 PPS improvements
• Forwarder performance

Network Load Balancer performance
Auto Scaling Group
Network Load
Balancer
Network Load Balancer
• L4 load balancer
• TCP
• High performance
• Availability Zone locality
• One IP per Availability Zone
Bandwidth limits:
Several gigabits of capacity at
launch
Scales horizontally

AWS HyperPlane performance
AWS Region
VPC A VPC B VPC A VPC B VPC A VPC B
AWS HyperPlane
Attachments

Distributed load testing (BWMG), executed in a loop
bees attack --url '<NLB>' --number 10000000 --concurrent 100000 –keepalive
Performance graph shows no errors, and content was served fine

Benefits of testing
Many potential performance factors:
• Flow location, type, size
• Instance family, type
• Application profile
Packet sizes, protocols, CPU/memory/disk usage
AWS recommendation: test it yourself

Performance testing: How?
With your application and environment
• Load testing
• Transaction testing
• “Race track numbers”
Benchmark testing
• Scenario testing: intra-VPC, internet traffic, VPN
• Bandwidth, latency, jitter, throughput, packets per second

Testing tips
Use multiple flows
#iperf –c 192.168.1.1 –P 10
Try UDP as well
#iperf –c 192.168.1.1 –u –p <port number>
Distributed testing tools
• https://github.com/newsapps/beeswithmachineguns
Network benchmark testing
• https://aws.amazon.com/premiumsupport/knowledge-
center/network-throughput-benchmark-linux-ec2/
Operating system configuration for enhanced networking
• https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/en
hanced-networking-os.html

Summary
Understand latency and TCP dynamics
• Tune and reduce latency
Understand what AWS offers to help
• Enhanced networking, jumbo MTU, instance families
Use the right architecture
• Know the limits, distribute the load
Test it out

Thank you!
Nick Matthews
@nickpowpow
Vishvesh Sahasrabudhe

Optimizing Network Performance for Amazon EC2 Instances (CMP308-R1) - AWS re:Invent 2018

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Optimizing Network Performance for Amazon EC2 Instances (CMP308-R1) - AWS re:Invent 2018

Similar to Optimizing Network Performance for Amazon EC2 Instances (CMP308-R1) - AWS re:Invent 2018 (20)

More from Amazon Web Services

More from Amazon Web Services (20)

Optimizing Network Performance for Amazon EC2 Instances (CMP308-R1) - AWS re:Invent 2018