A Day in the Life of a Billion Packets (CPN401) | AWS re:Invent 2013
Upcoming SlideShare
Loading in...5
×
 

A Day in the Life of a Billion Packets (CPN401) | AWS re:Invent 2013

on

  • 2,377 views

In this talk, we walk through the VPC network presentation, and describe the problems we were trying to solve. Next, we walk through how these problems are traditionally solved, and why those ...

In this talk, we walk through the VPC network presentation, and describe the problems we were trying to solve. Next, we walk through how these problems are traditionally solved, and why those solutions are not scalable, cheap, or secure enough for AWS. Finally, we provide an overview of the solution that we've implemented and discuss some of the unique mechanisms that we use to ensure customer isolation.

Statistics

Views

Total Views
2,377
Views on SlideShare
2,346
Embed Views
31

Actions

Likes
4
Downloads
89
Comments
0

4 Embeds 31

https://d2si.bluekiwi.net 12
http://discourse.luma 9
http://humangas.hatenablog.jp 8
https://www.linkedin.com 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

A Day in the Life of a Billion Packets (CPN401) | AWS re:Invent 2013 A Day in the Life of a Billion Packets (CPN401) | AWS re:Invent 2013 Presentation Transcript

  • A Day in the Life of a Billion Packets Eric Brandwine, AWS Security November 14, 2013 © 2013 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc.
  • We have the cloud EC2 RDS Elastic Load Balancing EBS Redshift ElastiCache AWS Cloud
  • Customers have Data centers
  • Whiteboard Engineering EC2 RDS Elastic Load Balancing EBS Redshift ElastiCache AWS Cloud
  • EC2 as it was 10.44.12.5 10.44.12.4 10.44.92.17 10.44.12.27 10.108.6.4 Amazon EC2
  • Why that doesn’t work 192.168.0.0/16 10.44.0.0/16 10.44.12.5 10.44.12.4 10.44.92.17 Routing Table • 192.168.0.0/16: • 10.44.12.4/32: • 10.44.92.17/32: • 10.108.6.4/32: 10.44.12.27 stay here AWS AWS AWS 10.108.6.4 Amazon EC2
  • Requirements • Customer Selected IP Addresses • Route Aggregation for External Connectivity • Conformance with Existing Network Designs
  • Virtual Private Cloud 192.168.0.0/16 172.31.1.7 172.31.1.8 Routing Table • 192.168.0.0/16: • 172.31.0.0/18: stay here AWS 172.31.2.12 172.31.2.51 172.31.1.9 172.31.1.0/24 172.31.0.0/18 172.31.2.0/24
  • This is just virtual networking! • Subnet ~= VLAN • VPC ~= VRF (Virtual Routing and Forwarding) • But…
  • Scaling Challenges • VLAN ID space is constrained – 12 bits => 4096 total VLANs • VRF support is constrained – Large routers => 1-2 thousand VRFs • Fixed ratio of VLANs:VRFs
  • Router and capacity dimensions Big Router Big Router Control Plane Control Plane Data Plane Data Plane
  • An Example • • • • • • Average Router Configuration Line: Config per VPC: Subnets per VPC: Config per Subnet: Total VPCs: Config size: 50 chars 10 lines 4 5 lines 2,000 3MB
  • Silos of Capacity 2 3 1 0 /4 4 2 3 1 0 /4 A A C C B B D D D C C A A E D D D D F G G G G G F F F F F F 15 10/40 9 7 3 0 A F F B B B B B B B B B B B B B B B B B B B B B B 40 18 /40 9 2 0
  • Implementation Requirements • Scale to millions of environments the size of Amazon.com • Any server, anywhere in a region can host an instance attached to any Subnet in any VPC
  • Concepts Mapping Service Server 192.168.1.3 Server 192.168.0.3 10.0.0.2 10.0.0.3 10.0.0.2 10.0.0.4 Server 192.168.1.4 Server 192.168.0.4 10.0.0.4 10.0.0.5 10.0.0.3 … … Mapping VPC VPC:ID: Instance: Server: Service: Distributed lookup Identifier host VPC Amazon Virtual Physical EC2 in an for a service. Maps such as owned Private Cloud by instance datacentera Amazon vpc- VPC + Instance 1a2b3c4d owned by customer aIP to server customer
  • L2 - Ethernet Ethernet Switch 10.0.0.2 10.0.0.3 The switch floods the L2 Src: MAC(10.0.0.2) MAC(10.0.0.3) snoops the ARP request out all L2 Dst: MAC(10.0.0.2) response and MAC(10.0.0.3) ff:ff:ff:ff:ff:ff L3 Src: learns ports the port for 10.0.0.2 L3 Dst: MAC(10.0.0.3). ARP Who hasis at 10.0.0.310.0.0.3 MAC(10.0.0.3) 10.0.0.3? ICMP/TCP/UDP/…
  • L2 - VPC Mapping Service Server 192.168.1.3 Server 192.168.0.3 10.0.0.2 10.0.0.3 10.0.0.2 10.0.0.4 Server 192.168.1.4 Server 192.168.0.4 10.0.0.3 10.0.0.4 Src: 192.168.0.3 L2 Src: MAC(10.0.0.2) Mapping Service MAC(10.0.0.3) Dst: Mapping Service L2 Dst: MAC(10.0.0.2) 192.168.0.3 ff:ff:ff:ff:ff:ff 10.0.0.5 … Reply: Query: ARP Who hasis at 10.0.0.3 MAC(10.0.0.3) 10.0.0.3?10.0.0.3 Host: 192.168.1.4 Orange MAC: MAC(10.0.0.3)
  • L2 - VPC Mapping Service Server 192.168.1.3 Server 192.168.0.3 10.0.0.3 10.0.0.2 … Src: 192.168.0.3 Dst: 192.168.1.4 10.0.0.2 Server 192.168.0.4 VPC: Orange 10.0.0.4 Server 192.168.1.4 10.0.0.3 10.0.0.4 10.0.0.5 … Src: 192.168.1.4 Mapping Service L2 Src: MAC(10.0.0.2) Dst: Mapping Service 192.168.1.4 L2 Dst: MAC(10.0.0.3) L3 Src: 10.0.0.2 Mapping Validate: valid: L3 Dst: 10.0.0.3 Orange 10.0.0.2 is at 192.168.0.3 ICMP/TCP/UDP/…
  • VPC Isolation Mapping Service Server 192.168.1.3 Server 192.168.0.3 10.0.0.2 10.0.0.3 10.0.0.2 10.0.0.4 Server 192.168.1.4 Server 192.168.0.4 10.0.0.3 10.0.0.4 L2 Src: MAC(10.0.0.4) Src: 192.168.0.4 L2 Dst: ff:ff:ff:ff:ff:ff Dst: Mapping Service 10.0.0.5 … ARP Who has Query: 10.0.0.3? Grey 10.0.0.3
  • VPC Isolation Mapping Service Server 192.168.1.3 Server 192.168.0.3 10.0.0.2 10.0.0.3 10.0.0.2 10.0.0.4 Server 192.168.1.4 Server 192.168.0.4 10.0.0.3 10.0.0.4 10.0.0.5 … 192.168.0.4 is not L2 Src: MAC(10.0.0.4) Src: 192.168.0.4 hosting any instances L2 Dst: ff:ff:ff:ff:ff:ff Dst: Mapping Service in VPC Orange. ARP Who has Query: Mapping 10.0.0.3 10.0.0.3?Denied Orange Alarm Raised
  • VPC Isolation Mapping Service Server 192.168.1.3 Server 192.168.0.3 10.0.0.3 10.0.0.2 … Src: 192.168.0.4 Dst: 192.168.1.4 10.0.0.2 Server 192.168.0.4 VPC: Orange 10.0.0.4 Server 192.168.1.4 10.0.0.3 10.0.0.4 10.0.0.5 … 192.168.1.4 does Src: 192.168.1.4 not Mapping Service L2 Src: MAC(10.0.0.4) deliver the packet to Dst: Mapping Service 192.168.1.4 L2 Dst: MAC(10.0.0.3) theSrc: 10.0.0.4 L3 instance. Mapping Validate: invalid! L3 Dst: 10.0.0.3 Alarm Raised. Orange 10.0.0.4 is at 192.168.0.4 ICMP/TCP/UDP/…
  • L3 – IP Routing Ethernet Switch Router 10.0.0.2 Ethernet Switch 10.0.1.3 L2 Src: MAC(10.0.0.2) MAC(10.0.0.1) L2 Dst: MAC(10.0.0.2) MAC(10.0.0.1) ff:ff:ff:ff:ff:ff L3 Src: 10.0.0.2 L3 Dst: 10.0.1.3 ARP Who hasis at 10.0.0.1 MAC(10.0.0.1) 10.0.0.1? ICMP/TCP/UDP/… L2 Src: MAC(10.0.1.1) L2 Dst: MAC(10.0.1.3) L3 Src: 10.0.0.2 L3 Dst: 10.0.1.3 ICMP/TCP/UDP/…
  • L3 - VPC Mapping Service Server 192.168.1.3 Server 192.168.0.3 10.0.0.2 10.0.0.3 10.0.0.2 10.0.0.4 Server 192.168.1.4 Server 192.168.0.4 10.0.1.3 10.0.0.4 Src: 192.168.0.3 L2 Src: MAC(10.0.0.2) Mapping Service MAC(10.0.0.1) Dst: Mapping Service L2 Dst: MAC(10.0.0.2) 192.168.0.3 ff:ff:ff:ff:ff:ff 10.0.0.5 … Reply: Query: ARP Who hasis at 10.0.0.1 MAC(10.0.0.1) 10.0.0.1?10.0.0.1 Host: Gateway Orange MAC: MAC(10.0.0.1)
  • L3 - VPC Mapping Service Server 192.168.1.3 Server 192.168.0.3 10.0.0.3 10.0.0.2 10.0.0.2 Src: 192.168.0.3 Dst: 192.168.1.4 Server 192.168.0.4 10.0.0.4 10.0.0.5 10.0.0.4 Server 192.168.1.4 VPC: Orange … L2 Src: MAC(10.0.0.2) Src: 192.168.0.3 192.168.1.4 Mapping Service MAC(10.0.1.1) L2 Dst: MAC(10.0.0.1) Dst: Mapping Service 192.168.1.4 192.168.0.3 MAC(10.0.1.3) L3 Src: 10.0.0.2 Mapping Validate: L3 Dst: Reply: valid: Query: 10.0.1.3 Host: 192.168.1.4 Orange 10.0.1.3 is at 10.0.0.2 ICMP/TCP/UDP/… 192.168.0.3 MAC: MAC(10.0.1.3) 10.0.1.3
  • Caching Mapping Service Server 192.168.1.3 Server 192.168.0.3 10.0.0.2 10.0.0.3 10.0.0.2 10.0.0.4 Server 192.168.1.4 Server 192.168.0.4 10.0.0.3 10.0.0.4 10.0.0.5 ICMP/TCP/UDP/… … … L2 Src: MAC(10.0.1.1) L2 Dst: MAC(10.0.1.3) L3 Src: 10.0.0.2 L3 Dst: 10.0.1.3
  • VPC Pricing Cost per VPC: Cost per Subnet: Upcharge per Instance: $0.00 $0.00 $0.00
  • Nov 10, 2010
  • VPC as a Platform 172.31.1.7 172.31.2.12 172.31.1.8 172.31.2.51 172.31.1.0/24 172.31.2.0/24 172.31.0.0/18
  • VPC as a Platform • • • • • • VPN and Direct Connect Security Group Egress Filtering Network ACLs Routing Tables Elastic Network Interfaces (ENIs) Multiple IPs
  • EC2 VPC Simple Complex Limited Flexible
  • Default VPC 172.31.1.7 172.31.2.12 172.31.1.8 172.31.2.51 172.31.1.9 172.31.1.0/24 172.31.0.0/18 172.31.2.0/24
  • EC2 - VPC Simple Complex Limited Flexible
  • Other VPC Sessions ARC202: High Availability Application Architectures in Amazon VPC ARC401: From One to Many: Evolving VPC Design CPN208: Selecting the Best VPC Network Architecture (single VPC vs. multiple VPCs) CPN301: Amazon EC2 to Amazon VPC: A case study (this is the migration story)
  • Please give us your feedback on this presentation CPN401 As a thank you, we will select prize winners daily for completed surveys!