© 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or i...
Disclaimer:
Do Try This at Home!
All these designs are in use
by customers
Design…
then spend a lot of time building and deploying
Build and deploy virtual datacenters as fast as you
design them
ve...
Route Table Elastic Network
Interface
Amazon VPC Router
Internet
Gateway
Customer
Gateway
Virtual
Private
Gateway
VPN
Conn...
Availability Zone A Availability Zone B
Subnet
Availability Zone A
Subnet
Availability Zone B
VPC CIDR: 10.1.0.0 /16
Plan your VPC IP space before
creating it
• Consider future AWS region expansion
• Consider future connectivity to your in...
Public Subnet
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
VPC CIDR: 10.1.0.0 /16
Availability Zone A
Public Subnet
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
Instance A
10.1.1.11 /24
Instance C
10.1.3.3...
Public Subnet
Availability Zone A
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
Instance A
10.1.1.11 /24...
Public Subnet
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
Instance A
10.1.1.11 /24
Instance C
10.1.3.3...
Leave the Main Route Table Alone
Availability Zone B
Public Subnet
Availability Zone A
Private Subnet
Public Subnet
Private Subnet
Instance A
10.1.1.11 /24...
Network ACLs vs. Security Groups
NACLs
• Applied to subnets (1 per)
• Stateless
• Allow & deny (blacklist)
• Rules process...
VPC Network ACLs: What Are They Good For?
• Enforcing baseline security policy
– Example:
“No TFTP, NetBIOS or SMTP shall ...
VPC Network ACLs: Best Practices
• Use sparingly, keep it simple
• Avoid ephemeral port range allows
• Create rule #’s wit...
Create an IAM VPC Admin Group
Examples of “High Blast Radius” VPC API calls that should be restricted:
AttachInternetGatew...
Example IAM Policy for NACL Admin
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DeleteNe...
Public Subnet
Availability Zone A
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
Instance A
10.1.1.11 /24...
Public Subnet
Availability Zone A
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
Instance A
10.1.1.11 /24...
Public Subnet
Availability Zone A
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
Instance A
10.1.1.11 /24...
Ways to Assign Public IPs
Elastic IP address (EIP)
• Associated with AWS account and not a specific instance
• 1 public IP...
Ways to Assign Public IPs
Automatic dynamic public IP assignment
• Done on instance launch into VPC subnet
• Public IP is ...
Public Subnet
Availability Zone A
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
Instance A
Public: 54.20...
Examples of AWS outside the VPC
• AWS API endpoints
– Think about which APIs you might be calling from instances within th...
Public Subnet
Availability Zone A
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
Instance A
Public: 54.20...
Public Subnet
Availability Zone A
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
NAT A
Public: 54.200.129...
What makes up the
Amazon Linux NAT AMI?
$echo 1 > /proc/sys/net/ipv4/ip_forward
$echo 0 > /proc/sys/net/ipv4/conf/eth0/sen...
Public Subnet
Availability Zone A
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
NAT A
Public: 54.200.129...
Public Subnet
Availability Zone A
Private Subnet
Public Subnet
Availability Zone B
Private Subnet
NAT A
Public: 54.200.129...
Scalable and Available NAT
Do bandwidth-intensive processes need to be
behind a NAT?
• Separate out application components with bandwidth needs
• Run...
Availability Zone A Availability Zone B
Private Subnet
Internet
Amazon S3 Amazon Dynamo DB
AWS
region
Public Subnet Public...
Auto Scaling Support for
Automatic Public IP Assignment
$aws autoscaling create-launch-configuration --launch-configuratio...
Availability Zone A
Private Subnet
Availability Zone B
Private Subnet
Internet
Amazon S3
AWS
region
Public Subnet Public S...
Auto Scaling for Availability
$aws autoscaling create-auto-scaling-group --auto-scaling-group-name ha-
nat-asg --launch-co...
HA NAT User Data sample:
PRIVATE_SUBNETS="`aws ec2 describe-subnets --query 'Subnets[*].SubnetId’ --filters Name=availabil...
Tag Early, Tag Often!
• Tagging strategy should be part of early design
• Project code, cost center, environment, version,...
IAM EC2 Role for HA NAT Instance
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"ec2:DescribeI...
Automating HA NAT with EC2 User Data
Latest version of the script:
https://github.com/ralex-aws/vpc
If Design Requirements Keep High
Bandwidth Streams Behind NAT:
• Use the 1 HA NAT per Availability Zone design
• Verticall...
Take Advantage of Enhanced
Networking
• Only available in VPC
• Higher PPS, Lower Latency, Lower Jitter
• Supported by C3,...
One VPC, Two VPC
AWS
region
Considering Multiple VPCs
Public-facing
web app
Internal
company
app
What’s next?
VPN
connection
Customer
data ...
Common Customer Use Cases:
• Application isolation
• Scope of audit containment
• Risk level separation
• Separate product...
Controlling the Border
AWS
region
Internal Application to VPC
Public-facing
web app
Internal
company
app
VPN
connection
Customer
data center
Availability Zone A
Private Subnet Private Subnet
AWS
region
Virtual
Private
Gateway
VPN
connection
Customer
data center
I...
But… the app will leverage this for storing data
Amazon S3
Availability Zone A
Private Subnet Private Subnet
AWS
region
Virtual
Private
Gateway
VPN
connection
Customer
data center
I...
Control IGW Access through a Proxy Layer
• Deploy a proxy control layer between application and IGW
• Restrict all outboun...
Availability Zone A
Private Subnet Private Subnet
AWS region
VPN
connection
Customer
data center
Intranet
App
Intranet
App...
Put Elastic Load Balancers in Their
Own Subnets
• Elastic Load Balancing is Amazon EC2 in your subnets
• Elastic Load Bala...
Availability Zone A
Private Subnet(s) Private Subnet(s)
AWS region
VPN
connection
Customer
data center
Intranet
App
Intran...
Squid.conf Sample Config:
# CIDR AND Destination Domain based Allow
# CIDR Subnet blocks for Internal ELBs
acl int_elb_cid...
Using Squid Proxy Instances for Web Service Access
in Amazon VPC:
http://aws.amazon.com/articles/5995712515781075
AWS region
Public-facing
web app
Internal
company
app
What’s next?
VPN
connection
Customer data center
AWS region
Public-facing
web app
Internal
company
app #1
HA pair VPN
endpoints
Internal
company
app #2
Internal
company
ap...
Public-facing
web app
Internal
company
app #2
HA pair VPN
endpointsCustomer data center
Internal
company
app #3
Internal
c...
AWS
region
Public-facing
web app
Internal
company
app #1
HA pair VPN
endpoints
Customer data center
VPN Hub and Spoke an o...
VPC Peering
10.1.0.0/16
10.0.0.0/16
• VPCs within same Region
Peer
Request
Peer
Accept
• Same or Different Accounts
• IP Space Cannot ...
10.1.0.0/16
10.0.0.0/16 10.0.0.0/16
✔
10.1.0.0/16
10.0.0.0/16
Route Table
Destination Target
10.1.0.0/16 local
10.0.0.0/16 PCX-1
Route Table
Destination Target
...
10.0.0.0/16 10.0.0.0/16
PCX-1 PCX-2
Subnet 1
10.1.1.0/24
Subnet 2
10.1.2.0/24
10.1.0.0/16
Route Table Subnet 1
Destination...
10.0.0.0/16 10.0.0.0/16
PCX-1 PCX-2
Subnet 1
10.1.1.0/24
Subnet 2
10.1.2.0/24
10.1.0.0/16
Route Table Subnet 1
Destination...
10.1.0.0/16
10.0.0.0/16 10.0.0.0/16
10.3.0.0/16
172.16.0.0/16
192.168.0.0/16
10.2.0.0/16
172.17.0.0/16
CA
10.1.0.0/16
10.0.0.0/16 10.0.0.0/16
10.3.0.0/16
172.16.0.0/16
192.168.0.0/16
10.2.0.0/16
172.17.0.0/16
company data center...
10.1.0.0/16
10.0.0.0/16 10.0.0.0/16
10.3.0.0/16
172.16.0.0/16
192.168.0.0/16
10.2.0.0/16
172.17.0.0/16
company data center...
the number of unique
connections in a
network of a number
of nodes (n) can be
expressed
mathematically as the
triangular n...
10.0.0.0/16 10.0.0.0/16
172.16.0.0/16
192.168.0.0/16
172.17.0.0/16
10.1.0.0/16 10.2.0.0/1610.3.0.0/16
10.0.0.0/16
10.0.0.0/16
10.3.0.0/16
172.16.0.0/16
192.168.1.0/24
10.2.0.0/16
172.17.0.0/16
AWS
region
Public-facing
web app
Internal
company
app #1
HA pair VPN
endpoints
company data center
Peer Review
Internal
co...
Use IAM to Define & Enforce a
VPC’s Operational State
Use EC2 Run Resource Permissions to control:
• What AMI can be launc...
AWS
region
Public-facing
web app
HA pair VPN
endpoints
Customer data center
AWS
region
Prod QA Dev
Bringing It All Back Home
Customer
data center
AWS Direct Connect
location
AWS Direct Connect Private Virtual
Interface (PVI) connects to VGW on
VPC...
A few bits on AWS Direct Connect…
• Dedicated, private pipes into AWS
• Create private (VPC) or public interfaces to AWS
•...
VPC 1
Private Virtual Interface 1
VLAN Tag 101
BGP ASN 7224
BGP Announce 10.1.0.0/16
Interface IP 169.254.251.5/30 10.1.0....
AWS Direct Connect in the United States
AWS Direct Connect
Equinix, San Jose
us-west-1
us-west-2
us-east-1
AWS Private Net...
Customer internal
network
VPC 1
Public Virtual Interface 1
VLAN Tag 501
BGP ASN 7224
BGP Announce AWS Regional
Public CIDR...
See What Your VGW Sees
Before: Enable:
After:
Customer routers
Customer internal
network
AWS DX routers
AWS
region
AWS Direct Connect
location
Multiple physical connect...
Customer
routers
Customer global
MPLS backbone
network
US-East-1
AWS
region
AWS Direct Connect
location:
Virginia or NYC
G...
With AWS regions just another spoke on your global network,
it’s easy to bring the cloud down to you as you expand around ...
Evolving VPC Design: Recap
• Elements of VPC Design
• Scalable and Available NAT
• One VPC, Two VPC
• Controlling the Bord...
Upcoming SlideShare
Loading in...5
×

From One to Many: Evolving VPC Design

1,765

Published on

In this advanced technical session, learn how you can use AWS to build and deploy virtual data centers as fast as you design them. This session follows the evolution of a single regional Amazon Virtual Private Cloud (VPC) into a multi-VPC, multi-region design with diverse connectivity into on-premises systems and infrastructure. Along the way, we investigate best practice designs in use by AWS customers to simplify and optimize as they grow. Topics covered include: automating virtual data centers with CloudFormation, scaling and securing outbound VPC traffic, Peering VPCs within a region, and running global hybrid networks with VPC and Direct Connect.

Published in: Technology, Business

From One to Many: Evolving VPC Design

  1. 1. © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified, or distributed in whole or in part without the express consent of Amazon.com, Inc. From One to Many: Evolving VPC Design Yinal Ozkan, AWS Principal Solutions Architect
  2. 2. Disclaimer: Do Try This at Home! All these designs are in use by customers
  3. 3. Design… then spend a lot of time building and deploying Build and deploy virtual datacenters as fast as you design them version
  4. 4. Route Table Elastic Network Interface Amazon VPC Router Internet Gateway Customer Gateway Virtual Private Gateway VPN ConnectionSubnet Elements of VPC Design
  5. 5. Availability Zone A Availability Zone B
  6. 6. Subnet Availability Zone A Subnet Availability Zone B VPC CIDR: 10.1.0.0 /16
  7. 7. Plan your VPC IP space before creating it • Consider future AWS region expansion • Consider future connectivity to your internal networks • Consider subnet design • VPC can be /16 down to /28 • CIDR cannot be modified after creation • Overlapping IP spaces = future headache
  8. 8. Public Subnet Private Subnet Public Subnet Availability Zone B Private Subnet VPC CIDR: 10.1.0.0 /16 Availability Zone A
  9. 9. Public Subnet Private Subnet Public Subnet Availability Zone B Private Subnet Instance A 10.1.1.11 /24 Instance C 10.1.3.33 /24 Instance B 10.1.2.22 /24 Instance D 10.1.4.44 /24 VPC CIDR: 10.1.0.0 /16 Availability Zone A
  10. 10. Public Subnet Availability Zone A Private Subnet Public Subnet Availability Zone B Private Subnet Instance A 10.1.1.11 /24 Instance C 10.1.3.33 /24 Instance B 10.1.2.22 /24 Instance D 10.1.4.44 /24 VPC CIDR: 10.1.0.0 /16 .1 .1 .1 .1
  11. 11. Public Subnet Private Subnet Public Subnet Availability Zone B Private Subnet Instance A 10.1.1.11 /24 Instance C 10.1.3.33 /24 Instance B 10.1.2.22 /24 Instance D 10.1.4.44 /24 VPC CIDR: 10.1.0.0 /16 Route Table Destination Target 10.1.0.0/16 local Availability Zone A
  12. 12. Leave the Main Route Table Alone
  13. 13. Availability Zone B Public Subnet Availability Zone A Private Subnet Public Subnet Private Subnet Instance A 10.1.1.11 /24 Instance C 10.1.3.33 /24 Instance B 10.1.2.22 /24 Instance D 10.1.4.44 /24 VPC CIDR: 10.1.0.0 /16 Route Table Destination Target 10.1.0.0/16 local 10.1.1.0/24 Instance B
  14. 14. Network ACLs vs. Security Groups NACLs • Applied to subnets (1 per) • Stateless • Allow & deny (blacklist) • Rules processed in order Security groups • Applied to instance ENI (up to 5 per) • Stateful • Allow only (whitelist) • Rules evaluated as a whole • Can reference other security groups in the same VPC VPC Subnet Elastic network interface Security group Network ACL
  15. 15. VPC Network ACLs: What Are They Good For? • Enforcing baseline security policy – Example: “No TFTP, NetBIOS or SMTP shall egress this subnet” • Catchall for holes in instance security groups • Segregation of security between network ops and dev ops VPC Subnet Instance
  16. 16. VPC Network ACLs: Best Practices • Use sparingly, keep it simple • Avoid ephemeral port range allows • Create rule #’s with room to grow • Use IAM to control tightly who can alter or delete NACLs Pushing this will hurt! Default network ACL:
  17. 17. Create an IAM VPC Admin Group Examples of “High Blast Radius” VPC API calls that should be restricted: AttachInternetGateway AssociateRouteTable CreateRoute DeleteCustomerGateway DeleteInternetGateway DeleteNetworkAcl DeleteNetworkAclEntry DeleteRoute DeleteRouteTable DeleteDhcpOptions ReplaceNetworkAclAssociation DisassociateRouteTable {Support Resource Permissions
  18. 18. Example IAM Policy for NACL Admin { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ec2:DeleteNetworkAcl", "ec2:DeleteNetworkAclEntry" ], "Resource": "arn:aws:ec2:us-west-2:123456789012:network-acl/*", "Condition": { "StringEquals": { "ec2:ResourceTag/Environment": "prod" }, "Null": { "aws:MultiFactorAuthAge": "false" } } } ] }
  19. 19. Public Subnet Availability Zone A Private Subnet Public Subnet Availability Zone B Private Subnet Instance A 10.1.1.11 /24 Instance C 10.1.3.33 /24 Instance B 10.1.2.22 /24 Instance D 10.1.4.44 /24 VPC CIDR: 10.1.0.0 /16 Creating ways “out” of a VPC
  20. 20. Public Subnet Availability Zone A Private Subnet Public Subnet Availability Zone B Private Subnet Instance A 10.1.1.11 /24 Instance C 10.1.3.33 /24 Instance B 10.1.2.22 /24 Instance D 10.1.4.44 /24 VPC CIDR: 10.1.0.0 /16 Virtual Private Gateway Internet Gateway Only 1 IGW and 1 VGW per VPC VPN connection Customer data center Customer data center AWS Direct Connect Route Table Destination Target 10.1.0.0/16 local Internal CIDR VGW
  21. 21. Public Subnet Availability Zone A Private Subnet Public Subnet Availability Zone B Private Subnet Instance A 10.1.1.11 /24 Instance C 10.1.3.33 /24 Instance B 10.1.2.22 /24 Instance D 10.1.4.44 /24 VPC CIDR: 10.1.0.0 /16 Route Table Route Table Destination Target 10.1.0.0/16 local 0.0.0.0/0 IGW
  22. 22. Ways to Assign Public IPs Elastic IP address (EIP) • Associated with AWS account and not a specific instance • 1 public IP to 1 private IP static NAT mapping • Instance does not “see” an EIP associated to it • Persists independently of the instance • Can be assigned while instance is stopped or running • Can be moved, reassigned to other ENIs
  23. 23. Ways to Assign Public IPs Automatic dynamic public IP assignment • Done on instance launch into VPC subnet • Public IP is dynamic and could change if instance is stopped and restarted • Does not count against AWS account EIP limits • Works only on instances with a single ENI
  24. 24. Public Subnet Availability Zone A Private Subnet Public Subnet Availability Zone B Private Subnet Instance A Public: 54.200.129.18 Private: 10.1.1.11 /24 Instance C 10.1.3.33 /24 Instance B 10.1.2.22 /24 Instance D 10.1.4.44 /24 Route Table Internet Amazon S3 Amazon Dynamo DB AWS region AWS outside the VPC
  25. 25. Examples of AWS outside the VPC • AWS API endpoints – Think about which APIs you might be calling from instances within the VPC – Good examples: Amazon EC2, AWS CloudFormation, Auto Scaling, Amazon SWF, Amazon SQS, Amazon SNS • Regional services – Amazon S3 – Amazon Dynamo DB • Software and patch repositories – Amazon Linux repo allows access only from AWS public IP blocks
  26. 26. Public Subnet Availability Zone A Private Subnet Public Subnet Availability Zone B Private Subnet Instance A Public: 54.200.129.18 Private: 10.1.1.11 /24 Instance C 10.1.3.33 /24 Instance B 10.1.2.22 /24 Instance D 10.1.4.44 /24 Route Table Internet Amazon S3 AWS region And what if instance C in a private subnet needs to reach outside the VPC? It has no route to the IGW and no public IP. Amazon Dynamo DB
  27. 27. Public Subnet Availability Zone A Private Subnet Public Subnet Availability Zone B Private Subnet NAT A Public: 54.200.129.18 Private: 10.1.1.11 /24 Instance C 10.1.3.33 /24 Instance B 10.1.2.22 /24 Instance D 10.1.4.44 /24 Internet Amazon S3 AWS region Deploy an instance that functions as a N etwork A ddress T ranslat(or) Route Table Destination Target 10.1.0.0/16 local 0.0.0.0/0 NAT instanc e Amazon Dynamo DB
  28. 28. What makes up the Amazon Linux NAT AMI? $echo 1 > /proc/sys/net/ipv4/ip_forward $echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects $/sbin/iptables -t nat -A POSTROUTING -o eth0 –s 10.1.0.0/16 -j MASQUERADE $/sbin/iptables-save $aws ec2 modify-instance-attributes –instance-id i-xxxxxxxx –source-dest- check “{”Value”:false}” Not much to it: 1. IP forwarding enabled 2. IP NAT Masquerading enabled in iptables for VPC CIDR block 3. Source/destination check is turned off on primary interface
  29. 29. Public Subnet Availability Zone A Private Subnet Public Subnet Availability Zone B Private Subnet NAT A Public: 54.200.129.18 Private: 10.1.1.11 /24 Instance C 10.1.3.33 /24 Instance B 10.1.2.22 /24 Instance D 10.1.4.44 /24 Internet Amazon S3 AWS region Other private subnets can share the same routing table and use the NAT But… Amazon Dynamo DB
  30. 30. Public Subnet Availability Zone A Private Subnet Public Subnet Availability Zone B Private Subnet NAT A Public: 54.200.129.18 Private: 10.1.1.11 /24 Instance B 10.1.2.22 /24 Internet Amazon S3 AWS region … you could reach a bandwidth bottleneck if your private instances grow and their NAT- bound traffic grows with them. Amazon Dynamo DB
  31. 31. Scalable and Available NAT
  32. 32. Do bandwidth-intensive processes need to be behind a NAT? • Separate out application components with bandwidth needs • Run components from public subnet instances • Goal is full instance bandwidth out of VPC • Auto Scaling with Public IP makes this easy • NAT still in place for remaining private instances • Most common use case: Multi-Gbps streams to Amazon S3
  33. 33. Availability Zone A Availability Zone B Private Subnet Internet Amazon S3 Amazon Dynamo DB AWS region Public Subnet Public Subnet NAT Customers Public load balancer Web Servers • Image processing app with high outbound network to Amazon S3 Direct to Amazon S3 Public ELB Subnet Private Subnet Public ELB Subnet Multi-AZ Auto Scaling group Auto Scaling group • Public Elastic Load Balancer receives incoming customer HTTP/S requests • Auto Scaling assigns public IP to new web servers • With public IPs, web servers initiate outbound requests directly to Amazon S3 • NAT device still in place for private subnets
  34. 34. Auto Scaling Support for Automatic Public IP Assignment $aws autoscaling create-launch-configuration --launch-configuration-name hi-bandwidth- public --image-id ami-xxxxxxxx --instance-type m1.xlarge --associate-public-ip-address Sample launch configuration (named “hi-bandwidth-public”):
  35. 35. Availability Zone A Private Subnet Availability Zone B Private Subnet Internet Amazon S3 AWS region Public Subnet Public Subnet NAT • Use Auto Scaling for NAT availability • Create 1 NAT per Availability Zone • All private subnet route tables to point to same zone NAT • 1 Auto Scaling group per NAT with min and max size set to 1 • Let Auto Scaling monitor the health and availability of your NATs • NAT bootstrap script updates route tables programmatically Auto scale HA NAT NAT Amazon DynamoDB
  36. 36. Auto Scaling for Availability $aws autoscaling create-auto-scaling-group --auto-scaling-group-name ha- nat-asg --launch-configuration-name ha-nat-launch --min-size 1 --max-size 1 --vpc-zone-identifier subnet-xxxxxxxx Sample HA NAT Auto Scaling group (named “ha-nat-asg”):
  37. 37. HA NAT User Data sample: PRIVATE_SUBNETS="`aws ec2 describe-subnets --query 'Subnets[*].SubnetId’ --filters Name=availability- zone,Values=$AVAILABILITY_ZONE Name=vpc-id,Values=$VPC_ID Name=state,Values=available Name=tag:network,Values=private`” if [ -z "$PRIVATE_SUBNETS" ]; then die "No private subnets found to modify for HA NAT." else log "Modifying Route Tables for following private subnets: $PRIVATE_SUBNETS" fi for subnet in $PRIVATE_SUBNETS; do ROUTE_TABLE_ID=`aws ec2 describe-route-tables --query 'RouteTables[*].RouteTableId’ --filters Name=association.subnet-id,Values=$subnet`; if [ "$ROUTE_TABLE_ID" = "$MAIN_RT" ]; then log "$subnet is associated with the VPC Main Route Table. HA NAT script will NOT edit Main Route Table.” elif [ -z "$ROUTE_TABLE_ID" ]; then log "$subnet is not associated with a Route Table. Skipping this subnet." else aws ec2 create-route --route-table-id $ROUTE_TABLE_ID --destination-cidr-block 0.0.0.0/0 --instance-id $INSTANCE_ID && log "$ROUTE_TABLE_ID associated with $subnet modified to point default route to $INSTANCE_ID." if [ $? -ne 0 ] ; then aws ec2 replace-route --route-table-id $ROUTE_TABLE_ID --destination-cidr-block 0.0.0.0/0 --instance-id $INSTANCE_ID fi fi done
  38. 38. Tag Early, Tag Often! • Tagging strategy should be part of early design • Project code, cost center, environment, version, team, business unit • Tag resources right after creation • Tags supported for resource permissions • AWS Billing also supports tags • Tight IAM controls on the creation and editing of tags
  39. 39. IAM EC2 Role for HA NAT Instance { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ec2:DescribeInstances", "ec2:ModifyInstanceAttribute", "ec2:DescribeSubnets", "ec2:DescribeRouteTables", "ec2:CreateRoute", "ec2:ReplaceRoute" ], "Resource": "*" } ] }
  40. 40. Automating HA NAT with EC2 User Data Latest version of the script: https://github.com/ralex-aws/vpc
  41. 41. If Design Requirements Keep High Bandwidth Streams Behind NAT: • Use the 1 HA NAT per Availability Zone design • Vertically scale your NAT instance type to one with a High Network Performance rating • Keep a close watch on your network metrics m1.small Low m1.large Moderate m1.xlarge, c3.2xlarge High t1.micro Very Low
  42. 42. Take Advantage of Enhanced Networking • Only available in VPC • Higher PPS, Lower Latency, Lower Jitter • Supported by C3, I2, R3 instance types • Built into Amazon Linux, but supported in many flavors (including Windows) http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking.html
  43. 43. One VPC, Two VPC
  44. 44. AWS region Considering Multiple VPCs Public-facing web app Internal company app What’s next? VPN connection Customer data center
  45. 45. Common Customer Use Cases: • Application isolation • Scope of audit containment • Risk level separation • Separate production from non-production • Multi-tenant isolation • Business unit alignment
  46. 46. Controlling the Border
  47. 47. AWS region Internal Application to VPC Public-facing web app Internal company app VPN connection Customer data center
  48. 48. Availability Zone A Private Subnet Private Subnet AWS region Virtual Private Gateway VPN connection Customer data center Intranet App Intranet App Availability Zone B Internal customers Internal Application to VPC Route Table Destination Target 10.1.0.0/16 local Corp CIDR VGW
  49. 49. But… the app will leverage this for storing data Amazon S3
  50. 50. Availability Zone A Private Subnet Private Subnet AWS region Virtual Private Gateway VPN connection Customer data center Intranet App Intranet App Availability Zone B And you don’t really want to do this: Amazon S3 Internet Customer border router Customer VPN Internet
  51. 51. Control IGW Access through a Proxy Layer • Deploy a proxy control layer between application and IGW • Restrict all outbound HTTP/S access to only approved URL destinations like Amazon S3 • No route to IGW for private subnets • Control access to proxy through security groups • Must configure proxy setting in OS of instances
  52. 52. Availability Zone A Private Subnet Private Subnet AWS region VPN connection Customer data center Intranet App Intranet App Availability Zone B Internal customers Controlling the Border Internal Load balancer Elastic Load Balancing Private Subnet Elastic Load Balancing Private Subnet ELB Multi AZ Auto Scaling group • Deploy internal Elastic Load Balancing layer across Availability Zones • Add all instances allowed outside access to a security group • Use this security group as the only source allowed access to the proxy port in the load balancer’s security group
  53. 53. Put Elastic Load Balancers in Their Own Subnets • Elastic Load Balancing is Amazon EC2 in your subnets • Elastic Load Balancing is using your private addresses • Separate subnets = separate control • Distinguish load balancing layer from app layers
  54. 54. Availability Zone A Private Subnet(s) Private Subnet(s) AWS region VPN connection Customer data center Intranet App Intranet App Availability Zone B Internal customers Controlling the Border Internal Load balancer Elastic Load Balancing Private Subnet Elastic Load Balancing Private Subnet • Squid Proxy layer deployed between internal load balancer and the IGW border. Proxy Public Subnet Proxy Public Subnet Amazon S3 HTTP/S Multi AZ Auto Scaling group • Only proxy subnets have route to IGW. • Proxy security group allows inbound only from Elastic Load Balancing security group. • Proxy restricts which URLs may pass. In this example, s3.amazonaws.com is allowed. • Egress NACLs on proxy subnets enforce HTTP/S only.
  55. 55. Squid.conf Sample Config: # CIDR AND Destination Domain based Allow # CIDR Subnet blocks for Internal ELBs acl int_elb_cidrs src 10.1.3.0/24 10.1.4.0/24 # Destination domain for target S3 bucket acl s3_v2_endpoints dstdomain $bucket_name.s3.amazonaws.com # Squid does AND on both ACLs for allow match http_access allow int_elb_cidrs s3_v2_endpoints # Deny everything else http_access deny all
  56. 56. Using Squid Proxy Instances for Web Service Access in Amazon VPC: http://aws.amazon.com/articles/5995712515781075
  57. 57. AWS region Public-facing web app Internal company app What’s next? VPN connection Customer data center
  58. 58. AWS region Public-facing web app Internal company app #1 HA pair VPN endpoints Internal company app #2 Internal company app #3 Internal company app #4 Customer data center Customer gateways (CGW): • 1 per VPN tunnel • 1 public IP per CGW • AWS provides 2 tunnel destinations per region
  59. 59. Public-facing web app Internal company app #2 HA pair VPN endpointsCustomer data center Internal company app #3 Internal company app #4 Internal company app #1 Internal company Dev Internal company QA AWS region BackupAD, DNS Monitoring Logging
  60. 60. AWS region Public-facing web app Internal company app #1 HA pair VPN endpoints Customer data center VPN Hub and Spoke an option… Internal company app #2 Internal company app #3 Internal company app #4 Services VPC • Amazon EC2 VPN instances to central virtual private gateway • For HA, two Amazon EC2- based VPN endpoints in each spoke • Control VPC contains common services for all app VPCs • Dynamic routing protocol (BGP, OSPF) between spokes and hub
  61. 61. VPC Peering
  62. 62. 10.1.0.0/16 10.0.0.0/16 • VPCs within same Region Peer Request Peer Accept • Same or Different Accounts • IP Space Cannot Overlap • Only 1 between any 2 VPCs
  63. 63. 10.1.0.0/16 10.0.0.0/16 10.0.0.0/16 ✔
  64. 64. 10.1.0.0/16 10.0.0.0/16 Route Table Destination Target 10.1.0.0/16 local 10.0.0.0/16 PCX-1 Route Table Destination Target 10.0.0.0/16 local 10.1.0.0/16 PCX-1 PCX-1 • No IGW or VGW Required A B • No SPoF • No Bandwidth Bottlenecks
  65. 65. 10.0.0.0/16 10.0.0.0/16 PCX-1 PCX-2 Subnet 1 10.1.1.0/24 Subnet 2 10.1.2.0/24 10.1.0.0/16 Route Table Subnet 1 Destination Target 10.1.0.0/16 local 10.0.0.0/16 PCX-1 Route Table Subnet 2 Destination Target 10.1.0.0/16 local 10.0.0.0/16 PCX-2 A B C
  66. 66. 10.0.0.0/16 10.0.0.0/16 PCX-1 PCX-2 Subnet 1 10.1.1.0/24 Subnet 2 10.1.2.0/24 10.1.0.0/16 Route Table Subnet 1 Destination Target 10.1.0.0/16 local 10.0.1.11/32 PCX-1 Route Table Subnet 2 Destination Target 10.1.0.0/16 local 10.0.0.0/16 PCX-2 A B CSubnet 3 Route Table Subnet 3 Destination Target 10.0.0.0/16 local 10.1.1.0/24 PCX-1 10.0.1.11 Route Table Subnet 1 Destination Target 10.1.0.0/16 local 10.0.0.0/16 PCX-1
  67. 67. 10.1.0.0/16 10.0.0.0/16 10.0.0.0/16 10.3.0.0/16 172.16.0.0/16 192.168.0.0/16 10.2.0.0/16 172.17.0.0/16 CA
  68. 68. 10.1.0.0/16 10.0.0.0/16 10.0.0.0/16 10.3.0.0/16 172.16.0.0/16 192.168.0.0/16 10.2.0.0/16 172.17.0.0/16 company data center 10.10.0.0/16
  69. 69. 10.1.0.0/16 10.0.0.0/16 10.0.0.0/16 10.3.0.0/16 172.16.0.0/16 192.168.0.0/16 10.2.0.0/16 172.17.0.0/16 company data center 10.10.0.0/16
  70. 70. the number of unique connections in a network of a number of nodes (n) can be expressed mathematically as the triangular number n(n − 1)/2 How many connections ?
  71. 71. 10.0.0.0/16 10.0.0.0/16 172.16.0.0/16 192.168.0.0/16 172.17.0.0/16 10.1.0.0/16 10.2.0.0/1610.3.0.0/16
  72. 72. 10.0.0.0/16 10.0.0.0/16 10.3.0.0/16 172.16.0.0/16 192.168.1.0/24 10.2.0.0/16 172.17.0.0/16
  73. 73. AWS region Public-facing web app Internal company app #1 HA pair VPN endpoints company data center Peer Review Internal company app #2 Internal company app #3 Internal company app #4 Services VPC • Shared Infrastructure Services moved to VPC Internal company Dev Internal company QA AD, DNS Monitoring Logging • 1 to 1 Peering = App Isolation • Security Groups and NACLs still apply • Security Groups still bound to single VPC
  74. 74. Use IAM to Define & Enforce a VPC’s Operational State Use EC2 Run Resource Permissions to control: • What AMI can be launched • What VPC or subnet can be targeted • What Security Groups must be in place • Which VPCs allow Peering http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_IAM.html For more policy examples:
  75. 75. AWS region Public-facing web app HA pair VPN endpoints Customer data center AWS region Prod QA Dev
  76. 76. Bringing It All Back Home
  77. 77. Customer data center AWS Direct Connect location AWS Direct Connect Private Virtual Interface (PVI) connects to VGW on VPC • 1 PVI per VPC • 802.1Q VLAN Tags isolate traffic across AWS Direct Connect Private fiber connection One or multiple 50 – 500 Mbps, 1 Gbps or 10 Gbps pipes Simplify with AWS Direct Connect Public-facing web app AWS region Prod QA Dev
  78. 78. A few bits on AWS Direct Connect… • Dedicated, private pipes into AWS • Create private (VPC) or public interfaces to AWS • Cheaper data-out rates than Internet (data-in still free) • Consistent network performance compared to Internet • At least 1 location to each AWS region (even GovCloud!) • Recommend redundant connections • Multiple AWS accounts can share a connection
  79. 79. VPC 1 Private Virtual Interface 1 VLAN Tag 101 BGP ASN 7224 BGP Announce 10.1.0.0/16 Interface IP 169.254.251.5/30 10.1.0.0/16 VGW 1 Multiple VPCs Over AWS Direct Connect Customer Switch + Router Customer Interface 0/1.101 VLAN Tag 101 BGP ASN 65001 BGP Announce Customer Internal Interface IP 169.254.251.6/30 VLAN 101 VLAN 102 VLAN 103 VPC 2 10.2.0.0/16 VGW 2 VPC 3 10.3.0.0/16 VGW 3 Private Virtual Interface 2 VLAN Tag 102 BGP ASN 7224 BGP Announce 10.2.0.0/16 Interface IP 169.254.251.9/30 Customer Interface 0/1.102 VLAN Tag 102 BGP ASN 65002 BGP Announce Customer Internal Interface IP 169.254.251.10/30 Customer Interface 0/1.103 VLAN Tag 103 BGP ASN 65003 BGP Announce Customer Internal Interface IP 169.254.251.14/30 Private Virtual Interface 3 VLAN Tag 103 BGP ASN 7224 BGP Announce 10.3.0.0/16 Interface IP 169.254.251.13/30 Route Table Destination Target 10.1.0.0/16 PVI 1 10.2.0.0/16 PVI 2 10.3.0.0/16 PVI 3 Customer Internal Network
  80. 80. AWS Direct Connect in the United States AWS Direct Connect Equinix, San Jose us-west-1 us-west-2 us-east-1 AWS Private Network Disaster Recovery VPN to VGW
  81. 81. Customer internal network VPC 1 Public Virtual Interface 1 VLAN Tag 501 BGP ASN 7224 BGP Announce AWS Regional Public CIDRs Interface IP Public /30 Provided 10.1.0.0/16 VGW 1 Public AWS + VPCs Over AWS Direct Connect Customer Switch + Router Customer Interface 0/1.501 VLAN Tag 501 BGP ASN 65501 (or Public) BGP Announce Customer Public Interface IP Public /30 Provided VLAN 101 VLAN 102 VLAN 103 VLAN 501 VPC 2 10.2.0.0/16 VGW 2 VPC 3 10.3.0.0/16 VGW 3 Public US AWS Regions Route Table Destination Target 10.1.0.0/16 PVI 1 10.2.0.0/16 PVI 2 10.3.0.0/16 PVI 3 Public AWS PVI 5 NAT / PAT security layer
  82. 82. See What Your VGW Sees Before: Enable: After:
  83. 83. Customer routers Customer internal network AWS DX routers AWS region AWS Direct Connect location Multiple physical connections: • Active / Active links via BGP multi-pathing • Active / Passive also an option • BGP MEDs or local preference can influence route • Bidirectional Forwarding Detection (BFD) protocol supported
  84. 84. Customer routers Customer global MPLS backbone network US-East-1 AWS region AWS Direct Connect location: Virginia or NYC Going Global Customer routers AWS DX routers AWS Direct Connect location: Ireland or London EU-West-1 AWS region AWS DX routers
  85. 85. With AWS regions just another spoke on your global network, it’s easy to bring the cloud down to you as you expand around the world. US customer data center EU-West-1 region EU customer data center Customer MPLS backbone AWS Direct Connect PoP Ireland or London US-West-1 region AWS Direct Connect PoP Virginia or NYC AP-Southeast-1 region AWS Direct Connect PoP Singapore AP customer data center
  86. 86. Evolving VPC Design: Recap • Elements of VPC Design • Scalable and Available NAT • One VPC, Two VPC • Controlling the Border • Directory and Name Services in the VPC • VPC Peering • Bringing It All Back Home
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×