Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

(BAC404) Deploying High Availability and Disaster Recovery Architectures with AWS | AWS re:Invent 2014

2,625 views

Published on

In this session, we show how to architect, deploy, and scale an application for high availability within a region along with failing over to another AWS region in the event of a disaster at your primary region. During the session, we use real-time live demos and code examples for high availability and disaster recovery scenarios.

Published in: Technology

(BAC404) Deploying High Availability and Disaster Recovery Architectures with AWS | AWS re:Invent 2014

  1. 1. November 12, 2014 | Las Vegas, NV Vikram Garlapati
  2. 2. •Higher costs for disaster recovery sites or data centers •Costs for storage, backup, archival and retrieval tools, and processes •Capacity planning, procuring, and deploying can be challenging •Very low total cost of ownership •Scalable storage services and consistent tools and processes across various AWS regions and Availability Zones.
  3. 3. Backup and restore Pilot light Warm standby Multi- site
  4. 4. On-site infrastructure S3 Bucket Over the Internet AWS Import/Export or AWS Storage Gateway AWS region Corporate data center Using AWS Direct Connect Availability Zone Amazon Elastic Compute Cloud (EC2) Amazon S3 bucket Backup and restore
  5. 5. Availability Zone AWS region AMI Instance Amazon S3 bucket Pre-bundled with OS and applications Instance quickly provisioned from AMI Data copied from objects in S3 Data Volume Amazon EC2 On-site infrastructure AWS Import/Export or AWS Storage Gateway Using AWS Direct Connect Backup and restore
  6. 6. Pilot light www.example.com Data Mirroring/ Replication Not Running Databa se Server Data Volume Corporate data center Web Application Server Master Database Server
  7. 7. Pilot light Not Running Database Server Data Volume Corporate data center Web Application Server Master Database Server www.example.com Application Server Web Server Server
  8. 8. Warm standby Data Mirroring/ Replication Application Data Source Cut Over Elastic Load Balancer Active Production Amazon Route 53 www.example.com Corporate data center Data Volume Application Server Slave Database Server Web AWS region Web Application Server Master Database Server Non Production Traffic
  9. 9. Warm standby Elastic Load Balancer Active Production Amazon Route 53 www.example.com Corporate data center Data Volume Application Server Slave Database Server Web AWS region Web Application Server Master Database Server
  10. 10. AWSdatabase licenseslow infrastructure budgetsvariable loads. 99.99% 0-2 minutes0-15 minutes Backup and restore Pilot light Warm standby Multi-site
  11. 11. awsdrdemo.com
  12. 12. Amazon Route 53 Elastic Load BalancingAuto ScalingRedundant nodes / clusteringRedundant nodesDeploy Multi-AZDeploy multi-region
  13. 13. Infrastructure Foundation Services Regions Availability Zones Storage (Object, Block and Archive) Networking Security & Access Control Platform Services Databases Relational NoSQL Caching Analytics Hadoop Real-time Data warehouse App Services Queuing Orchestration App streaming Transcoding Email Search Deployment & Management Containers Dev/ops Tools Resource Templates Mobile Services Identity Sync Mobile Analytics Notifications Enterprise Applications Virtual Desktops Data Workflows Usage Tracking Monitoring and Logs Compute (VMs, Auto-scaling and Load Balancing) CDN and Points of Presence Collaboration and Sharing
  14. 14. Oregon North Virginia
  15. 15. Oregon Availability Zone #1 Availability Zone #2 Public subnet Private subnet Public subnet Private subnet Northern Virginia Availability Zone #1 Availability Zone #2 Public subnet Private subnet Public subnet Private subnet 10.0.0.0/16 192.168.0.0/16
  16. 16. Oregon Availability Zone #1 Availability Zone #2 Public subnet Private subnet Public subnet Private subnet Northern Virginia Availability Zone #1 Availability Zone #2 Public subnet Private subnet Public subnet Private subnet VPN Instance A VPN Instance B Secondary IPsec Primary IPsec
  17. 17. Oregon Availability Zone #1 Availability Zone #2 Public subnet Private subnet Public subnet Private subnet Northern Virginia Availability Zone #1 Availability Zone #2 Public subnet Private subnet Public subnet Private subnet VPN Instance A VPN Instance B Secondary IPsec
  18. 18. G2 GPU enabled M3 General purpose Memory optimized R3 CR1 M2 Storage and I/O optimized C3 Compute optimized CC2 C1 I2 HI1 HS1 CG1 M1 © 2014 Amazon.com, Inc. and its affiliates. All rights reserved. May not be copied, modified or distributed in whole or in part without the express consent of Amazon.com, Inc. Web, app, and database servers
  19. 19. Northern Virginia Availability Zone #1 Availability Zone #1 RDPGW Web / app WSFC2 Oregon Availability Zone #2 www.awsdrdemo.com
  20. 20. Northern Virginia Availability Zone #1 Availability Zone #1 RDP GW / NAT WSFC2 Oregon Availability Zone #2 RDP GW / NAT RDP GW / NAT
  21. 21. Northern Virginia Availability Zone #1 Availability Zone #1 RDPGW Web / app WSFC2 Oregon Availability Zone #2 AD1 AD2 AD3
  22. 22. Northern Virginia Availability Zone #1 Availability Zone #1 RDPGW WSFC2 Oregon Availability Zone #2 Web / app Web / app Web & App AMI
  23. 23. Northern Virginia Availability Zone #1 Availability Zone #1 Web / app Oregon Availability Zone #2 AD1 SQL1 AD2 SQL2 AD3 SQL3 ASYNC Data Replication
  24. 24. Availability Zone #1 Availability Zone #2 www.awsdrdemo.com RDP GW RDP RDGPW GW Web / app Web / app Web / app Web / app Web / app Web / app AD SQL2 AD1 SQL1 AD2 Sync Data Replication
  25. 25. Northern Virginia Availability Zone #1 Availability Zone #1 RDP GW RDPGW Web / app Oregon Availability Zone #2 RDP GW AD1 AD2 AD3 Web / app Web / app SQL1 SQL2 SQL3 www.awsdrdemo.com Async Data Replication VPN
  26. 26. Northern Virginia Availability Zone #1 APP failover.awsdrdemo.com
  27. 27. http://www.awsdrdemo.com/
  28. 28. http://www.awsdrdemo.com/
  29. 29. Check live status at http://status.awsdrdemo.com/ha
  30. 30. Check live status at http://status.awsdrdemo.com/dr
  31. 31. HttpWebResponse webresponse; string uri = http://169.254.169.254/latest/meta-data/placement/availability- zone; string retValue; /* Create webrequest for the uri */ HttpWebRequest webrequest = (HttpWebRequest)WebRequest.Create(uri); /* Capture response and read the stream */ webresponse = (HttpWebResponse)webrequest.GetResponse(); Encoding enc= System.Text.Encoding.GetEncoding(1252); StreamReaderloResponseStream= new StreamReader(webresponse.GetResponseStream(), enc); retValue = loResponseStream.ReadToEnd();
  32. 32. "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "AWS CloudFormation Template ELBWithEC2Instances: Create a load balanced, Auto Scaled sample website where theinstances are locked down to only accept traffic from the load balancer. This script creates an Auto Scaling group behind a load balancer with a simple health check. The web site is available on port 80, however, the instances can be configured to listen on any port (8888 by default).", "Parameters" : { "KeyPairName" : { "Description" : "Name of an existing Amazon EC2 key pair for SSH access", "Type" : "String", "Default" : "kamalkey" }, "InstanceType" : { "Description" : "WebServerEC2 instance type", "Type" : "String", "Default" : "m3.medium", "AllowedValues" : [ "t1.micro","m1.small","m1.medium","m1.large","m1.xlarge","m2.xlarge","m2.2xlarge","m2.4xlarge","m3.medium","m3.large","m3.xlarge","m3.2xlarge","c1.medium","c1.xlarge"], "ConstraintDescription" : "must be a valid EC2 instance type." }, "WebServerPort" : { "Description" : "TCP/IP port of the web server", "Type" : "String", "Default" : "80" }, "HostedZoneId" : { "Type" : "String", "Description" : "The Record Set's Hosted Zone Id for the existing hosted zone", "Default" : "Z1M58G0W56PQJA" } }, "Mappings" : { "AWSInstanceType2Arch" : { "t1.micro" : { "Arch" : "64" }, "m1.small" : { "Arch" : "64" }, "m1.medium" : { "Arch" : "64" }, "m1.large" : { "Arch" : "64" }, "m1.xlarge" : { "Arch" : "64" }, "m2.xlarge" : { "Arch" : "64" }, "m2.2xlarge" : { "Arch" : "64" }, "m2.4xlarge" : { "Arch" : "64" }, "m3.medium" : { "Arch" : "64" }, "m3.large" : { "Arch" : "64" }, "m3.xlarge" : { "Arch" : "64" }, "m3.2xlarge" : { "Arch" : "64" }, "c1.medium" : { "Arch" : "64" }, "c1.xlarge" : { "Arch" : "64" } }, "AWSRegionArch2AMI" : { "us-east-1":{"32" : "ami-06ef6b6e", "64" : "ami-06ef6b6e" } } }, "Resources" : { "WebServerGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "AvailabilityZones" : [ "us-east-1c","us-east-1d"], "LaunchConfigurationName" : { "Ref" : "LaunchConfig" }, "MinSize" : "2", "MaxSize" : "2", "LoadBalancerNames" : [ { "Ref" : "ElasticLoadBalancer" }], "VPCZoneIdentifier" : ["subnet-d8f2edf0","subnet-2fc76658"], "Tags" :[ { "Key" : "Name", "Value" : "DR-DEMOAPPWEBSERVER", "PropagateAtLaunch" : true }] } }, "LaunchConfig" : { "Type" : "AWS::AutoScaling::LaunchConfiguration", "Properties" : { "ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" }, { "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" }, "Arch" ] } ] }, "UserData" : { "Fn::Base64" : { "Ref" : "WebServerPort" }}, "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ], "InstanceType" : { "Ref" : "InstanceType" }, "KeyName" : { "Ref" : "KeyPairName" }, "AssociatePublicIpAddress" : "false" } }, "ElasticLoadBalancer" : { "Type" : "AWS::ElasticLoadBalancing::LoadBalancer", "Properties" : { "SecurityGroups" : [ { "Ref" : "LoadBalancerSecurityGroup" } ], "Subnets" : ["subnet-90cdd6b8","subnet-6024f917"], "Listeners" : [ { "LoadBalancerPort" : "80", "InstancePort" : { "Ref" : "WebServerPort" }, "Protocol" : "HTTP" } ], "HealthCheck" : { "Target" : { "Fn::Join" : [ "", ["HTTP:", { "Ref" : "WebServerPort" }, "/index.html"]]}, "HealthyThreshold" : "2", "UnhealthyThreshold" : "10", "Interval" : "10", "Timeout" : "3" } } }, "LoadBalancerSecurityGroup" : { "Type" : "AWS::EC2::SecurityGroup", "Properties" : { "GroupDescription" : "Enable HTTP access on port 80", "VpcId" : "vpc-0eef506b", "SecurityGroupIngress" : [ { "IpProtocol" : "tcp", "FromPort" : "80", "ToPort" : "80", "CidrIp" : "0.0.0.0/0" } ], "SecurityGroupEgress" : [ { "IpProtocol" : "tcp", "FromPort" : { "Ref" : "WebServerPort" }, "ToPort" : { "Ref" : "WebServerPort" }, "CidrIp" : "0.0.0.0/0" } ] } }, "myDNS" : { "Type" : "AWS::Route53::RecordSetGroup", "Properties" : { "HostedZoneName" : "awsdrdemo.com.", "Comment" : "Zone apex alias targeted to myELBLoadBalancer.", "RecordSets" : [ { "Name" : "www.awsdrdemo.com.", "Type" : "A", "AliasTarget" : { "HostedZoneId" : { "Fn::GetAtt" : ["ElasticLoadBalancer", "CanonicalHostedZoneNameID"] }, "DNSName" : { "Fn::GetAtt" : ["ElasticLoadBalancer","CanonicalHostedZoneName"] } } } ] } }, "InstanceSecurityGroup" : { "Type" : "AWS::EC2::SecurityGroup", "Properties" : { "GroupDescription" : "Enable RDP access and HTTP access on the inbound port", "VpcId" : "vpc-0eef506b", "SecurityGroupIngress" : [ { "IpProtocol" : "tcp", "FromPort" : { "Ref" : "WebServerPort" }, "ToPort" : { "Ref" : "WebServerPort" }, "CidrIp" : "0.0.0.0/0" }, { "IpProtocol" : "tcp", "FromPort" : "3389", "ToPort" : "3389", "CidrIp" : "0.0.0.0/0" } ] } } }, "Outputs" : { "URL" : { "Description" : "URL of the website", "Value" : { "Fn::Join" : [ "", [ "http://", { "Fn::GetAtt" : [ "ElasticLoadBalancer", "DNSName" ]}]]} } } } Create ELB Create Auto Scaling group Launch instances Route 53 recordset updates Status notifications
  33. 33. # Retrieve ELB Zone Id from existing Route53 zone zoneid=$(aws --output text route53 list-resource-record-sets --hosted-zone-id $hostedzoneid --start-record-name $domainname--start-record-type A --max-items 1 | grepALIASTARGET | awk{'print $4'}) # Retrieve ELB DNS name from existing Route53 zone dns=$(aws --output text route53 list-resource-record-sets --hosted-zone-id $hostedzoneid --start-record-name $domainname--start-record-type A --max-items 1 | grepALIASTARGET | awk{'print $2'}) # Remove existing Alias from Route53 hosted zone aws --region us-west-1 route53 change-resource-record-sets --hosted-zone-id $hostedzoneid --change-batch "{ "Changes": [ { "Action": "DELETE", "ResourceRecordSet": { "Name": "$domainname", "Type": "A", "AliasTarget": { "HostedZoneId": "$zoneid", "DNSName": "$dns", "EvaluateTargetHealth": false } } } ]}" # Launch DR stack using CloudFormation script launchedstackid=$(aws --region $snsregion--output text cloudformation create-stack --stack-name $stackname--template- body file:///usr/local/bin/ELBWithEC2Instances.template --notification-arns$snsarn--parameters ParameterKey="HostedZoneId",ParameterValue="$hostedzoneid") # Check the CloudFormationstatus do cmd=$(aws cloudformation describe-stacks --region $snsregion--output text --stack-name $stackname--max-items 1 | grepCREATE_COMPLETE)
  34. 34. "Parameters" : { "KeyPairName" : { "Description" : "Name of an existing Amazon EC2 key pair for SSH access", "Type" : "String", "Default" : “drkey" }, "InstanceType" : { "Description" : "WebServerEC2 instance type", "Type" : "String", "Default" : "m3.medium", "AllowedValues" : [ "t1.micro","m1.small","m1.medium","m1.large","m1.xlarge","m2.xlarge","m2.2xlarge","m2.4xlarge","m3.medium","m3.large","m3.xlarge","m3.2xlarge","c1.medium","c1.xlarge"], "ConstraintDescription" : "must be a valid EC2 instance type." }, "WebServerPort" : { "Description" : "TCP/IP port of the web server", "Type" : "String", "Default" : "80" }, "HostedZoneId" : { "Type" : "String", "Description" : "The Record Set's Hosted Zone Id for the existing hosted zone", "Default" : "Z1M58G0W56PQJA" } },
  35. 35. "Resources" : { "WebServerGroup" : { "Type" : "AWS::AutoScaling::AutoScalingGroup", "Properties" : { "AvailabilityZones" : [ "us-east-1c","us-east-1d"], "LaunchConfigurationName" : { "Ref" : "LaunchConfig" }, "MinSize" : "2", "MaxSize" : "2", "LoadBalancerNames" : [ { "Ref" : "ElasticLoadBalancer" }], "VPCZoneIdentifier" : ["subnet-d8f2edf0","subnet-2fc76658"], "Tags" :[ { "Key" : "Name", "Value" : "DR-DEMOAPPWEBSERVER", "PropagateAtLaunch" : true }]}}, "LaunchConfig" : { "Type" : "AWS::AutoScaling::LaunchConfiguration", "Properties" : { "ImageId" : { "Fn::FindInMap" : [ "AWSRegionArch2AMI", { "Ref" : "AWS::Region" }, { "Fn::FindInMap" : [ "AWSInstanceType2Arch", { "Ref" : "InstanceType" },"Arch" ] } ] }, "UserData" : { "Fn::Base64" : { "Ref" : "WebServerPort" }}, "SecurityGroups" : [ { "Ref" : "InstanceSecurityGroup" } ], "InstanceType" : { "Ref" : "InstanceType" }, "KeyName" : { "Ref" : "KeyPairName" }, "AssociatePublicIpAddress" : "false" }
  36. 36. "Resources" : { "ElasticLoadBalancer" : { "Type" : "AWS::ElasticLoadBalancing::LoadBalancer", "Properties" : { "SecurityGroups" : [ { "Ref" : "LoadBalancerSecurityGroup" } ], "Subnets" : ["subnet-90cdd6b8","subnet-6024f917"], "Listeners" : [ { "LoadBalancerPort" : "80", "InstancePort" : { "Ref" : "WebServerPort" }, "Protocol" : "HTTP”} ], "HealthCheck" : { "Target" : { "Fn::Join" : [ "", ["HTTP:", { "Ref" : "WebServerPort" }, "/index.html"]]}, "HealthyThreshold" : "2", "UnhealthyThreshold" : "10", "Interval" : "10", "Timeout" : "3”}}}, "LoadBalancerSecurityGroup" : { "Type" : "AWS::EC2::SecurityGroup", "Properties" : { "GroupDescription" : "Enable HTTP access on port 80", "VpcId" : "vpc-0eef506b", "SecurityGroupIngress" : [ { "IpProtocol" : "tcp", "FromPort" : "80", "ToPort" : "80", "CidrIp" : "0.0.0.0/0" } ]
  37. 37. "myDNS" : { "Type" : "AWS::Route53::RecordSetGroup", "Properties" : { "HostedZoneName" : "awsdrdemo.com.", "Comment" : "Zone apex alias targeted to myELBLoadBalancer.", "RecordSets" : [ { "Name" : "www.awsdrdemo.com.", "Type" : "A", "AliasTarget" : { "HostedZoneId" : { "Fn::GetAtt" : ["ElasticLoadBalancer", "CanonicalHostedZoneNameID"] }, "DNSName" : { "Fn::GetAtt" : ["ElasticLoadBalancer","CanonicalHostedZoneName"] } } } ]
  38. 38. BufferedReaderin = newBufferedReader(newInputStreamReader(request.getInputStream(); while((line = in.readLine()) != null) { jsonBody+=line; } if(request.getHeader("x-amz-sns-message-type").equalsIgnoreCase("SubscriptionConfirmation")){ System.out.println("ProcessingSubscriptionMessage!"); StringsubscribeURL= SNSJSONParser.parseJSONString(jsonBody, "SubscribeURL"); try{ sendGet(subscribeURL); System.out.println("Send Confirmation GET to subscription URL"); } catch(Exception e) { e.printStackTrace(); System.err.println("Error confirming subscription via GET request..");} }else{ System.out.println("Processing Notification Message !"); StringsnsMessage= SNSJSONParser.parseJSONString(jsonBody, "Message");s String resourceStatusReason = getSubString(snsMessage, "ResourceStatusReason"); message+="<li>"+ resourceId+ " :: "+ resourceStatus+ " :: "+ resourceStatusReason+ "</li>”;}else{ message+= "<li>"+ snsMessage+ "</li>"; } }
  39. 39. (99.99%) (0-2 minutes) 0-15 minutes)
  40. 40. http://bit.ly/awsevals
  41. 41. Availability Zone #1 Availability Zone #2 www.awsdrdemo.com RDP GW RDP RDGPW GW Web / app Web / app Web / app Web / app Web / app Web / app AD sql2 AD1 AD2 SQL1
  42. 42. Northern Virginia Availability Zone #1 Availability Zone #1 RDP GW RDPGW RDP GW Web / app Oregon Availability Zone #2 RDP GW AD1 AD2 AD3 Web / app Web / app SQL1 SQL2 SQL3 www.awsdrdemo.com Web / app Web / app

×