AWS Summit 2013 | Auckland - Your First Week with Amazon EC2


Published on

Amazon Elastic Compute Cloud (Amazon EC2) provides resizable compute capacity in the cloud and is often the starting point for your first week using AWS. This session will introduce these concepts, along with the fundamentals of EC2, by employing an agile approach that is made possible by the cloud. Attendees will experience the reality of what a first week on EC2 looks like from the perspective of someone deploying an actual application on EC2. You will follow them as they progress from deploying their entire application from an EC2 AMI on day 1 to more advanced features and patterns available in EC2 by day 5. Throughout the process we will identify cloud best practices that can be applied to your first week on EC2 and beyond.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

AWS Summit 2013 | Auckland - Your First Week with Amazon EC2

  1. 1. Adrian WhiteYour First Week on Amazon EC2Solutions Architect, AWSSteven Stones-HavasSenior Developer, BiomattersGuest presenter:
  2. 2. Questions for Your First Week on Amazon EC2• What is Amazon EC2?• Where do I start with EC2?– What are the components of EC2?– What are the big picture architecture cloud patterns?– What other Amazon Web Services should I use?• How do I map my existing infrastructure architecture to EC2?– How do I configure my environment for high availability?– How do manage my environment in the cloud?– How do I monitor my environment in the cloud?
  3. 3. An Approach to Your First Week on Amazon EC2• Leverage what you already know about web architectures• Understand enough to get started with EC2• Take an iterative approach– Refactor and evolve– Pay for what you use• Understand and apply cloud best practices– Capacity on demand– Elasticity– Design for failure– Infrastructure automation
  4. 4. Day 1 – Identify and Deploy Application on EC2RegionAvailability ZoneLinuxApacheRubyMySQLSource Protocol Port0.0.0.0/0 HTTP 800.0.0.0/0 SSH 22
  5. 5. Day 1 – Launching Your First EC2 Instance1. Login to the AWS Management Console and go to the Amazon EC2 console2. Choose an Amazon Machine Image (AMI)3. Choose an instance size4. Create a key pair for SSH access5. Create port-based security rules6. Launch instance7. Upload code
  6. 6. Day 1 – Choose AMI
  7. 7. Day 1 – Instance Details
  8. 8. Day 1 – Tags
  9. 9. Day 1 – Create Key Pair
  10. 10. Day 1 – Configure Firewall
  11. 11. Day 1 – Instance Launched
  12. 12. Day 1 – Application Tasks[laptop]$ ssh -i ~/ec2.pem ec2-user@ec2-54-242-199-31.compute-1.amazonaws.com__| __|_ )_| ( / Amazon Linux AMI___|___|___| are 13 security update(s) out of 24 total update(s) availableRun "sudo yum update" to apply all updates.[ec2-user@ip-10-40-203-29 ~]$ sudo yum -y -q update[ec2-user@ip-10-40-203-29 ~]$ sudo yum -y -q install mysql-server ruby19[ec2-user@ip-10-40-203-29 ~]$ sudo service mysqld startStarting mysqld: [ OK ]
  13. 13. Day 1  Day 2Day 1 Recap Day 2 Considerations1. Created an AWS account2. Identified an application for clouddeployment3. Logged into the Amazon EC2 console4. Chose an AMI5. Launched an EC2 Instance6. Set up application• What options do we have for settingup a tiered architecture?• How can we apply security to ourinstances?• Are there options for serving staticcontent?• How can we capture our work effortsto make them repeatable?
  14. 14. Day 2 – Create a tiered architectureRegionAvailability ZoneSnapshot Amazon S3InternetUserHTTP (80)Source ProtocolPort0.0.0.0/0 HTTP 800.0.0.0/0 SSH 22Connection Type DetailsEC2 SecurityGroupwebS3 Bucket
  15. 15. Day 2 – Launching a Tiered Web Application1. Snapshot EC2 Instance– Stop MySQL– Bundle New AMI2. Create a Relational Database (RDS) Instance– We’ll use MySQL– Other options: Oracle, SQL Server3. Configure App to Use RDS MySQL Database
  16. 16. Day 2 – Create a snapshot of our AMI
  17. 17. Day 2 – RDS DB Instance Details
  18. 18. Day 2 – RDS Management Options
  19. 19. Day 2 – Granting EC2 App Access to RDS
  20. 20. Day 2 – Connect to RDS Database[ec2-user@ip-10-40-203-29 ~]$ mysql -uroot –p –D devdb –h to the MySQL monitor. Commands end with ; or g.Your MySQL connection id is 268Server version: 5.5.27-log Source distributionCopyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.Type help; or h for help. Type c to clear the current input statement.mysql>
  21. 21. Day 2 – Connect to RDS Database (encrypted)[ec2-user@ip-10-40-203-29 ~]$ wget[ec2-user@ip-10-40-203-29 ~]$ mysql -uroot –p –D devdb –h --ssl_ca=rds-ssl-ca-cert.pemWelcome to the MySQL monitor. Commands end with ; or g.Your MySQL connection id is 269Server version: 5.5.27-log Source distributionCopyright (c) 2000, 2012, Oracle and/or its affiliates. All rights reserved.Type help; or h for help. Type c to clear the current input statement.mysql>
  22. 22. Day 2  Day 3Day 2 Recap Day 3 Considerations1. Took a snapshot of AMI as a backup2. Created an RDS MySQL Database3. Created and validated security groups• What tools does AWS provide tomonitor EC2 and RDS?• How can we better monitor the ourenvironment (proactive vs. reactive)?• How can we be notified when ourservers hits certain thresholds?
  23. 23. Day 3 – Monitor EnvironmentRegionAvailability ZoneInternet UserS3 BucketAmazonCloudWatchUsersAlarmAdministratorEmail Notification
  24. 24. Day 3 – Create CloudWatch Alarm1. Select metric to monitor– Database write latency is an accurate indicator of our application’s health2. Define a threshold– Write latency that exceeds 500ms typically requires some intervention on our part3. Create a topic for our alarm and subscribe to the topic via email
  25. 25. Day 3 – Create Alarm
  26. 26. Day 3 – Create Alarm
  27. 27. Day 3 – Create Alarm
  28. 28. Day 3 – Alarm Created
  29. 29. Day 3  Day 4Day 3 Recap Day 4 Considerations1. Identified CloudWatch metricsavailable for EC2 and RDS2. Created a CloudWatch alarm3. Set up alarm to email on failure4. Reviewed CloudWatch dashboard• What happens if our EC2 instancefails?• What happens if an entire AZ isunavailable?• How can we elastically scale basedon increased/decreased traffic?• What happens if our primary RDSinstance fails?
  30. 30. Day 4 – Designing for High AvailabilityRegionAvailability ZoneInternetS3 BucketAmazonCloudWatchUsersAlarmAvailability ZoneRDS DB StandbyAuto scaling Group
  31. 31. Day 4 – Steps to High Availability1. Create an Elastic Load Balancer (ELB)– Balances traffic across multiple EC2 instances– Enables running instances in multiple Availability Zones (AZ’s)2. Configure Auto Scaling– Automatically scale up if demand increases– And scale down to save money3. Setup RDS Multi-AZ– Synchronous replication to standby in another AZ– Automatic fails over if needed– Also minimizes backup window (slave is used)
  32. 32. Day 4 – Define Load Balancer
  33. 33. Day 4 – Configure Health Check
  34. 34. Day 4 – Add EC2 Instance(s)
  35. 35. Day 4 – Elastic Load Balancer is Active
  36. 36. Day 4 – Configure Auto Scaling1. Use the Amazon Machine Image (AMI) we created2. Leverage multiple Availability Zones– Distribute instances across two AZ’s– Ensure at least two instances are up3. Create an Auto Scaling trigger– Same concept as CloudWatch alarm from earlier– Just now we’re proactively taking action
  37. 37. Day 4 – Find That AMI We Created
  38. 38. Day 4 – Set Up Auto Scaling[laptop]$ aws autoscaling create-launch-configuration --launch-configuration-name webcfg --image-id ami-08dc4461 --instance-type m1.small --region us-east-1[laptop]$ aws autoscaling create-auto-scaling-group --auto-scaling-group-name webscg --launch-configuration-name webcfg --availability-zones us-east-1a us-east-1c --min-size 2 --max-size 10 --load-balancer-names frontlb
  39. 39. Day 4 – Check on Our Instances
  40. 40. Day 4 – Set Up RDS Multi-AZ[laptop]$ aws rds modify-db-instance --db-instance-identifier nonprod --multi-az --region us-east-1Yep, that’s it.No mouse required. :)
  41. 41. Day 4  Day 5Day 4 Recap Day 5 Considerations1. Spread our application acrossAvailability Zones.2. Automated scaling across availabilityzone leveraging Auto Scaling.3. Implemented load balancing via AWSElastic Load Balancing.4. Implemented a highly availabledatabase by applying RDS multi-AZ.• How do we make use of a customDNS domain for our load balancer?• How can we configure accounts forother AWS users?• How can we template and replicateour server environment?
  42. 42. Day 5 – DNS, Identity & Access Management, Deployment AutomationRegionAvailability ZoneInternetS3 BucketAmazonCloudWatchUsersAlarmAvailability ZoneRDS DB StandbyAWS IAMwww.example.comAWS ManagementConsoleAWSCloudFormationTemplateStack
  43. 43. Day 5 – Route 53 (DNS)
  44. 44. Day 5 – Identity & Access Management
  45. 45. Day 5 – Deployment Automation
  46. 46. First Week on Amazon EC2• Evolution from Day 1  Day 5– Single AMI  Monitored  Tiered  HA  DNS, IAM, Automation• Cloud architecture best practices implemented in week 1 on EC2– Proactive scaling – Auto scaling triggers– Elasticity – EC2– Design for failure – ELB, Auto scaling groups, Availability Zones– Decouple your components – EC2, RDS– Infrastructure automation – CloudFormation
  47. 47. …and Beyond• Moving beyond week 1 on EC2– AWS Management Console is great but you have other options• Command Line Interface• API– Other AWS Services• Elasticache, OpsWorks, Beanstalk, DynamoDB, SQS– Operational Checklist•– Deployment Automation•– Links to whitepapers and architectures••
  48. 48. And now, a customer who went beyond…
  49. 49. BIOMATTERSSteven Stones-HavasSenior Developer, Biomatters
  50. 50. • Founded in 2003• Sophisticated, intuitive Visualisation and Interpretation ofGenetic data• Targeted Analysis Workflows• Actionable Results• We’d love to talk to you!
  51. 51. Genome Browser - Requirements• Smooth, intuitive experience in the browser– JavaScript/HTML5– Mobile friendly• Tile Rendering– Like Google Maps– Requires fast database lookups• Secure– Data must be encrypted at rest and in transit• Local-deployable– Some customers not ready for cloud
  52. 52. Architecture• Initial Architecture– On EC2– One autoscaling group (and ELB)– One Availability Zone• Revised Architecture– VPC across two Availability Zones– Private subnets for security
  53. 53. VPC ArchitectureELBPublic Subnets Private SubnetsAZ 1AZ 2MASTERMIN=0 MAX=2MIN=1 MAX=3DB ClusterInternetNAT
  54. 54. Web Stack• Tomcat behind Apache• Session info stored in Elasticache• Monitoring– Healthcheck Ping URL for the load balancer– Cloudwatch CPU alarms for autoscaling• Autoscaling– Scales from 2 to 6 machines depending on load– For > 6 machines, the database becomes the bottleneck• Deployment– Automatic deployment with no downtime
  55. 55. Automatic Deployment1. Deploy latest code to master web node (through Tomcatmanager)2. Shutdown master tomcat3. Take AMI snapshot4. Restart master webnode, and wait for ping URL to respond5. Teardown existing autoscaling config6. Set up new autoscaling config
  56. 56. Database• Local Deployment Requirement– Can’t use RDS or Dynamo• MongoDB– Highly scalable NoSQL– Supports Advanced features
  57. 57. Database• Base unit – pair of 50GB volumes in Raid0• 100GB Logical Volume (LVM)• Encryption Layer• XFS File System– Can grow without unmounting• Scaling– Storage scaling is manual– Performance scaling could be automatic• Need to scale preemptively
  58. 58. Job ProcessingDB ClusterIncoming Job QueueWeb AppProcessing NodeCompleted Job QueueNotification NodeS3Status=NEWStatus=PROCESSINGStatus=COMPLETEStatus=NOTIFIED
  59. 59. Overview• Multi-Availability Zone VPC with public and private subnets• ELB in front of Auto-Scaling web nodes• Statically scaled MongoDB Cluster• Encrypted volumes• Simple Queue Service for job processing• We’d love to talk to you!
  60. 60. AWS SupportSteven DaySite Leader, AWS Support
  61. 61. Built on top of Legacy of Customer Obsession
  62. 62. AWS Support is a Global Organizationwith an Australian presenceCurrent Sites2013 ExpansionRemote TAMOur team consists of professional,highly skilled engineers withlocations in North America,Europe, Australia, Asia and Africa.
  63. 63. Support Product FeatureMatrix• Different levels of support tomatch the support needs of ourcustomers
  64. 64. More Than Just Break-fix• AWS Support is much more than traditional, reactive troubleshooting.• In addition to 24/7/365 reactive break-fix with highly skilled engineers,support subscriptions includes an unlimited number of cases to:1. Help you get started with AWS2. Get recommendations to be more secure, lower cost, and more available3. Discuss your architecture and best practices4. Ask questions on how to successfully integrate the 150+ annual AWS feature releases5. Configuration help for a growing list of 3rd Party Software
  65. 65. Support Center & Trusted Advisor APIs• Customers can use their existing ticketing systems to manage theirsupport cases, receive case updates, and access AWS TA results.• April 30th: Announcing General Availability of Support Center andTrusted Advisor APIs
  66. 66. Thank you!