Masterclass Webinar: Amazon EC2

3,141 views

Published on

Developing applications on Amazon Web Services (AWS) or moving your business into the cloud is more straightforward than you think. Whether you are a developer eager to learn new skills, a solutions architect who wants to solve existing technology problems, the IT professional who wants access to cost-effective, on-demand computing resources, this workshop is for you.

These slides feature some of the most popular Amazon Web Services: Amazon Elastic Compute Service (EC2), Amazon Simple Storage Service (S3), Amazon CloudFront, Amazon Elastic Block Storage (EBS) and Amazon Relational Database Service (RDS).

Amazon EC2 YouTube Recording: http://youtu.be/TORzO9Oc9oU
Amazon EC2 Demo: http://youtu.be/kMExnVKhmYc

Published in: Technology
1 Comment
7 Likes
Statistics
Notes
No Downloads
Views
Total views
3,141
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
211
Comments
1
Likes
7
Embeds 0
No embeds

No notes for slide

Masterclass Webinar: Amazon EC2

  1. 1. MasterclassElastic Compute CloudRyan Shuttleworth – Technical Evangelist @ryanAWS
  2. 2. Masterclass A technical deep dive beyond the basicsHelp educate you on how to get the best from AWS technologies Show you how things work and how to get things done Broaden your knowledge in ~45 mins
  3. 3. Amazon EC2 On-demand compute to run application workloads Easy come easy go – disposable resourceWe provide the infrastructure, you decide what you run
  4. 4. Complete controlElastic capacity Flexible What is EC2? Reliable Secure Inexpensive
  5. 5. Elastic capacity Customer 1 Customer 2 … Customer n Hypervisor Securely Virtual Interfaces segregated Customer 1 Security Customer 2 Security … Customer n Security Shared Groups Groups Groups Firewallenvironment Physical Interfaces
  6. 6. Elastic capacity Customer 1 Customer 2 … Customer n Hypervisor Securely Virtual Interfaces segregated Customer 1 Security Customer 2 Security … Customer n Security Shared Groups Groups Groups Firewallenvironment Physical Interfaces
  7. 7. AMIAmazon Machine Image
  8. 8. Instance AMIAmazon Machine Running or Image Stopped machine
  9. 9. EC2 Instance VPC AMIAmazon Machine Running or Image Stopped machine
  10. 10. EC2 Instance VPC AMI AZAmazon Machine Running or Image Stopped machine Region
  11. 11. EC2 EC2 Instance VPC VPC AMI AZ Availability ZoneAmazon Machine Running or Image Stopped machine Region
  12. 12. EC2 EC2 Instance VPC VPC AMI EBS EBS EBS EBS EBS EBS AZ Availability ZoneAmazon Machine Running or Image Stopped machine Region
  13. 13. EC2 EC2 Instance VPC VPC AMI EBS EBS EBS EBS EBS EBS AZ Availability ZoneAmazon Machine Running or Image Stopped machine EBS S3 Buckets Snapshots S3 Region
  14. 14. Instance
  15. 15. Unit of controlInstance Unit of scale Unit of resilience
  16. 16. Unit of controlYour stack Instance Unit of scale Unit of resilience
  17. 17. Instance Unit of controlScale out Instance Unit of scale Instance Unit of resilience Instance
  18. 18. Instance Unit of controlInstance Unit of scaleInstance Unit of resilienceInstance
  19. 19. Instance Unit of controlInstance Unit of scaleInstance Unit of resilienceInstance
  20. 20. Instance Unit of controlInstance Unit of scale Unit of resilienceInstance
  21. 21. Instance Unit of controlInstance Unit of scaleInstance Unit of resilienceInstance
  22. 22. Instance typesChoose the right unit for your workload
  23. 23. High I/O 4XL 60.5 GB High Storage 8XL 117 GB Hi-Mem Cluster Compute 8XL 35 EC2 Compute Units 35 EC2 Compute Units, 244 GB 16 virtual cores 24 * 2 TB ephemeral drives 88 EC2 Compute Units 2*1024 GB SSD-based local instance storage 256 10 GB Ethernet 16 virtual cores 240 GB SSD Hi-Mem 4XL 68.4 GB 26 EC2 Compute Units 128 8 virtual cores 10 GB Hi-Mem 2XL 34.2 GB 13 EC2 Compute Units Inter-Instance Cluster Compute 8XL 60.5 GB 88 EC2 Compute Units 64 4 virtual cores Hi-Mem XL 17.1 GB Network Cluster Compute 4XL 23 GB 6.5 EC2 Compute Units 33.5 EC2 Compute Units 32 2 virtual coresMemory (GB) Cluster GPU 4XL 22 GB 33.5 EC2 Compute Units, Extra Large 15 GB 2 x NVIDIA Tesla “Fermi” 16 8 EC2 Compute Units 4 virtual cores M3 XL 15 GB M3 2XL 30 GB M2050 GPUs 13 EC2 Compute Units 26 EC2 Compute Units 4 virtual cores 8 virtual cores Medium 3.7 GB, EBS storage only EBS storage only 8 2 EC2 Compute Units 1 virtual core Large 7.5 GB 4 EC2 Compute Units High-CPU XL 7 GB 4 2 virtual cores 20 EC2 Compute Units 8 virtual cores Small 1.7 GB, High-CPU Med 1.7 GB 1 EC2 Compute Unit 5 EC2 Compute Units 1 virtual core 2 Micro 613 MB 2 virtual cores Up to 2 ECUs (for short bursts) 1 1 2 4 8 16 32 64 128 256 EC2 Compute Units
  24. 24. Start small Easy to up-size
  25. 25. AMIs Amazon Community Your machine maintained maintained imagesSet of Linux and Windows Images published by other AMIs you have created from images AWS users EC2 instancesKept up to date by Amazon Managed and maintained by Can be kept private or shared in each region Marketplace partners with other accounts
  26. 26. http://aws.amazon.com/amazon-linux-ami/
  27. 27. AMIs Linux Enterprise Linux WindowsSmall instance from Small instance from Small instance from $0.060 per hour $0.120 per hour $0.115 per hour Small instance from $0.090 per hour
  28. 28. Instance types On-demand instances Unix/Linux instances start at $0.02/hour Pay as you go for compute power Low cost and flexibility Pay only for what you use, no up-front commitments or long-term contracts Use Cases: Applications with short term, spiky, or unpredictable workloads; Application development or testing
  29. 29. Instance types On-demand instances Reserved instances Unix/Linux instances start at 1- or 3-year terms $0.02/hour Pay low up-front fee, receive significant hourly Pay as you go for compute power discount Low cost and flexibility Low Cost / Predictability Pay only for what you use, no up-front Helps ensure compute capacity is available commitments or long-term contracts when needed Use Cases: Use Cases: Applications with short term, spiky, or unpredictable workloads; Applications with steady state or predictable usage Application development or testing Applications that require reserved capacity, including disaster recovery
  30. 30. Instance types Heavy utilization RI > 80% utilization Lower costs up to 58% On-demand instances Reserved instances Use Cases: Databases, Large Scale HPC, Always-on infrastructure, Baseline Unix/Linux instances start at 1- or 3-year terms $0.02/hour Pay low up-front fee, receive significant hourly Pay as you go for compute power discount Low cost and flexibility Low Cost / Predictability Pay only for what you use, no up-front Helps ensure compute capacity is available commitments or long-term contracts when needed Use Cases: Use Cases: Applications with short term, spiky, or unpredictable workloads; Applications with steady state or predictable usage Application development or testing Applications that require reserved capacity, including disaster recovery
  31. 31. Instance types Heavy utilization RI > 80% utilization Lower costs up to 58% On-demand instances Reserved instances Use Cases: Databases, Large Scale HPC, Always-on infrastructure, Baseline Unix/Linux instances start at 1- or 3-year terms $0.02/hour Pay low up-front fee, receive significant hourly Medium utilization RI Pay as you go for compute power discount Low cost and flexibility Low Cost / Predictability 41-79% utilization Lower costs up to 49% Pay only for what you use, no up-front Helps ensure compute capacity is available Use Cases: Web applications, many heavy commitments or long-term contracts when needed processing tasks, running much of the time Use Cases: Use Cases: Applications with short term, spiky, or unpredictable workloads; Applications with steady state or predictable usage Application development or testing Applications that require reserved capacity, including disaster recovery
  32. 32. Instance types Heavy utilization RI > 80% utilization Lower costs up to 58% On-demand instances Reserved instances Use Cases: Databases, Large Scale HPC, Always-on infrastructure, Baseline Unix/Linux instances start at 1- or 3-year terms $0.02/hour Pay low up-front fee, receive significant hourly Medium utilization RI Pay as you go for compute power discount Low cost and flexibility Low Cost / Predictability 41-79% utilization Lower costs up to 49% Pay only for what you use, no up-front Helps ensure compute capacity is available Use Cases: Web applications, many heavy commitments or long-term contracts when needed processing tasks, running much of the time Use Cases: Use Cases: Applications with short term, spiky, or Light utilization RI unpredictable workloads; Applications with steady state or predictable usage Application development or testing 15-40% utilization Applications that require reserved capacity, Lower costs up to 34% including disaster recovery Use Cases: Disaster Recovery, Weekly / Monthly reporting, Elastic Map Reduce
  33. 33. Instance types On-demand instances Reserved instances Spot instances Unix/Linux instances start at 1- or 3-year terms Bid on unused EC2 capacity $0.02/hour Pay low up-front fee, receive significant hourly Spot Price based on supply/demand, Pay as you go for compute power discount determined automatically Low cost and flexibility Low Cost / Predictability Cost / Large Scale, dynamic workload handling Pay only for what you use, no up-front Helps ensure compute capacity is available commitments or long-term contracts when needed Use Cases: Use Cases: Use Cases: Applications with flexible start and end times Applications with short term, spiky, or unpredictable workloads; Applications with steady state or predictable Applications only feasible at very low compute usage prices Application development or testing Applications that require reserved capacity, including disaster recovery
  34. 34. Launch an instanceCommands, keypairs & security groups
  35. 35. RegionInstance size AMI Key pairSecurity group
  36. 36. key pairssecure access
  37. 37. Public Key Inserted by Amazon into each EC2 instance that you launch EC2 InstanceComms securedwith private key Private Key Downloaded and stored by you
  38. 38. Keypairs & Secrets Keypairs Credentials x.509Used to authenticate Access key and secret key Used to authenticatewhen accessing and used to authenticate against some APIs instance against APIs
  39. 39. security groups instance firewalling
  40. 40. Port 22 Name Security Group (SSH) DescriptionPort 80 Protocol(HTTP) Port range IP Address, range, or another security group instance
  41. 41. PS C:> New-EC2Instances -ImageId ami-269dbb63 -KeyName mykey -SecurityGroupId sg-9cf9e5d9 -InstanceType t1.micro
  42. 42. $> ec2-run-instances ami-54cf5c3d --instance-count 2 --group webservers --key mykey --instance-type m1.small
  43. 43. >>> import boto.ec2>>> conn = boto.ec2.connect_to_region("us-east-1")>>> conn.run_instances( ami-54cf5c3d, key_name=mykey, instance_type=m1.small, security_groups=[webservers])
  44. 44. Wait a minuteI want to use those tools too…
  45. 45. IAM Roles and EC2 tools1. Start an EC2 Linux instance2. Assign an IAM role at launch time:3. Sets up all the tools you need & manages { API access credentials "Statement": [ {1. Up and running with CLI tools in a couple "Effect": "Allow", "NotAction": "iam:*", of minutes – just SSH on and use "Resource": "*" }2. Terminate/stop instance when you are ] done }
  46. 46. Now you have tools Try this…
  47. 47. $> ec2-run-instances ami-54cf5c3d --instance-count 1
  48. 48. $> ec2-run-instances ami-54cf5c3d --instance-count 1 --group webservers --key mykey --instance-type m1.smallWhat about all this?
  49. 49. $> ec2-run-instances ami-54cf5c3d --instance-count 1 --group Default --key NONE --instance-type default(m1.small)Defaults
  50. 50. $> ec2-run-instances ami-54cf5c3d --instance-count 1 --group Default --key NONE --instance-type default(m1.small)
  51. 51. Instances don’t need keypairsBut how do you configure it if you can’t log onto it?
  52. 52. Bootstrapping Bake an AMI Start an instanceConfigure the instance Create an AMI from your instance Start new ones from the AMI
  53. 53. Bootstrapping Bake an AMI vs Configure dynamically Start an instance Launch an instanceConfigure the instance Use metadata service and cloud-init to Create an AMI from perform actions on your instance instance when it Start new ones from launches the AMI
  54. 54. Bootstrapping Bake an AMI + Configure dynamicallyBuild your base images Use bootstrapping to and setup custom pass custom initialisation scripts information in and perform post launchMaintain your ‘golden’ tasks like pulling code base from SVN
  55. 55. BootstrappingBake an AMI Configure dynamically Time consuming configuration (startup time) Static configurations (less change management)
  56. 56. BootstrappingBake an AMI Configure dynamically Continuous deployment (latest code) Environment specific (dev- test-prod)
  57. 57. Goal is bring an instance up in a useful stateThe balance will vary depending upon your application
  58. 58. Instance User request data
  59. 59. Instance User Meta-data request data service
  60. 60. Instance User Meta-data request data service Instance
  61. 61. Shell script in user-data will be executed on launch:#!/bin/shyum -y install httpd php mysql php-mysqlchkconfig httpd on/etc/init.d/httpd start
  62. 62. Amazon Windows EC2Config Service executes user-data on launch: <script>dir > c:test.log</script> <powershell>any command that you can run</powershell>AWS Powershell Tools (use IAM roles as before…) <powershell> Read-S3Object -BucketName myS3Bucket -Key myFolder/myFile.zip -File c:destinationFile.zip </powershell> 63
  63. 63. Automation Less fingers, less mistakes Security AvailabilityInstances locked Drive higherdown by default availability with self- healing Why do this? Flexible EfficiencyShell, Powershell, Audit and manageCloudFormation, your estate with less Chef, Puppet, Scale time & effort OpsWorks Manage large scale deployments and drive autoscaling
  64. 64. Some does and don’ts Do Use IAM roles Go keyless if you canStrike a balance between AMI and dynamic bootstrapping
  65. 65. Some does and don’ts Do Don’t Use IAM roles Put your API access keys into code (and then publish Go keyless if you can to GIT) or bake into AMIsStrike a balance between (and share) AMI and dynamic bootstrapping 
  66. 66. Block storageUnderstanding instance storage vs EBS
  67. 67. Instance Storage Local ‘on host’ disk volumesData dependent upon instance lifecycle
  68. 68. Instance Storage VS Elastic Block Storage Local ‘on host’ disk Network attached optimised volumes block storageData dependent upon Data independent of instance lifecycle instance lifecycle
  69. 69. Instance AInstance Storage Instance D Instance B Local ‘on host’ disk volumes Instance E Instance CData dependent upon Instance F instance lifecycle Instance Store Instance Store eph0 eph1 eph2 eph3 eph0 eph1 eph2 eph3 Host 1 Host 2
  70. 70. If an instance reboots (intentionally orInstance Storage unintentionally), data in the instance store persists Local ‘on host’ disk volumes Data on instance store volumes is lost underData dependent upon the following circumstances: instance lifecycle • Failure of an underlying drive • Stopping an Amazon EBS-backed instance • Terminating an instance
  71. 71. OptionsDiffering types ofinstance storage
  72. 72. OptionsDiffering types ofinstance storage
  73. 73. One or more ephemeral (temporary) drives (instance storage) One or more EBS (persistent) drives EBS snapshots (backup images) Elastic Block Storage Network attached optimised Workspace block storage Network Data independent of EBS snapshot instance lifecycle Hypervisor EC2 EBS S3
  74. 74. Boot cycle Elastic Block Storage Network attached optimised block storage Data independent of EBS snapshot instance lifecycle Hypervisor EC2 EBS S3
  75. 75. Boot cycle Elastic Block Storage Network attached optimised Workspace block storage Data independent of EBS snapshot instance lifecycle Hypervisor EC2 EBS S3
  76. 76. Boot cycle Elastic Block Storage Network attached optimised Workspace block storage Data independent of EBS snapshot instance lifecycle Hypervisor EC2 EBS S3
  77. 77. Boot cycle Elastic Block Storage Network attached optimised Workspace block storage Network Data independent of instance lifecycle Hypervisor EC2 EBS S3
  78. 78. EBS PersistenceEBS volume is off-instance storageYou pay for the volume usage as long as the datapersists1. By default, EBS volumes that are attached to a running instance automatically detach from the instance with their data intact when that instance is terminated2. By default, EBS volumes that are created and attached to an instance at launch are deleted when that instance is terminated. You can modify this behavior by changing the value of the flag DeleteOnTermination to false when you launch the instance.
  79. 79. Elastic Load Balancer Spreading the load and fronting EC2
  80. 80. A regional serviceLoad balance across availability zones
  81. 81. Elastic Load BalancerInstance Instance Instance Instance Instance Instance Availability Zone Availability Zone Availability Zone Region
  82. 82. Elastic Load Balancing Spread Offload Health check Go small and wide SSL processing on ELB Choose the right healthcheckBalance resources across Remove load from EC2 point AZs instances Check whole layers
  83. 83. 1. Persistent HTTP connections – enable them and ELB to Server will be optimized2. Never address underlying IP – always DNS name • There’s a set behind an ELB and real clients spread across them • They will change as the ELB scales to keep ahead of demand3. If you span ELB across AZs have an instance in all Azs4. De-register instances from an ELB before terminating
  84. 84. AutoScalingAutomate EC2 commissioning and decommisioning
  85. 85. Launch Configuration Auto-Scaling Group Auto-Scaling PolicyDescribes what Auto Scaling Auto Scaling managed Parameters for performing an will create when adding grouping of EC2 instances Auto Scaling action Instances Automatic health check to Scale Up/Down and by how much AMI maintain pool size Instance Type ChangeInCapacity (+/- #) Security Group ExactCapacity (#) Automatically scale the number of Instance Key Pair ChangeInPercent (+/- %) instances by policy – Min, Max, Desired Only one active launch Cool Down (seconds) configuration at a time Automatic Integration with ELB Policy can be triggered by Auto Scaling will terminate CloudWatch events Automatic distribution & balancing instances with old launch across AZs configuration first rolling update
  86. 86. Create a launch configuration: as-create-launch-config --image-id ami-54cf5c3d --instance-type m1.small --key mykey --group webservers --launch-config 101-launch-config
  87. 87. Create a launch configuration: as-create-launch-config --image-id ami-54cf5c3d --instance-type m1.small The usual --key mykey suspects --group webservers --launch-config 101-launch-config
  88. 88. Create an auto scaling group:as-create-auto-scaling-group 101-as-group --availability-zones us-east-1a us-east-1b us-east-1c --launch-configuration 101-launch-config --load-balancers myELB --max-size 5 --min-size 1
  89. 89. Create an auto scaling group:as-create-auto-scaling-group 101-as-group --availability-zones us-east-1a us-east-1b us-east-1c --launch-configuration 101-launch-config --load-balancers myELB --max-size 5 --min-size 1 What’s going to launch
  90. 90. Create an auto scaling group:as-create-auto-scaling-group 101-as-group --availability-zones us-east-1a us-east-1b us-east-1c --launch-configuration 101-launch-config --load-balancers myELB --max-size 5 --min-size 1 Integrate with an ELB?
  91. 91. Create an auto-scaling policy (scale up):as-put-scaling-policy 101ScaleUpPolicy --auto-scaling-group 101-as-group --adjustment=1 --type ChangeInCapacity --cooldown 300
  92. 92. Create an auto-scaling policy (scale up):as-put-scaling-policy 101ScaleUpPolicy --auto-scaling-group 101-as-group --adjustment=1 --type ChangeInCapacity --cooldown 300 Period before another action will take place (Damper)
  93. 93. Create an auto-scaling policy (scale down):as-put-scaling-policy 101ScaleDownPolicy --auto-scaling-group 101-as-group "--adjustment=-1" --type ChangeInCapacity --cooldown 300
  94. 94. CloudWatchKnow what is going on
  95. 95. Cloud Watch Alarm: Takes action: CPU >= 50% for 5 mins Scale up policy CPU < 30% for 10 mins Scale down policy
  96. 96. Cloud Watch Alarm: Takes action: CPU >= 50% for 5 mins Scale up policy
  97. 97. Cloud Watch Alarm: Takes action: CPU >= 50% for 5 mins Deliver message to Q SNS Topic Post to endpoint CPU < 30% for 10 mins Send Email
  98. 98. Cloud Watch Alarm: Takes action: CPU >= 50% for 5 mins SNS Topic
  99. 99. Comprehensive Billing, technical, aggregate & custom metrics SNS AlarmsIntegration Set custom alarmsPush alarms to and thresholds SNS topics CloudWatch HTTP Email Poke HTTP integration endpoints for Send alarm custom alarm Custom Metrics notifications to actions Write your own metrics in via emails SDKs
  100. 100. Other topics to look at:
  101. 101. Other topics…Resource tagging Route 53 Rolling deploymentsTag resources like EC2 Front EC2 and ELBs with Use Route 53 and ELBs to doand have it appear on Route 53 for control over rolling deployments, A/B billing reports DNS testing
  102. 102. Other topics… Beanstalk OpsWorks CloudFormation Manage an entire Manage stacks as layers Template everything from autoscaling stack for and implement Chef configuration of CloudWatchpopular containers such recipes to automate EC2 alarms, SNS topics, EC2 as ruby, python etc configuration instances
  103. 103. Summary
  104. 104. Stop doing these: Provisioning and fixing servers Treating compute as physical thingsThinking of compute as a finite commitment
  105. 105. Elasticity Stateless autoscaling Automation Security applications Create instances whenBuild systems secure by you need them, drop default them when not and start doing these Replace not fix Be cost awareBuild from scratch, don’t Unconstrained Tag resources, play with fix something instance types Say goodbye to traditional capacity planning
  106. 106. Watch a demo here:http://youtu.be/kMExnVKhmYc
  107. 107. aws.amazon.com

×