Intro to Cloud Computing and Amazon EC2


Published on

Find out what cloud computing is, why it's important, and how it can be used to build a highly scalable, cost-effective, agile infrastructure.

Presented by Michael T. Conigliaro at the Frontier Developers Meetup in Boulder Colorado on 2010-09-09.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Intro to Cloud Computing and Amazon EC2

  1. 1. Intro to Cloud Computing and Amazon EC2<br />Michael T. Conigliaro<br /><br /><br />2010-09-09<br />
  2. 2. The Old Days…<br />Need many physical servers for redundancy & future growth<br />One operating system per physical server<br />
  3. 3. What’s The Problem?<br /><ul><li>Hardware is expensive!
  4. 4. And don’t forget the electric bill!
  5. 5. Long lead time to procure new hardware (days? weeks?)
  6. 6. Encourages overestimation of hardware needs
  7. 7. Static infrastructure leads to underutilization
  8. 8. According to some studies, the average server is only ~20% utilized!
  9. 9. Application consolidation?
  10. 10. Be careful not to “put all your eggs in one basket”
  11. 11. Complicates upgrades</li></li></ul><li>Static infrastructure is wasteful<br />
  12. 12. Enter Virtualization<br />Many virtual “instances” per physical server<br />More efficient utilization of hardware<br />Rapid provisioning (measured in minutes or seconds!)<br />Live migration of virtual instances between physical servers!<br />Management API<br />
  13. 13. Problems with Virtualization<br />Still need to purchase physical hardware<br />Less wasteful, but infrastructure is still mostly static<br />“Agile” instances require shared storage (SAN) which can be very expensive<br /> Note: These may not be problems for large enterprises or SMBs, but they certainly are for startups!<br />
  14. 14. Enter Cloud Computing<br />Cloud computing is virtualization on someone else’s hardware<br />Utility model - Pay only for what you’re using (per hour, GB, etc.)<br />A management API for your entire infrastructure!<br />Specific API features depend on your cloud provider<br />
  15. 15. Dynamic infrastructure creates less waste<br />
  16. 16. Why Amazon EC2?<br />Very popular<br />Very mature management tools and APIs<br />Integration with other Amazon services<br />S3 (simple online storage)<br />CloudFront (content delivery network)<br />Other nice features<br />Elastic Block Storage<br />Elastic Load Balancers<br />
  17. 17. Amazon EC2 Pricing (1/2)<br />Not necessarily cheaper!<br />Save money by reducing time to provision new servers<br />APIs are an absolute must-have!<br />Save money by reducing waste<br />Make infrastructure follow your utilization curve<br />Provision entire environments on demand, then terminate when not in use (e.g. testing)<br />
  18. 18. Amazon EC2 Pricing (2/2)<br />On-demand Linux instances for US-East. Price/Month assumes running 24/7.<br />
  19. 19. Getting Started with Amazon EC2<br />Create an account at<br />Make note of your “access key id” and “secret access key”<br />Used for REST-based requests (e.g. many UI-based tools)<br />Create and download an X.509 certificate and private key<br />Used for SOAP-based requests (e.g. the command line tools)<br />Create an SSH key pair, and download the private key<br />Used to connect to your instances via SSH<br />
  20. 20. Amazon EC2 Management Tools<br />AWS Management Console<br /><br />Elasticfox extension for Firefox<br /><br />Amazon API tools (command line)<br /><br />“apt-get install ec2-api-tools”<br />Language specific libraries<br />Java, Ruby, Python, Etc.<br />Too many to list!<br />
  21. 21. Amazon EC2 Dashboard<br />
  22. 22. Demo: EC2 Instance Lifecycle<br />Launch instance<br /><ul><li>ec2-run-instances ami-2d4aa444 --instance-type m1.small --key ec2-keypair</li></ul>Describe instance<br />ec2-describe-instances <instance_id><br />Connect to instance<br />ssh -i /Users/mikec/projects/socialmedia/infrastructure/ec2-keypair.pem ubuntu@<address><br />Terminate instance<br />ec2-terminate-instances <instance_id><br />
  23. 23. Demo: Start “Staging” Environment on Demand<br />Issue EC2 commands to start staging instances<br />Wait a few minutes for instances to come up<br />Interact with staging environment<br />Issue EC2 commands to stop staging instances<br />
  24. 24. Amazon EC2 Concepts (1/5)<br />New instances consist of two main things:<br />Amazon Machine Image (AMI) usually provided by Amazon or the community:<br />Operating system (Linux distro, version, etc.)<br />Datacenter location (East/West coast, Europe, Asia, etc.)<br /> Instance “type”:<br />Processor architecture (x86 or x86_64)<br />Number of virtual CPUs (1 - 8)<br />Amount of memory (600MB - 70GB)<br />Amount of local disk storage (up to 1690GB)<br />I/O performance (“low,” “moderate,” or “high”)<br />
  25. 25. Amazon EC2 Concepts (2/5)<br /> Data is ephemeral by default!<br />Rebooting is harmless, but termination of an instance results in loss of all data on the disk!<br /> Solution: Elastic Block Store (EBS) volumes<br />Create a new EBS volume<br />Attach EBS volume to an instance<br />When instance terminates, EBS volume is left behind<br /> Many AMIs now have EBS boot volumes!<br />Allows instances to be “stopped” <br />Highly recommended for critical and “long-running” instances<br />
  26. 26. Amazon EC2 Concepts (3/5)<br /> IP addresses are temporary and assigned randomly via DHCP!<br /> Solution 1: Elastic IPs<br />Create a new elastic IP<br />Assign it to an instance<br />Instance can now be accessed reliably via the elastic IP address<br /> Solution 2: Dynamic DNS (e.g.<br />Each instance updates its own public DNS records via an API<br />See:<br />
  27. 27. Amazon EC2 Concepts (4/5)<br /> No way of knowing where your instances are physically located!<br />Example: Launch two new instances at the same time. Will they end up running… <br />On the same physical server? <br />On different servers in the same rack? <br />On different servers in different racks?<br /> Why does this matter? Latency-sensitive applications may not do well on EC2.<br />
  28. 28. Amazon EC2 Concepts (5/5)<br /> On-demand instance requests are not guaranteed to be fulfilled immediately, or even at all!<br />EC2 quotas<br />Insufficient EC2 capacity<br /> Solution: “Reserved instances”<br />Pay for instances up front rather than by the hour<br />Requires one-to-three year contract with Amazon<br />1-year: save ~34%<br />3-year: save ~49%<br />
  29. 29. Some Best Practices<br />Use “security groups” to secure instances from each other<br />Prevent unintentional termination of critical instances<br />ec2-modify-instance-attribute --disable-api-termination true <instance id><br />Avoid “image sprawl”<br />Start with one minimal “golden master image”<br />Use a configuration management system to add “roles” as necessary (e.g. Web server, Database server, NFS server, etc.)<br />
  30. 30. What’s Next?<br />Use a configuration management system<br />Chef, Puppet, Cfengine, etc.<br />“Infrastructure as Code”<br />Automate your “bootstrap” process<br />Plan for failure!<br />Every EC2 instance is as likely to fail as any other<br />Since failure cannot be prevented, have a good recovery plan<br />Command and Control<br />Capistrano, Fabric, etc.<br />
  31. 31. Example Chef Recipe<br />package "apache" do<br />package_name "apache2”<br /> action :install<br />end<br />service "apache" do<br />service_name "apache2"<br /> action [:enable, :start]<br />end<br />template "apache2.conf" do<br /> source "apache2.conf.erb"<br /> path "/etc/apache2/apache2.conf"<br /> notifies :restart, resources(:service => ”apache")<br />end<br />
  32. 32. Summary<br />Cloud Computing is virtualization on someone else’s hardware<br />Cloud computing allows your infrastructure to be dynamic, and thus, less wasteful<br />Cloud APIs promote automation and allow you to be more productive<br />Configuration management systems allow you to treat your infrastructure as code<br />
  33. 33. More Information<br />EC2 Getting Started Guide<br /><br />EC2 Developer Guide<br /><br />Chef wiki<br /><br />