Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Exploring The Cloud


Published on

Slides on exploring cloud computing technology given to VCGR

Published in: Education, Spiritual
  • Be the first to comment

  • Be the first to like this

Exploring The Cloud

  1. 1. Exploring the Cloud Chris Sosa, Dr. Andrew Grimshaw sosa, grimshaw @ University of Virginia
  2. 2. Introduction - 1 <ul><li>Ever-increasing demand for computing resources </li></ul><ul><li>During non-peak times, computing resources sit idle </li></ul><ul><ul><li>Still paying! </li></ul></ul><ul><ul><li>Power, cooling, etc </li></ul></ul><ul><li>Total Cost of Ownership (TCO) is much more than the cost of Hardware </li></ul><ul><ul><li>Maintenance </li></ul></ul><ul><ul><li>Administration </li></ul></ul><ul><ul><li>Cooling </li></ul></ul><ul><ul><li>Etc. </li></ul></ul>University of Virginia
  3. 3. Introduction - 2 <ul><li>Observation – load on main ITC clusters exhibit bimodal distribution </li></ul><ul><li>Can we only pay for what we use? </li></ul>University of Virginia
  4. 4. Enter Cloud Computing (field trip!) <ul><li>What is it? </li></ul><ul><ul><li>Infrastructure-related capabilities provided as a service </li></ul></ul><ul><ul><li>Also known as utility-computing and is associated with very basic API’s </li></ul></ul><ul><li>Lots of industry support </li></ul><ul><ul><li>Amazon Infrastructure Services: EC2, S3, … </li></ul></ul><ul><ul><li>Google App Engine </li></ul></ul><ul><ul><li>Microsoft Azure </li></ul></ul><ul><ul><li>IBM led initiatives </li></ul></ul>University of Virginia
  5. 5. Cloud Computing Paradigms <ul><li>Top-down: Client only provides program and deployment information </li></ul><ul><ul><li>Microsoft Azure </li></ul></ul><ul><ul><li>Google App Engine </li></ul></ul><ul><li>Bottom-up: Raw Infrastructure provided (virtualized hardware) </li></ul><ul><ul><li>Amazon </li></ul></ul><ul><ul><li>Nirvanix </li></ul></ul><ul><ul><li>Flexiscale </li></ul></ul><ul><ul><li>GoGrid </li></ul></ul>University of Virginia
  6. 6. Advantages and Disadvantages of Using the Cloud <ul><li>Advantages </li></ul><ul><ul><li>Pay for what you use – model is based on how long you use resources. You can allocate and deallocate them on-the-fly </li></ul></ul><ul><ul><li>Hardware cost, set-up time, maintenance, cooling all go down to zero </li></ul></ul><ul><ul><li>Can start developing immediately </li></ul></ul><ul><li>Disadvantages </li></ul><ul><ul><li>No control over physical resources. Do you trust Amazon? </li></ul></ul><ul><ul><li>SLA’s may not be good enough. Is 99.95% availability good enough? </li></ul></ul><ul><ul><li>Some limitations in what you can run. Must stay within the API / framework given </li></ul></ul>University of Virginia
  7. 7. Why Cloud Computing <ul><li>Only have to pay for what we use </li></ul><ul><li>Disadvantages do not affect most users in a batch system </li></ul>University of Virginia
  8. 8. Amazon Leading the Push <ul><li>Amazon has been most successful player so far </li></ul><ul><ul><li>Over 29 billion objects stored on S3 </li></ul></ul><ul><ul><li>Using over 60% of their resources for Cloud services </li></ul></ul><ul><li>EC2 just went out of Beta in October (new) </li></ul><ul><li>… rest of these slides will assume we use Amazon </li></ul>University of Virginia
  9. 9. Outline <ul><li>Introduction </li></ul><ul><li>Overview of Amazon Cloud Services </li></ul><ul><li>Proposal of Hybrid Scheduler </li></ul><ul><li>Questions to be Answered </li></ul><ul><li>Conclusion </li></ul>University of Virginia
  10. 10. Amazon S3 <ul><li>Simple Storage for the Internet </li></ul><ul><li>Applications can interact with various mechanisms </li></ul><ul><ul><li>REST </li></ul></ul><ul><ul><li>SOAP </li></ul></ul><ul><ul><li>Bit Torrent </li></ul></ul><ul><li>250 Mb/second network link </li></ul><ul><li>Objects stored in buckets </li></ul><ul><ul><li>Buckets have own namespace </li></ul></ul><ul><ul><li>Up to 100 buckets per account </li></ul></ul><ul><ul><li>Unlimited objects per bucket </li></ul></ul><ul><ul><li>5 GB limit on size of objects </li></ul></ul><ul><ul><li>Objects are write-once </li></ul></ul><ul><li>SLA guarantees 99.9% availability </li></ul>University of Virginia
  11. 11. S3 Pricing <ul><li>Storage </li></ul><ul><ul><li>$0.15 per GB-Month of storage used </li></ul></ul><ul><li>Data Transfer </li></ul><ul><ul><li>$0.10 per GB - all data transfer in </li></ul></ul><ul><ul><li>$0.18 per GB - first 10 TB / month data transfer out </li></ul></ul><ul><ul><li>$0.16 per GB - next 40 TB / month data transfer out </li></ul></ul><ul><ul><li>$0.13 per GB - data transfer out / month over 50 TB </li></ul></ul><ul><ul><li>FREE to EC2 </li></ul></ul><ul><li>Requests </li></ul><ul><ul><li>$0.01 per 1,000 PUT or LIST requests </li></ul></ul><ul><ul><li>$0.01 per 10,000 GET and all other requests* * No charge for delete requests </li></ul></ul>University of Virginia
  12. 12. Amazon EC2 <ul><li>Provides Virtual Compute Resources </li></ul><ul><ul><li>Purchase CPU’s on hourly basis </li></ul></ul><ul><ul><li>Can use provided virtual machine images, or make own </li></ul></ul><ul><ul><li>Virtual Machines run atop Xen </li></ul></ul><ul><li>Can do meta data operations with REST, SOAP, command-line tools </li></ul><ul><li>Instances assigned IP address for SSH, remote desktop, etc </li></ul><ul><li>SLA guarantees 99.95% availability </li></ul>University of Virginia
  13. 13. EC2 Pricing <ul><li>Instances </li></ul><ul><ul><li>$0.10 / hr - Small Instance - 1.7 GB of memory, 1 EC2 Compute Unit (1 virtual core - 1.7 GHz processor), 160 GB of instance storage, 32-bit platform (can buy in sets of 1, 4, 8) </li></ul></ul><ul><ul><li>$0.20 / hr - High-CPU Medium Instance 1.7 GB of memory, 5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each), 350 GB of instance storage, 32-bit platform (can buy in sets of 1 or 4) </li></ul></ul><ul><li>Data Transfer </li></ul><ul><ul><li>$0.10 per GB – data in </li></ul></ul><ul><ul><li>$0.18 per GB - first 10 TB out </li></ul></ul><ul><ul><li>FREE to S3 </li></ul></ul>University of Virginia
  14. 14. Overview <ul><li>Introduction </li></ul><ul><li>Overview of Amazon Cloud Services </li></ul><ul><li>Proposal of Hybrid Scheduler </li></ul><ul><li>Questions to be Answered </li></ul><ul><li>Conclusion </li></ul>University of Virginia
  15. 15. Main Idea <ul><li>Reduce the number of resources we have active and improve peak performance </li></ul><ul><li>Modify local scheduler </li></ul><ul><ul><li>When CPU usage is above threshold, allocate new machines from EC2 and schedule jobs </li></ul></ul><ul><ul><li>As usage decreases, deallocate resources and return to normal usage </li></ul></ul>University of Virginia
  16. 16. Design University of Virginia
  17. 17. Research Setup <ul><li>Instead of spending funds on running experiments using EC2 and S3, we will be using Eucalyptus to emulate EC2 </li></ul><ul><ul><li>Eucalyptus is an open-source implementation of the EC2 interface </li></ul></ul><ul><ul><li>Requires Xen be installed on host machines (need dedicated machines) </li></ul></ul><ul><li>Create a centralized repository for data for our tests (S3) </li></ul><ul><ul><li>NFS share </li></ul></ul><ul><ul><li>Other possibilities? </li></ul></ul>University of Virginia
  18. 18. Task Bar <ul><li>Decide on the software that will be installed on the virtual machines </li></ul><ul><ul><li>PBS licensing is complicated and expensive </li></ul></ul><ul><ul><li>Several alternatives such as Genesis II, Hadoop, etc. </li></ul></ul><ul><li>Create AMI image and register with Eucalyptus </li></ul><ul><li>Incorporate virtual machines from Eucalyptus into existing scheduler and create mechanism to do this on-the-fly </li></ul><ul><li>Modify scheduler to take into account a threshold </li></ul><ul><li>Build stubs to measure how much bandwidth, time, etc. is being used by the scheduler so that we can determine the price we would be charged by Amazon's EC2 and S3 </li></ul><ul><li>Incorporate these costs, build economic model using actual workloads at UVa, differing thresholds, and various ways of passing jobs to the Cloud </li></ul>University of Virginia
  19. 19. Overview <ul><li>Introduction </li></ul><ul><li>Overview of Amazon Cloud Services </li></ul><ul><li>Proposal of Hybrid Scheduler </li></ul><ul><li>Questions to be Answered </li></ul><ul><li>Conclusion </li></ul>University of Virginia
  20. 20. Questions to be Answered <ul><li>What is the Cost Model associated with working with Cloud computing? </li></ul><ul><li>What costs would be associated with common jobs being run at UVa? </li></ul><ul><li>What software will we have installed on the Virtual Machines in the Cloud? </li></ul><ul><li>How can we create a threshold such that we can decide on-the-fly when to start offloading resources to Cloud resources? </li></ul>University of Virginia
  21. 21. Overview <ul><li>Introduction </li></ul><ul><li>Overview of Amazon Cloud Services </li></ul><ul><li>Proposal of Hybrid Scheduler </li></ul><ul><li>Questions to be Answered </li></ul><ul><li>Conclusion </li></ul>University of Virginia
  22. 22. Conclusions <ul><li>Important to be concerned about reducing costs as well as getting bigger bang for your buck </li></ul><ul><li>Offloading job processing to Cloud computing infrastructures can save costs while improving peak throughput </li></ul>University of Virginia
  23. 23. Questions? University of Virginia