Building Private Clouds for HPC with OpenNebula<br />
Building Private Clouds for HPC with OpenNebula: Reference Deployments & Lessons Learned<br />Cloud Computing in a Nutshel...
Cloud Computing Disciplines<br />An Introduction to Cloud Computing<br />Software as a Service<br />Platform as a Service<...
Infrastructure as a Service (IaaS)<br />An Introduction to Cloud Computing<br />Public Cloud<br /><ul><li>Simple Web Inter...
 Raw Infrastructure Resources
 Pay-as-you-go (On-demand access)
 Elastic & “infinite” Capacity</li></li></ul><li>Infrastructure as a Service (IaaS)<br />An Introduction to Cloud Computin...
 Raw Infrastructure Resources
 Pay-as-you-go (On-demand access)
 Elastic & “infinite” Capacity</li></ul>Private Cloud<br />A “Public Cloud behind the firewall”<br /><ul><li> Simplify int...
 Dynamic allocation of resources
 Higher utilization & operational savings
 Security concerns</li></li></ul><li>Infrastructure as a Service (IaaS)<br />An Introduction to Cloud Computing<br />Publi...
 Raw Infrastructure Resources
 Pay-as-you-go (On-demand access)
 Elastic & “infinite” Capacity</li></ul>Private Cloud<br />A “Public Cloud behind the firewall”<br /><ul><li> Simplify int...
Upcoming SlideShare
Loading in …5
×

Building Private Clouds for HPC with OpenNebula: Reference Deployments & Lessons Learned

2,084 views
2,011 views

Published on

Published in: Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,084
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
45
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Building Private Clouds for HPC with OpenNebula: Reference Deployments & Lessons Learned

  1. 1. Building Private Clouds for HPC with OpenNebula<br />
  2. 2. Building Private Clouds for HPC with OpenNebula: Reference Deployments & Lessons Learned<br />Cloud Computing in a Nutshell<br />HPC & IaaS Clouds: Two+ Basic Approaches<br />Sample Deployments: CERN & Fermilab<br />Lessons Learned<br />A Third Approach: Grid & Cloud Computing<br />
  3. 3. Cloud Computing Disciplines<br />An Introduction to Cloud Computing<br />Software as a Service<br />Platform as a Service<br />Infrastructure as a <br />Service<br />Physical Infrastructure<br />
  4. 4. Infrastructure as a Service (IaaS)<br />An Introduction to Cloud Computing<br />Public Cloud<br /><ul><li>Simple Web Interface
  5. 5. Raw Infrastructure Resources
  6. 6. Pay-as-you-go (On-demand access)
  7. 7. Elastic & “infinite” Capacity</li></li></ul><li>Infrastructure as a Service (IaaS)<br />An Introduction to Cloud Computing<br />Public Cloud<br /><ul><li> Simple Web Interface
  8. 8. Raw Infrastructure Resources
  9. 9. Pay-as-you-go (On-demand access)
  10. 10. Elastic & “infinite” Capacity</li></ul>Private Cloud<br />A “Public Cloud behind the firewall”<br /><ul><li> Simplify internal operations
  11. 11. Dynamic allocation of resources
  12. 12. Higher utilization & operational savings
  13. 13. Security concerns</li></li></ul><li>Infrastructure as a Service (IaaS)<br />An Introduction to Cloud Computing<br />Public Cloud<br /><ul><li> Simple Web Interface
  14. 14. Raw Infrastructure Resources
  15. 15. Pay-as-you-go (On-demand access)
  16. 16. Elastic & “infinite” Capacity</li></ul>Private Cloud<br />A “Public Cloud behind the firewall”<br /><ul><li> Simplify internal operations
  17. 17. Dynamic allocation of resources
  18. 18. Higher utilization & operational savings
  19. 19. Security concerns</li></ul>Hybrid Cloud<br /><ul><li>Suplement the capacity of the Private Cloud
  20. 20. Utility Computing dream made a reality!</li></li></ul><li>High Performance & IaaS Cloud Computing<br />The pre-cloud time<br />Access<br />Grid Middleware<br />LRMS (LSF, PBS, SGE…)<br />The pre-virtualization/cloud<br />times<br />Provision<br />
  21. 21. High Performance & IaaS Cloud Computing<br />Two Basic Approaches: AnInfrastructureTool<br /><ul><li>Traditional interfaces
  22. 22. Grid Integration
  23. 23. Batch Processing
  24. 24. Custom Execution Environments</li></ul>Access<br />Grid Middleware<br />LRMS (LSF, PBS, SGE…)<br />Service-Provisioning<br />Decoupling<br />Service<br />Virtual WorkerNodes<br /><ul><li>Consolidation of WNs
  25. 25. Simplified management
  26. 26. Physical – Virtual WNs
  27. 27. Dynamic capacity partitioning</li></ul>Provision<br />
  28. 28. High Performance & IaaS Cloud Computing<br />Two Basic Approaches: A ProvisioningTool<br /><ul><li>Simple Provisioning Interface
  29. 29. Raw/Appliance VMs </li></ul>Access<br />IaaS Interface<br />PilotJobs, SSH…<br /><ul><li>Custom access to capacity
  30. 30. Not only batch workloads
  31. 31. Not only scientific workloads</li></ul>Service<br /><ul><li>Improve utilization
  32. 32. Reduced service management
  33. 33. Cost efficiency</li></ul>Provision<br />
  34. 34. High Performance & Cloud Computing<br />Usinghybridcloudcomputing<br /><ul><li>Extend provisioning layer
  35. 35. IT managed as an Utility
  36. 36. Interoperability
  37. 37. Networking Management
  38. 38. Appliance Management
  39. 39. Cost
  40. 40. Peaks
  41. 41. Fault-Tolerance</li></ul>Provision<br />External Providers<br />Physical Resources<br />
  42. 42. Examples of HPC Cloud Deployments<br />TheCERN’slxcloud<br />Goal & Characteristics<br />Virtualization of Batch Nodes: limted life-time, dynamic capacity<br />Golden Nodes + CERNVM<br />EC2 Query – OpenNebula API <br />Some Technical Notes<br />Custom Network Integration MAC/IP pinning<br />Disk I/O Tunning<br />Fast boot of VMs: Bittorrent + LVM snapshots<br />OpenNebula tests up to 16000 VMs (great feedback!)<br />
  43. 43. Examples of HPC Cloud Deployments<br />Fermicloud<br />Goal & Characteristics<br />Scientific stakeholders get access to VMs (no sysadmin needed)<br />Developers & Integrators of new Grid applications <br />Improve utilization of power, cooling and humans <br />Legacy applications<br />Some Technical Notes<br />VMs access Fermilab Networking and Storage Services<br />Using KVM<br />OpenNebula + X509 support (contributed back!)<br />Other Areas: HA, Batch queues look-ahead, cluster on-demand…<br />
  44. 44. Lessons Learned<br />HPC & Cloud Computing<br />Do not hide the Complexity, Architect the Cloud!<br /><ul><li>Let IT staff adapt/desgin/develop their cloud deployments
  45. 45. Design to Automate
  46. 46. Scale your deployments: Storage</li></ul>Be Interoperable, Have a Rich set of API’s & Interfaces <br /><ul><li>Be prepared for completely different uses cases (don’t look at the service)
  47. 47. Be prepared to deal with different solutions (storage, hypervisors, networks)
  48. 48. Be prepared to interoperate at different levels (API’s, Cloud services)</li></ul>And for HPC Cloud builders…<br /><ul><li>Scientists want it their way (VM images, do not put access restrictions)
  49. 49. Deal with tunning : CPU & I/O (image formats, hypervisor I/O policies)</li></li></ul><li>A Third Approach: Grid and Clouds<br />TheStratusLab Project<br />Clouds<br />Grids<br />Usage<br /><ul><li>Raw infrastructure
  50. 50. Elasticity & Pay-per-use
  51. 51. Simple Web Interface </li></ul>Achievements <br /><ul><li>Agile Infrastructures
  52. 52. IT another Utility</li></ul>But… <br /><ul><li>Interoperability
  53. 53. Federation</li></ul>Usage<br /><ul><li>Job Processing
  54. 54. Big Batch System
  55. 55. File Sharing Services </li></ul>Achievements <br /><ul><li>Federation of Resources
  56. 56. VO Concept</li></ul>But… <br /><ul><li>User experience
  57. 57. Complexity</li></ul>Resource Sharing<br />Resource Management<br />Scientific Applications<br />Customize Environments<br />Flexibility & Simplicity<br />Uniform Security<br />
  58. 58. A Third Approach: Grid & Clouds<br />TheStratusLab Project<br />GridServices<br />MarketPlace<br /><ul><li> Federation facilities
  59. 59. Security
  60. 60. Sharing existing VM images</li></ul>Multi Cloud<br />Cloud/GridSites<br />GridServices<br />Cloud API<br />GridServices<br />Cloud API<br />Cloud Bursting<br /><ul><li>Needs to consider Image & VLANs
  61. 61. Needs to consider placement</li></ul>ExternalClouds<br />
  62. 62. Building Private Clouds for HPC with OpenNebula: Reference Deployments & Lessons Learned<br />@ruben_s_montero<br />blog.opennebula.org<br />The research leading to these results has received funding from the European Union's Seventh Framework Programme ([FP7/2007-2013] ) under grant agreement n°261552 (StratusLab Project)<br />© OpenNebula Project. Creative Commons Attribution-NonCommercial-ShareAlike License<br />

×