This document discusses using private clouds for high-performance computing and describes deployments at CERN and Fermilab. It outlines two approaches for HPC and IaaS clouds and lessons learned, including the need to automate and scale deployments, ensure interoperability, and address scientists' requirements for customization and access. It also proposes a hybrid grid/cloud approach under the StratusLab project to achieve agility while maintaining federation and a uniform user experience.

1. Building Private Clouds for HPC with OpenNebula

2. Building Private Clouds for HPC with OpenNebula: Reference Deployments & Lessons Learned Cloud Computing in a Nutshell HPC & IaaS Clouds: Two+ Basic Approaches Sample Deployments: CERN & Fermilab Lessons Learned A Third Approach: Grid & Cloud Computing

3. Cloud Computing Disciplines An Introduction to Cloud Computing Software as a Service Platform as a Service Infrastructure as a Service Physical Infrastructure

5. Raw Infrastructure Resources

6. Pay-as-you-go (On-demand access)

8. Raw Infrastructure Resources

9. Pay-as-you-go (On-demand access)

11. Dynamic allocation of resources

12. Higher utilization & operational savings

14. Raw Infrastructure Resources

15. Pay-as-you-go (On-demand access)

17. Dynamic allocation of resources

18. Higher utilization & operational savings

22. Grid Integration

23. Batch Processing

25. Simplified management

26. Physical – Virtual WNs

27. Dynamic capacity partitioningProvision

30. Not only batch workloads

32. Reduced service management

33. Cost efficiencyProvision

35. IT managed as an Utility

36. Interoperability

37. Networking Management

38. Appliance Management

39. Cost

40. Peaks

41. Fault-ToleranceProvision External Providers Physical Resources

42. Examples of HPC Cloud Deployments TheCERN’slxcloud Goal & Characteristics Virtualization of Batch Nodes: limted life-time, dynamic capacity Golden Nodes + CERNVM EC2 Query – OpenNebula API Some Technical Notes Custom Network Integration MAC/IP pinning Disk I/O Tunning Fast boot of VMs: Bittorrent + LVM snapshots OpenNebula tests up to 16000 VMs (great feedback!)

43. Examples of HPC Cloud Deployments Fermicloud Goal & Characteristics Scientific stakeholders get access to VMs (no sysadmin needed) Developers & Integrators of new Grid applications Improve utilization of power, cooling and humans Legacy applications Some Technical Notes VMs access Fermilab Networking and Storage Services Using KVM OpenNebula + X509 support (contributed back!) Other Areas: HA, Batch queues look-ahead, cluster on-demand…

45. Design to Automate

47. Be prepared to deal with different solutions (storage, hypervisors, networks)

50. Elasticity & Pay-per-use

54. Big Batch System

57. ComplexityResource Sharing Resource Management Scientific Applications Customize Environments Flexibility & Simplicity Uniform Security

59. Security

61. Needs to consider placementExternalClouds

62. Building Private Clouds for HPC with OpenNebula: Reference Deployments & Lessons Learned @ruben_s_montero blog.opennebula.org The research leading to these results has received funding from the European Union's Seventh Framework Programme ([FP7/2007-2013] ) under grant agreement n°261552 (StratusLab Project) © OpenNebula Project. Creative Commons Attribution-NonCommercial-ShareAlike License