Download as PDF, PPTX
Uploaded byIgnacio M. Llorente
Federated Cloud Computing - The OpenNebula Experience v1.0s
The document discusses federated cloud computing with a focus on the OpenNebula project, covering its capabilities in managing virtualized data centers and its benefits for scientific and high-performance computing. It explains various deployment models, the significance of OpenNebula in building private cloud infrastructures, and examples of its application in institutions like CERN and Fermilab. Additionally, it highlights the challenges of interoperability and the vision for integrating cloud and grid infrastructures.
More Related Content
Similar to Federated Cloud Computing - The OpenNebula Experience v1.0s
Federated Cloud Computing - The OpenNebula Experience v1.0s
- 1. Workshop GARR -Calcolo e Storage Distribuito Rome, Italy, November 30th, 2012 Federated Cloud Computing The OpenNebula Experience Ignacio M. Llorente Project Director © OpenNebula Project. Creative Commons Attribution-NonCommercial-ShareAlike License
- 2. Contents Federated Cloud Computing- The OpenNebula Experience! This presentation is about: • Infrastructure as a Service Cloud Computing • Private Clouds in Science and HPC • Federated Cloud Computing • Cloud federation in Grid infrastructures Federated Cloud Computing - The OpenNebula Experience! 2/32
- 3. What is CloudComputing? Provision of IT Capabilities as a Service! What Who On-demand End-user Software as a Service access to any (does not care about hw or sw) application Platform for Developer Platform as a Service building and (no managing of the underlying hw delivering web & swlayers) applications Infrastructure as a Raw computer System Administrator ﾺ Service infrastructure (complete management of the computer infrastructure) Physical Infrastructure Federated Cloud Computing - The OpenNebula Experience! 3/32
- 4. What is CloudComputing? IaaS Clouds for Provision of Virtualized Resources as a Service! • Raw infrastructure • Management Console resources • Simple REST API’s Manage Instances • Pay-as-you-go & elastic capacity Federated Cloud Computing - The OpenNebula Experience! 4/32
- 5. What is CloudComputing? Different Models of Deployment! Model Infrastructure Cloud Cases Owned by a single • Optimize and simplify internal operation Private organization and made • SaaS/PaaS support available only to the • IT consolidation within large organizations organization (Goverment Clouds, University Clouds…) Owned by a single • Commercial cloud providers, mostly organization and made hosting providers, with limited control/ Public available to other security organizations over the • Science public clouds by ICT service Internet centers to enable scientific projects or experiment with cloud computing Owned by a single • Telecom cloud providers with premium Private Virtual organization and made solutions with additional control/security available to other organization over a dedicated private network Composition of two or • Cloudbursting to address peak demands • Cloud Federation to share infrastructure Hybrid more clouds with partners • Cloud Aggregation to provide a larger resource infrastructure Federated Cloud Computing - The OpenNebula Experience! 5/32
- 6. What is OpenNebula? OpenCloud Management Solution for Building and Managing Virtualized Data Centers! Public Private Hybrid Federated Cloud Computing - The OpenNebula Experience! 6/32
- 7. What is OpenNebula? MostAdvanced and Flexible, Enterprise-grade IaaS Cloud Manager! Adaptable ● Customizable and Extensible Proven ● Many Massive Scale Production Deployments Powerful ● Most Advanced Enterprise-class Functionality No Lock-in ● Platform Independent and Interoperable Interoperable ● Most popular cloud APIs and standard based Openness ● Fully open-source, Apache license Federated Cloud Computing - The OpenNebula Experience! 7/32
- 8. What is OpenNebula? User-drivenOpen Platform for Innovation! Adopt as innovation Standards Contribute to platform or Projects interoperability tool standards Contribute innovative Adopt functionality standards Distribution channel Linux Distributions Requirements Feedback Contributions Adopt open-source Industry and Research Federated Cloud Computing - The OpenNebula Experience! 8/32
- 9. Private Cloud inScience and HPC The Pre-cloud Era! Access Grid Middleware LRMS (LSF, PBS, SGE…) Provision Federated Cloud Computing - The OpenNebula Experience! 9/32
- 10. Private Cloud inScience and HPC OpenNebula as an Infrastructure Tool! Access Service/Provisioning Decoupling! • Common interfaces • Grid integration Grid Middleware LRMS (LSF, PBS, SGE…) • Custom environments Service • Dynamic elasticity Virtual Worker Nodes • Consolidation of WNs • Simplified management Provision • Physical – Virtual WNs • Dynamic capacity partitioning • Faster upgrades Federated Cloud Computing - The OpenNebula Experience! 10/32
- 11. Private Cloud inScience and HPC OpenNebula as an Provisioning Tool! • Simple Provisioning Interface Access • Raw/Appliance VMs IaaS Interface Pilot Jobs, SSH… Service • Dynamic scalable computing • Custom access to capacity • Not only batch workloads • Not only scientific workloads Provision • Improve utilization • Reduced service management • Cost efficiency Federated Cloud Computing - The OpenNebula Experience! 11/32
- 12. Private Cloud inScience and HPC Examples: CERN’s lxcloud! http://blog.opennebula.org/?p=620 Goal • Virtualized HTC Batch Nodes: limited life-time, dynamic capacity • EC2 Query: CernVM Deployment Notes • Custom network integration MAC/IP pinning • Fast image distribution & boot: BitTorrent + LVM snapshots • OpenNebula tests up to 20,000 VMs (great feedback!) Federated Cloud Computing - The OpenNebula Experience! 12/32
- 13. Private Cloud inScience and HPC Examples: FermiCloud! http://www-fermicloud.fnal.gov/ Goal • Scientific stakeholders get access to on-demand VMs • Developers & integrators of new Grid applications • MPI and legacy applications Deployment Notes • VMs access Fermilab Networking and Storage Services • OpenNebula + X509 support (contributed back!) • Other areas: HA, Batch queues look-ahead, cluster on-demand… Federated Cloud Computing - The OpenNebula Experience! 13/32
- 14. Private Cloud inScience and HPC Examples: SARA! https://www.cloud.sara.nl/ Goal • HPC clusters on-demand: Self service, elastic and fully configurable HPC systems • Offer production infrastructure for several engineering and scientific communities: Bioinformatics, ecology… Deployment Notes • Low latency network for HPC • Extensions in accounting, GUI, firewalling… (contributed back!) Federated Cloud Computing - The OpenNebula Experience! 14/32
- 15. Private Cloud inScience and HPC One of Our Main User Communities! Supercomputing Centers Research Centers Distributed Computing Infrastructures Federated Cloud Computing - The OpenNebula Experience! 15/32
- 16. Federated Cloud Computing NextStep in the Evolution of an Utility! Utility Utility Generation Distribution Utility Grid Federated Cloud Computing - The OpenNebula Experience! 16/32
- 17. Federated Cloud Computing Benefitsof Federation! Scalability ● Cloudbursting to address peak demands Collaboration ● Sharing of infrastructure between partners Multi-site Deployments ● Infrastructure aggregation across distributed data centers Reliability ● Fault tolerance architectures across sites Performance ● Deployment of services closer to end users Cost ● Dynamic placement to reduce the overall infrastructure cost Energy Consumption ● Minimize energy consumption Federated Cloud Computing - The OpenNebula Experience! 17/32
- 18. Federated Cloud Computing TheCluster Computing Use Case! Scalability Performance Cost Fault tolerance Federated Cloud Computing - The OpenNebula Experience! 18/32
- 19. Federated Cloud Computing DifferentLevels of Control, Monitoring, Cross-site Functionality and Security! Private Cloud Partner Company Cloud Cloud Cloud OS Public (Broker) Cloud Public Public Public Cloud 1 Cloud 2 Cloud 3 Federated Cloud Computing - The OpenNebula Experience! 19/32
- 20. Federated Cloud Computing LooselyCoupled Federation - Cloudbursting! Federation with a Cloud without Interoperation Support Corporate cloud Cloud OS Public Public Cloud 1 Cloud 2 Local data center Aim • Meet peak demands • Basic operations over VMs (start, shutdown, restart…) Control • Different instance types Monitoring & • Basic virtual resource monitoring (resource consumption…) Accounting Cross-site • None Security • Single account representing the organization Federated Cloud Computing - The OpenNebula Experience! 20/32
- 21. Federated Cloud Computing DynamicCombination of Local with Remote Cloud Resources! Service Level Agreement • SLO 1: Average_CPU_Utilization > 75% • Action: Execute remote WN Grid Services Federated Cloud Computing - The OpenNebula Experience! 21/32
- 22. Federated Cloud Computing PartiallyCoupled Federation - Aggregated Cloud Architecture! ! Federation with a Cloud with Partial Interoperation Support Corporate Corporate cloud 1 cloud 2 Cloud OS Cloud OS Local data center Local data center Aim • Sharing of resources to meet peak demands • Advanced operations over VMs (live migration…) Control • VM location and affinity constraints Monitoring & • Advanced virtual resource monitoring (energy consumption, Accounting VM placement…) • Virtual networks Cross-site • Virtual storage Security • Framework agreement Federated Cloud Computing - The OpenNebula Experience! 22/32
- 23. Federated Cloud Computing CloudBroker Architecture! Building Service Testbeds on FIRE Agreement 257386 (2010-2013) Design, build and operate a multi-site cloud-based facility to New Infrastructure Paradigms support research across applications, services and systems and Experimental Facilities targeting services research community on Future Internet www.BonFIRE-Project.eu Source: BonFIRE Project Federated Cloud Computing - The OpenNebula Experience! 23/32
- 24. Federated Cloud Computing TightlyCoupled Federation - Multi-tier Architecture! Federation with a Cloud with Advanced Interoperation Support Corporate cloud Cloud OS Cloud Cloud site 1 site 2 Cloud OS Cloud OS Local data center Local data center Aim • Very large scale or geographically distributed data centers • Placement on specific physical resources Control • Same instance types Monitoring & • Physical resource consumption Accounting • Live migration Cross-site • High availability Security • User space sharing Federated Cloud Computing - The OpenNebula Experience! 24/32
- 25. Federated Cloud Computing Multi-tierCloud Architecture! CloudFront • Portal • Cloud API (EC2, OCCI) User • Global AuthN OpenNebula Zone OpenNebula Zone • Multi-tenancy • Multi-tier arch. • Runs Instances • Scalable • Stores Images • Isolation • Networking • Multiple-site • Authorization support Federated Cloud Computing - The OpenNebula Experience! 25/32
- 26. Federated Cloud Computing Challengesfor Interoperability! Transparent Combination of Local Resources with Cloud Resources 1. Management Interfaces 2. Management Interfaces for Virtual Workloads for Data Elements VM VM VM VM VM 4. Contextualization 3. VM Image Description 5. Cross-site Networking and Instance Type and Latency 6. Common framework for security and QoS Federated Cloud Computing - The OpenNebula Experience! 26/32
- 27. Federated Cloud Computing LeveragingExisting Standards and Implementing Interoperation! Standardization ● Implement standards ● Integrate with standards Which Standard? ● Different de jure standards ● Several de facto standards Interoperation ● Implement adaptors ● Use transformers Federated Cloud Computing - The OpenNebula Experience! 27/32
- 28. Cloud Federation inGrid Infrastructures Grid and Cloud as Complementary Computing Models! Usage Grids Usage Clouds Job Processing Raw infrastructure Big Batch System Elasticity & Pay-per-use File Sharing Services Simple Web Interface Achievements Achievements Federation of Resources Agile Infrastructures VO Concept IT is another Utility But… User experience But… Interoperability Complexity Federation Resource Sharing Resource Management Scientific Applications Customize Environments Uniform Security Flexibility & Simplicity Federated Cloud Computing - The OpenNebula Experience! 28/32
- 29. Cloud Federation inGrid Infrastructures The OpenNebula Vision for Grid Sites! • Batch Job Processing • Industry Applications • Custom Execution Environments • Other WMS (pilots) Access • Grid Service Integration • Complete Services (cluster) Grid Middleware IaaS Interface LRMS (LSF, PBS…) Service Virtual CE, WN… Other (web, mail...) Raw machines Provision Grid Site Federated Cloud Computing - The OpenNebula Experience! External Providers 29/32
- 30. Cloud Federation inGrid Infrastructures The OpenNebula Vision for Grid Infrastructures ! Grid Services MarketPlace • Federation facilities • Security • Sharing existing VM images • Grid specific services • Registry of metadata • Image are kept elsewhere • Supports trust Appliance Repo • Storage VM images • Distributed • Multi-protocol Grid Services Cloud API Grid Services Cloud API Cloud/Grid Site Cloud/Grid Site Federated Cloud Computing - The OpenNebula Experience! 30/32
- 31. Join our GrowingCommunity! OpenNebula is Driven by User Needs ! How to contribute ● Join our mailing list ● Follow the development at dev.opennebula.org ● Contributions of by users: RIM, Akamai, Logica, FermiLab, SARA, Terradue, China Mobile… (approx. 100 listed at http://www.opennebula.org/ about:contributors) ● 500 validated users at dev.opennebula Ecosystem projects ● OpenNebula hosts an ecosystem catalog ● Promote and discuss ecosystem projects in our ecosystem mailing list IRC Channel ● #opennebula on irc.freenode.net Federated Cloud Computing - The OpenNebula Experience! 31/32
- 32. Questions? We Will BeHappy to Answer Any Question ! CloudPlan.org @imllorente Federated Cloud Computing - The OpenNebula Experience! 32/32