Federated Cloud Computing - The OpenNebula Experience v1.0s

3,029 views

Published on

The talk mostly focuses on private cloud computing to support Science and High Performance Computing environments, the different architectures to federate cloud infrastructures, the existing challenges for cloud interoperability, and the OpenNebula's vision for the future of existing Grid infrastructures.

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,029
On SlideShare
0
From Embeds
0
Number of Embeds
1,451
Actions
Shares
0
Downloads
100
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Federated Cloud Computing - The OpenNebula Experience v1.0s

  1. 1. Workshop GARR - Calcolo e Storage Distribuito Rome, Italy, November 30th, 2012 Federated Cloud Computing The OpenNebula Experience Ignacio M. Llorente Project Director© OpenNebula Project. Creative Commons Attribution-NonCommercial-ShareAlike License
  2. 2. ContentsFederated Cloud Computing - The OpenNebula Experience! This presentation is about: •  Infrastructure as a Service Cloud Computing •  Private Clouds in Science and HPC •  Federated Cloud Computing •  Cloud federation in Grid infrastructuresFederated Cloud Computing - The OpenNebula Experience! 2/32
  3. 3. What is Cloud Computing?Provision of IT Capabilities as a Service! What Who On-demand End-user Software as a Service access to any (does not care about hw or sw) application Platform for Developer Platform as a Service building and (no managing of the underlying hw delivering web & swlayers) applications Infrastructure as a Raw computer System Administrator ᄎ Service infrastructure (complete management of the computer infrastructure) Physical InfrastructureFederated Cloud Computing - The OpenNebula Experience! 3/32
  4. 4. What is Cloud Computing?IaaS Clouds for Provision of Virtualized Resources as a Service! •  Raw infrastructure •  Management Console resources •  Simple REST API’s Manage Instances •  Pay-as-you-go & elastic capacityFederated Cloud Computing - The OpenNebula Experience! 4/32
  5. 5. What is Cloud Computing?Different Models of Deployment! Model Infrastructure Cloud Cases Owned by a single •  Optimize and simplify internal operation Private organization and made •  SaaS/PaaS support available only to the •  IT consolidation within large organizations organization (Goverment Clouds, University Clouds…) Owned by a single •  Commercial cloud providers, mostly organization and made hosting providers, with limited control/ Public available to other security organizations over the •  Science public clouds by ICT service Internet centers to enable scientific projects or experiment with cloud computing Owned by a single •  Telecom cloud providers with premium Private Virtual organization and made solutions with additional control/security available to other organization over a dedicated private network Composition of two or •  Cloudbursting to address peak demands •  Cloud Federation to share infrastructure Hybrid more clouds with partners •  Cloud Aggregation to provide a larger resource infrastructureFederated Cloud Computing - The OpenNebula Experience! 5/32
  6. 6. What is OpenNebula?Open Cloud Management Solution for Building and Managing Virtualized Data Centers! Public Private HybridFederated Cloud Computing - The OpenNebula Experience! 6/32
  7. 7. What is OpenNebula?Most Advanced and Flexible, Enterprise-grade IaaS Cloud Manager!Adaptable● Customizable and ExtensibleProven● Many Massive Scale Production DeploymentsPowerful● Most Advanced Enterprise-class FunctionalityNo Lock-in● Platform Independent and InteroperableInteroperable● Most popular cloud APIs and standard basedOpenness● Fully open-source, Apache licenseFederated Cloud Computing - The OpenNebula Experience! 7/32
  8. 8. What is OpenNebula?User-driven Open Platform for Innovation! Adopt as innovation Standards Contribute to platform or Projects interoperability tool standards Contribute innovative Adopt functionality standards Distribution channel Linux Distributions Requirements Feedback Contributions Adopt open-source Industry and ResearchFederated Cloud Computing - The OpenNebula Experience! 8/32
  9. 9. Private Cloud in Science and HPCThe Pre-cloud Era! Access Grid Middleware LRMS (LSF, PBS, SGE…) ProvisionFederated Cloud Computing - The OpenNebula Experience! 9/32
  10. 10. Private Cloud in Science and HPCOpenNebula as an Infrastructure Tool! Access Service/Provisioning Decoupling! •  Common interfaces •  Grid integration Grid Middleware LRMS (LSF, PBS, SGE…) •  Custom environments Service •  Dynamic elasticity Virtual Worker Nodes •  Consolidation of WNs •  Simplified management Provision •  Physical – Virtual WNs •  Dynamic capacity partitioning •  Faster upgradesFederated Cloud Computing - The OpenNebula Experience! 10/32
  11. 11. Private Cloud in Science and HPCOpenNebula as an Provisioning Tool! •  Simple Provisioning Interface Access •  Raw/Appliance VMs IaaS Interface Pilot Jobs, SSH… Service •  Dynamic scalable computing •  Custom access to capacity •  Not only batch workloads •  Not only scientific workloads Provision •  Improve utilization •  Reduced service management •  Cost efficiencyFederated Cloud Computing - The OpenNebula Experience! 11/32
  12. 12. Private Cloud in Science and HPCExamples: CERN’s lxcloud! http://blog.opennebula.org/?p=620 Goal •  Virtualized HTC Batch Nodes: limited life-time, dynamic capacity •  EC2 Query: CernVM Deployment Notes •  Custom network integration MAC/IP pinning •  Fast image distribution & boot: BitTorrent + LVM snapshots •  OpenNebula tests up to 20,000 VMs (great feedback!)Federated Cloud Computing - The OpenNebula Experience! 12/32
  13. 13. Private Cloud in Science and HPCExamples: FermiCloud! http://www-fermicloud.fnal.gov/ Goal •  Scientific stakeholders get access to on-demand VMs •  Developers & integrators of new Grid applications •  MPI and legacy applications Deployment Notes •  VMs access Fermilab Networking and Storage Services •  OpenNebula + X509 support (contributed back!) •  Other areas: HA, Batch queues look-ahead, cluster on-demand…Federated Cloud Computing - The OpenNebula Experience! 13/32
  14. 14. Private Cloud in Science and HPCExamples: SARA! https://www.cloud.sara.nl/ Goal •  HPC clusters on-demand: Self service, elastic and fully configurable HPC systems •  Offer production infrastructure for several engineering and scientific communities: Bioinformatics, ecology… Deployment Notes •  Low latency network for HPC •  Extensions in accounting, GUI, firewalling… (contributed back!)Federated Cloud Computing - The OpenNebula Experience! 14/32
  15. 15. Private Cloud in Science and HPCOne of Our Main User Communities! Supercomputing Centers Research Centers Distributed Computing InfrastructuresFederated Cloud Computing - The OpenNebula Experience! 15/32
  16. 16. Federated Cloud ComputingNext Step in the Evolution of an Utility! Utility Utility Generation Distribution Utility GridFederated Cloud Computing - The OpenNebula Experience! 16/32
  17. 17. Federated Cloud ComputingBenefits of Federation!Scalability● Cloudbursting to address peak demandsCollaboration● Sharing of infrastructure between partnersMulti-site Deployments● Infrastructure aggregation across distributed data centersReliability● Fault tolerance architectures across sitesPerformance● Deployment of services closer to end usersCost● Dynamic placement to reduce the overall infrastructure costEnergy Consumption● Minimize energy consumptionFederated Cloud Computing - The OpenNebula Experience! 17/32
  18. 18. Federated Cloud ComputingThe Cluster Computing Use Case! Scalability Performance Cost Fault toleranceFederated Cloud Computing - The OpenNebula Experience! 18/32
  19. 19. Federated Cloud ComputingDifferent Levels of Control, Monitoring, Cross-site Functionality and Security! Private Cloud Partner Company Cloud Cloud Cloud  OS   Public (Broker)   Cloud Public   Public   Public   Cloud  1   Cloud  2   Cloud  3  Federated Cloud Computing - The OpenNebula Experience! 19/32
  20. 20. Federated Cloud ComputingLoosely Coupled Federation - Cloudbursting!Federation with a Cloud without Interoperation Support Corporate cloud Cloud  OS   Public   Public   Cloud  1   Cloud  2   Local data center Aim •  Meet peak demands •  Basic operations over VMs (start, shutdown, restart…) Control •  Different instance types Monitoring & •  Basic virtual resource monitoring (resource consumption…) Accounting Cross-site •  None Security •  Single account representing the organizationFederated Cloud Computing - The OpenNebula Experience! 20/32
  21. 21. Federated Cloud ComputingDynamic Combination of Local with Remote Cloud Resources! Service Level Agreement •  SLO 1: Average_CPU_Utilization > 75% •  Action: Execute remote WN Grid ServicesFederated Cloud Computing - The OpenNebula Experience! 21/32
  22. 22. Federated Cloud ComputingPartially Coupled Federation - Aggregated Cloud Architecture!!Federation with a Cloud with Partial Interoperation Support Corporate Corporate cloud 1 cloud 2 Cloud  OS   Cloud  OS   Local data center Local data center Aim •  Sharing of resources to meet peak demands •  Advanced operations over VMs (live migration…) Control •  VM location and affinity constraints Monitoring & •  Advanced virtual resource monitoring (energy consumption, Accounting VM placement…) •  Virtual networks Cross-site •  Virtual storage Security •  Framework agreement Federated Cloud Computing - The OpenNebula Experience! 22/32
  23. 23. Federated Cloud ComputingCloud Broker Architecture! Building Service Testbeds on FIRE Agreement 257386 (2010-2013) Design, build and operate a multi-site cloud-based facility to New Infrastructure Paradigms support research across applications, services and systems and Experimental Facilities targeting services research community on Future Internet www.BonFIRE-Project.eu Source: BonFIRE ProjectFederated Cloud Computing - The OpenNebula Experience! 23/32
  24. 24. Federated Cloud ComputingTightly Coupled Federation - Multi-tier Architecture!Federation with a Cloud with Advanced Interoperation Support Corporate cloud Cloud  OS   Cloud Cloud site 1 site 2 Cloud  OS   Cloud  OS   Local data center Local data center Aim •  Very large scale or geographically distributed data centers •  Placement on specific physical resources Control •  Same instance types Monitoring & •  Physical resource consumption Accounting •  Live migration Cross-site •  High availability Security •  User space sharingFederated Cloud Computing - The OpenNebula Experience! 24/32
  25. 25. Federated Cloud ComputingMulti-tier Cloud Architecture! CloudFront •  Portal •  Cloud API (EC2, OCCI) User •  Global AuthN OpenNebula Zone OpenNebula Zone •  Multi-tenancy •  Multi-tier arch. •  Runs Instances •  Scalable •  Stores Images •  Isolation •  Networking •  Multiple-site •  Authorization supportFederated Cloud Computing - The OpenNebula Experience! 25/32
  26. 26. Federated Cloud ComputingChallenges for Interoperability!Transparent Combination of Local Resources with Cloud Resources 1. Management Interfaces 2. Management Interfaces for Virtual Workloads for Data Elements VM VM VM VM VM 4. Contextualization 3. VM Image Description 5. Cross-site Networking and Instance Type and Latency 6. Common framework for security and QoSFederated Cloud Computing - The OpenNebula Experience! 26/32
  27. 27. Federated Cloud ComputingLeveraging Existing Standards and Implementing Interoperation! Standardization ●  Implement standards ●  Integrate with standards Which Standard? ●  Different de jure standards ●  Several de facto standards Interoperation ●  Implement adaptors ●  Use transformersFederated Cloud Computing - The OpenNebula Experience! 27/32
  28. 28. Cloud Federation in Grid InfrastructuresGrid and Cloud as Complementary Computing Models! Usage Grids Usage Clouds   Job Processing   Raw infrastructure   Big Batch System   Elasticity & Pay-per-use   File Sharing Services   Simple Web Interface Achievements Achievements   Federation of Resources   Agile Infrastructures   VO Concept   IT is another Utility But…   User experience But…   Interoperability   Complexity   Federation Resource Sharing Resource Management Scientific Applications Customize Environments Uniform Security Flexibility & SimplicityFederated Cloud Computing - The OpenNebula Experience! 28/32
  29. 29. Cloud Federation in Grid InfrastructuresThe OpenNebula Vision for Grid Sites! •  Batch Job Processing •  Industry Applications •  Custom Execution Environments •  Other WMS (pilots) Access •  Grid Service Integration •  Complete Services (cluster) Grid Middleware IaaS Interface LRMS (LSF, PBS…) Service Virtual CE, WN… Other (web, mail...) Raw machines Provision Grid SiteFederated Cloud Computing - The OpenNebula Experience! External Providers 29/32
  30. 30. Cloud Federation in Grid InfrastructuresThe OpenNebula Vision for Grid Infrastructures ! Grid Services MarketPlace•  Federation facilities•  Security •  Sharing existing VM images•  Grid specific services •  Registry of metadata •  Image are kept elsewhere •  Supports trust Appliance Repo •  Storage VM images •  Distributed •  Multi-protocol Grid Services Cloud API Grid Services Cloud API Cloud/Grid Site Cloud/Grid SiteFederated Cloud Computing - The OpenNebula Experience! 30/32
  31. 31. Join our Growing Community!OpenNebula is Driven by User Needs ! How to contribute ● Join our mailing list ● Follow the development at dev.opennebula.org ● Contributions of by users: RIM, Akamai, Logica, FermiLab, SARA, Terradue, China Mobile… (approx. 100 listed at http://www.opennebula.org/ about:contributors) ● 500 validated users at dev.opennebula Ecosystem projects ● OpenNebula hosts an ecosystem catalog ● Promote and discuss ecosystem projects in our ecosystem mailing list IRC Channel ● #opennebula on irc.freenode.netFederated Cloud Computing - The OpenNebula Experience! 31/32
  32. 32. Questions?We Will Be Happy to Answer Any Question ! CloudPlan.org @imllorenteFederated Cloud Computing - The OpenNebula Experience! 32/32

×