Advertisement
Cloud Computing Architecture with Open Nebula - HPC Cloud Use Cases - NASA Ames 2012
May. 17, 2012•0 likes•6,954 views
6 likes
Be the first to like this
Show More
views
Total views
0
On Slideshare
0
From embeds
0
Number of embeds
0
Download to read offline
Report
Technology
Business
OpenNebula is a fully open-source cloud management platform, with excellent performance and scalability to manage tens of thousands of virtual machines, and with the most advanced functionality for building virtualized enterprise data centers and private cloud infrastructures. OpenNebula is the result of many years of research and development in efficient and scalable management of virtual machines on large-scale distributed infrastructures. Its innovative features have been developed to address the requirements of business use cases from leading companies in the context of flagship international projects in cloud computing. OpenNebula is being used by many supercomputing and leading research centers to build HPC and science clouds for hosting virtualized computational environments, such as batch farms and computing clusters, or for providing users with new "HPC as a service" resource provisioning models. The talk describes how to design a cloud architecture with OpenNebula and its innovative features to enable the execution of flexible and elastic cluster and high performance computing services on demand while reducing the associated cost of building the datacenter infrastructure.
Ignacio M. LlorenteFollow
Director and Chief Executive Advisor at OpenNebula/C12GAdvertisement
Cloud Computing Architecture with Open Nebula - HPC Cloud Use Cases - NASA Ames 2012
- NASA Ames NASA Advanced Supercomputing (NAS) Division California, May 24th, 2012 Cloud Computing Architecture with OpenNebula HPC Cloud Use Cases Ignacio M. Llorente Project Director © OpenNebula Project. Creative Commons Attribution-NonCommercial-ShareAlike License
- Contents Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! This presentation is about: • Cloud case studies in HPC • Private cloud computing using OpenNebula • The anatomy of a Private Cloud • OpenNebula innovative features for HPC Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 2/26
- What is OpenNebula? Open Cloud Solution for Building and Managing Virtualized Data Centers! Public Private Hybrid Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 3/26
- The HPC Cloud Use Case The Pre-cloud Era! Access Grid Middleware LRMS (LSF, PBS, SGE…) Provision Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 4/26
- The HPC Cloud Use Case OpenNebula as an Infrastructure Tool! Access Service/Provisioning Decoupling! • Common interfaces • Grid integration Grid Middleware LRMS (LSF, PBS, SGE…) • Custom environments Service • Dynamic elasticity Virtual Worker Nodes • Consolidation of WNs • Simplified management Provision • Physical – Virtual WNs • Dynamic capacity partitioning • Faster upgrades Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 5/26
- The HPC Cloud Use Case OpenNebula as an Provisioning Tool! • Simple Provisioning Interface Access • Raw/Appliance VMs IaaS Interface Pilot Jobs, SSH… Service • Dynamic scalable computing • Custom access to capacity • Not only batch workloads • Not only scientific workloads Provision • Improve utilization • Reduced service management • Cost efficiency Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 6/26
- The HPC Cloud Use Case Examples: CERN’s lxcloud! http://blog.opennebula.org/?p=620 Goal • Virtualized HTC Batch Nodes: limited life-time, dynamic capacity • EC2 Query: CernVM Deployment Notes • Custom network integration MAC/IP pinning • Fast image distribution & boot: BitTorrent + LVM snapshots • OpenNebula tests up to 20,000 VMs (great feedback!) Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 7/26
- The HPC Cloud Use Case Examples: FermiCloud! http://www-fermicloud.fnal.gov/ Goal • Scientific stakeholders get access to on-demand VMs • Developers & integrators of new Grid applications • MPI and legacy applications Deployment Notes • VMs access Fermilab Networking and Storage Services • OpenNebula + X509 support (contributed back!) • Other areas: HA, Batch queues look-ahead, cluster on-demand… Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 8/26
- The HPC Cloud Use Case Examples: SARA! https://www.cloud.sara.nl/ Goal • HPC clusters on-demand: Self service, elastic and fully configurable HPC systems • Offer production infrastructure for several engineering and scientific communities: Bioinformatics, ecology… Deployment Notes • Low latency network for HPC • Extensions in accounting, GUI, firewalling… (contributed back!) Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 9/26
- The HPC Cloud Use Case One of Our Main User Communities! Supercomputing Centers Research Centers Distributed Computing Infrastructures Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 10/26
- The Anatomy of the Cloud Different Perspectives of the Cloud! Cloud User Cloud Provider Cloud Integrator Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 11/26
- Using the Cloud How Can I Create and Manage my Virtual Infrastructure?! … standards (de facto and de jure) Cloud APIs to leverage existing ecosystems and ensure portability across providers, and self-service portal …. " Use AWS existing tools like ElasticFox • Manage images, " Execute complete networks and computing cluster instances " Manage persistent scientific data " Catalog with pre-defined environments " Sharing between users or groups Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 12/26
- Building the Cloud What are the Main Components to Build a Cloud Infrastructure?! Instance Networks Internet • Guests • Public and private networks " Fair share of resources " Place VM close to data Front-end Hosts • Authentication • Multiple hypervisors • Authorization • Up to 500 hosts • ACLs, roles, • Automatic failover and HA groups… • Automatic resource allocation • Accounting • Logging • Resource quotas Service Networks Datastores • Monitoring, control… • VM image storage • Live migration… • Multiple datastores • Storage access… • Heterogeneous configurations • Shared or non-shared Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 13/26
- Building the Cloud Broad Commodity and Enterprise Platform Support! Instance Network • VLAN per user (layer2) Internet • Open vSwitch, 802.1q • Ebtables Front-end Hosts • X509, LDAP, ssh keys • VMware • ACLs, roles, groups… • Xen • KVM Datastores Service Network • NAS: NFS, Gluster, GlusterFS… • Ganglia/Nagios • SAN: Fibre Channel, iSCSI, LVM… • Additional monitor agents • DFS: Bit Torrent, ssh… Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 14/26
- Innovative Features Clustering the Physical Resources! Clusters • Pools of hosts that share datastores and networks • Group different configurations for performance and security " Different Hw for different workloads profiles (HPC vs HTC) " Isolation and security levels Multiple Datastores per Cluster • Balance I/O operations between storage servers • Define different SLA policies (e.g. backup) and performance features for different VM types or users Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 15/26
- Innovative Features Centralized Management of Multiple OpenNebula Instances (Zones)! " Full Isolation Clouds Federation of of performance-sensitive • Multi-tier architecture • Scalability applications oZones Server • Isolation " Isolation and security • Portal • Multiple-site support levels Cloud • Cloud API (EC2, OCCI) " Multi-tier Consumer • Global AuthN " Scalability " Multiple-site Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 16/26
- Innovative Features On-demand Provision of Virtual Data Centers! " Virtual Private Cloud Computing with Provide VOs or Departments ● their own isolated cloud Typical scenario in large organizations and cloud providers compartment " ● Associated provision of fully- On-demand to clusters with specific Hw and isolated VDC with full configurable and Sw profiles, or control and capacity to administer its security levels users and resources Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 17/26
- Innovative Features Hybrid Cloud Computing! Hybrid ● Extension of the local private infrastructure with resources from remote clouds ● Cloudbursting to meet peak or fluctuating demands Cloud User " HTC workloads " Managed by the admin Cloud Provider Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 18/26
- Operating the Cloud How Can I Operate my Cloud Infrastructure?! … programming APIs (create new tools and integrate), web interfaces (simplify operation), and command lined interface (create scripts)… " Automatic/Elastic Management of Computing Service CLI API ● XML-RPC ● OCA (Ruby, Java, Python) Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 19/26
- Integrating the Cloud The Ecosystem! … truly open (fully open-source, Apache license) and adaptable (modular and extensible)… because no two data centers are the same Standards Adapters CDMI OVF CIMI Virtualization Drivers Configuration Storage Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 20/26
- About the OpenNebula Project A Project Aimed at Building the Industry Standard Open Cloud Management Tool! Third party scalability • Develop & innovate tests: 16,000 VMs • Support the community • Collaborate Commercial Support TP v1.0 v1.2 v1.4 v2.0 v2.2 V3.0 V3.2 V3.4 2005 2008 2009 2010 2011 2012 dsa group doing research… 4,000 downloads/ month European Funding Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 21/26
- About the OpenNebula Project A Quickly Growing Community More than Doubling Each Year! 25,200 Downloads ● 900 downloads in the last week 12,828 ● Linux distro and code repo 4,861 1,865 140% Annual Growth 2008 2009 2010 2011 Site Visits 579,571 ● 15,300 visits and 194,000 page 254,186 views in the last week 111,541 35,842 150% Annual Growth 2008 2009 2010 2011 Mails 4,341 ● 800 registered users at present 2,293 1,141 227 170% Annual Growth Date: November 7th, 2011 2008 2009 2010 2011 Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 22/26
- About the OpenNebula Project Open Cloud Enabler in the ICT Industry! Enabling Hosting Companies and Telcos to Offer Cloud Services Enabling Technology Companies to Offer Cloud Products Enabling Service Companies to Offer Cloud Consulting and Integration Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 23/26
- About the OpenNebula Project Open Cloud Enabler for Building and Operating Virtualized Data Centers! Industry Supercomputing Centers Research Centers Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 24/26
- About the OpenNebula Project Open Cloud Enabler for Building and Research and Innovation! Distributed Computing Infrastructures Research Projects 16,000 VMs! Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 25/26
- Questions? We Will Be Happy to Answer Any Question ! Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 26/26
Advertisement