Cloud Computing Architecture with Open Nebula  - HPC Cloud Use Cases - NASA Ames 2012
Upcoming SlideShare
Loading in...5
×
 

Cloud Computing Architecture with Open Nebula - HPC Cloud Use Cases - NASA Ames 2012

on

  • 6,790 views

OpenNebula is a fully open-source cloud management platform, with excellent performance and scalability to manage tens of thousands of virtual machines, and with the most advanced functionality for ...

OpenNebula is a fully open-source cloud management platform, with excellent performance and scalability to manage tens of thousands of virtual machines, and with the most advanced functionality for building virtualized enterprise data centers and private cloud infrastructures. OpenNebula is the result of many years of research and development in efficient and scalable management of virtual machines on large-scale distributed infrastructures. Its innovative features have been developed to address the requirements of business use cases from leading companies in the context of flagship international projects in cloud computing. OpenNebula is being used by many supercomputing and leading research centers to build HPC and science clouds for hosting virtualized computational environments, such as batch farms and computing clusters, or for providing users with new "HPC as a service" resource provisioning models. The talk describes how to design a cloud architecture with OpenNebula and its innovative features to enable the execution of flexible and elastic cluster and high performance computing services on demand while reducing the associated cost of building the datacenter infrastructure.

Statistics

Views

Total Views
6,790
Views on SlideShare
3,911
Embed Views
2,879

Actions

Likes
1
Downloads
133
Comments
0

26 Embeds 2,879

http://blog.opennebula.org 1943
http://blog.cloudplan.org 742
http://4973727423887134232_38b62c36e6cbbc4052db5743c6258cb60002f8df.blogspot.com 39
http://opennebula.org 29
http://ignaciollorente.ulitzer.com 25
http://eclipse.sys-con.com 18
http://virtualization.sys-con.com 16
http://www.linkedin.com 12
http://translate.googleusercontent.com 8
http://sap.sys-con.com 7
http://opennebula.ulitzer.com 7
http://cloudcomputing.sys-con.com 5
http://ignaciollorente.sys-con.com 3
http://infrastructure20.ulitzer.com 3
http://cloudonomics.ulitzer.com 3
http://www.sys-con.com 3
http://opensource.ulitzer.com 3
http://python.sys-con.com 3
http://opensource.sys-con.com 2
http://javacloud.ulitzer.com 2
http://ciocto.ulitzer.com 1
http://seoautomated.com 1
http://www.tuicool.com 1
http://www.feedspot.com 1
http://127.0.0.1 1
http://www.google.com 1
More...

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cloud Computing Architecture with Open Nebula  - HPC Cloud Use Cases - NASA Ames 2012 Cloud Computing Architecture with Open Nebula - HPC Cloud Use Cases - NASA Ames 2012 Presentation Transcript

  • NASA Ames NASA Advanced Supercomputing (NAS) Division California, May 24th, 2012 Cloud Computing Architecture with OpenNebula HPC Cloud Use Cases Ignacio M. Llorente Project Director© OpenNebula Project. Creative Commons Attribution-NonCommercial-ShareAlike License
  • ContentsCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! This presentation is about: •  Cloud case studies in HPC •  Private cloud computing using OpenNebula •  The anatomy of a Private Cloud •  OpenNebula innovative features for HPCCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 2/26
  • What is OpenNebula?Open Cloud Solution for Building and Managing Virtualized Data Centers! Public Private HybridCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 3/26
  • The HPC Cloud Use CaseThe Pre-cloud Era! Access Grid Middleware LRMS (LSF, PBS, SGE…) ProvisionCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 4/26
  • The HPC Cloud Use CaseOpenNebula as an Infrastructure Tool! Access Service/Provisioning Decoupling! •  Common interfaces •  Grid integration Grid Middleware LRMS (LSF, PBS, SGE…) •  Custom environments Service •  Dynamic elasticity Virtual Worker Nodes •  Consolidation of WNs •  Simplified management Provision •  Physical – Virtual WNs •  Dynamic capacity partitioning •  Faster upgradesCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 5/26
  • The HPC Cloud Use CaseOpenNebula as an Provisioning Tool! •  Simple Provisioning Interface Access •  Raw/Appliance VMs IaaS Interface Pilot Jobs, SSH… Service •  Dynamic scalable computing •  Custom access to capacity •  Not only batch workloads •  Not only scientific workloads Provision •  Improve utilization •  Reduced service management •  Cost efficiencyCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 6/26
  • The HPC Cloud Use CaseExamples: CERN’s lxcloud! http://blog.opennebula.org/?p=620 Goal •  Virtualized HTC Batch Nodes: limited life-time, dynamic capacity •  EC2 Query: CernVM Deployment Notes •  Custom network integration MAC/IP pinning •  Fast image distribution & boot: BitTorrent + LVM snapshots •  OpenNebula tests up to 20,000 VMs (great feedback!)Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 7/26
  • The HPC Cloud Use CaseExamples: FermiCloud! http://www-fermicloud.fnal.gov/ Goal •  Scientific stakeholders get access to on-demand VMs •  Developers & integrators of new Grid applications •  MPI and legacy applications Deployment Notes •  VMs access Fermilab Networking and Storage Services •  OpenNebula + X509 support (contributed back!) •  Other areas: HA, Batch queues look-ahead, cluster on-demand…Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 8/26
  • The HPC Cloud Use CaseExamples: SARA! https://www.cloud.sara.nl/ Goal •  HPC clusters on-demand: Self service, elastic and fully configurable HPC systems •  Offer production infrastructure for several engineering and scientific communities: Bioinformatics, ecology… Deployment Notes •  Low latency network for HPC •  Extensions in accounting, GUI, firewalling… (contributed back!)Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 9/26
  • The HPC Cloud Use CaseOne of Our Main User Communities! Supercomputing Centers Research Centers Distributed Computing InfrastructuresCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 10/26
  • The Anatomy of the CloudDifferent Perspectives of the Cloud! Cloud UserCloud Provider Cloud IntegratorCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 11/26
  • Using the CloudHow Can I Create and Manage my Virtual Infrastructure?!… standards (de facto and de jure) Cloud APIs to leverage existing ecosystems and ensure portability across providers, and self-service portal …. " Use AWS existing tools like ElasticFox •  Manage images, " Execute complete networks and computing cluster instances " Manage persistent scientific data " Catalog with pre-defined environments " Sharing between users or groupsCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 12/26
  • Building the CloudWhat are the Main Components to Build a Cloud Infrastructure?! Instance Networks Internet •  Guests •  Public and private networks " Fair share of resources " Place VM close to data Front-end Hosts •  Authentication •  Multiple hypervisors •  Authorization •  Up to 500 hosts •  ACLs, roles, •  Automatic failover and HA groups… •  Automatic resource allocation •  Accounting •  Logging •  Resource quotas Service Networks Datastores •  Monitoring, control… •  VM image storage •  Live migration… •  Multiple datastores •  Storage access… •  Heterogeneous configurations •  Shared or non-sharedCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 13/26
  • Building the CloudBroad Commodity and Enterprise Platform Support! Instance Network •  VLAN per user (layer2) Internet •  Open vSwitch, 802.1q •  Ebtables Front-end Hosts •  X509, LDAP, ssh keys •  VMware •  ACLs, roles, groups… •  Xen •  KVM Datastores Service Network •  NAS: NFS, Gluster, GlusterFS… •  Ganglia/Nagios •  SAN: Fibre Channel, iSCSI, LVM… •  Additional monitor agents •  DFS: Bit Torrent, ssh…Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 14/26
  • Innovative FeaturesClustering the Physical Resources! Clusters •  Pools of hosts that share datastores and networks •  Group different configurations for performance and security " Different Hw for different workloads profiles (HPC vs HTC) " Isolation and security levels Multiple Datastores per Cluster •  Balance I/O operations between storage servers •  Define different SLA policies (e.g. backup) and performance features for different VM types or usersCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 15/26
  • Innovative FeaturesCentralized Management of Multiple OpenNebula Instances (Zones)! " Full Isolation Clouds Federation of of performance-sensitive •  Multi-tier architecture •  Scalability applications oZones Server •  Isolation " Isolation and security •  Portal •  Multiple-site support levels Cloud •  Cloud API (EC2, OCCI) " Multi-tier Consumer •  Global AuthN " Scalability " Multiple-siteCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 16/26
  • Innovative FeaturesOn-demand Provision of Virtual Data Centers!" Virtual Private Cloud Computing with Provide VOs or Departments ● their own isolated cloud Typical scenario in large organizations and cloud providers compartment" ● Associated provision of fully- On-demand to clusters with specific Hw and isolated VDC with full configurable and Sw profiles, or control and capacity to administer its security levels users and resourcesCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 17/26
  • Innovative FeaturesHybrid Cloud Computing! Hybrid ● Extension of the local private infrastructure with resources from remote clouds ● Cloudbursting to meet peak or fluctuating demands Cloud User " HTC workloads " Managed by the admin Cloud ProviderCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 18/26
  • Operating the CloudHow Can I Operate my Cloud Infrastructure?!… programming APIs (create new tools and integrate), web interfaces (simplify operation), and command lined interface (create scripts)… " Automatic/Elastic Management of Computing Service CLI API ●  XML-RPC ●  OCA (Ruby, Java, Python)Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 19/26
  • Integrating the CloudThe Ecosystem! … truly open (fully open-source, Apache license) and adaptable (modular and extensible)… because no two data centers are the same Standards Adapters CDMI OVF CIMI Virtualization Drivers Configuration StorageCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 20/26
  • About the OpenNebula ProjectA Project Aimed at Building the Industry Standard Open Cloud Management Tool! Third party scalability • Develop & innovate tests: 16,000 VMs • Support the community • Collaborate Commercial Support TP v1.0 v1.2 v1.4 v2.0 v2.2 V3.0 V3.2 V3.42005 2008 2009 2010 2011 2012dsa group doing research… 4,000 downloads/ month European FundingCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 21/26
  • About the OpenNebula ProjectA Quickly Growing Community More than Doubling Each Year! 25,200 Downloads ●  900 downloads in the last week 12,828 ●  Linux distro and code repo 4,8611,865 140% Annual Growth 2008 2009 2010 2011 Site Visits 579,571 ●  15,300 visits and 194,000 page 254,186 views in the last week 111,54135,842 150% Annual Growth 2008 2009 2010 2011Mails 4,341 ●  800 registered users at present 2,293 1,141 227 170% Annual Growth Date: November 7th, 2011 2008 2009 2010 2011Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 22/26
  • About the OpenNebula ProjectOpen Cloud Enabler in the ICT Industry! Enabling Hosting Companies and Telcos to Offer Cloud Services Enabling Technology Companies to Offer Cloud Products Enabling Service Companies to Offer Cloud Consulting and IntegrationCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 23/26
  • About the OpenNebula ProjectOpen Cloud Enabler for Building and Operating Virtualized Data Centers! Industry Supercomputing Centers Research CentersCloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 24/26
  • About the OpenNebula ProjectOpen Cloud Enabler for Building and Research and Innovation! Distributed Computing Infrastructures Research Projects 16,000 VMs!Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 25/26
  • Questions?We Will Be Happy to Answer Any Question !Cloud Computing Architecture with OpenNebula - HPC Cloud Use Cases! 26/26