StratusLab: A IaaS Cloud Distribution Focusing on Simplicity

832 views

Published on

The StratusLab collaboration provides an Infrastructure as a Service (IaaS) cloud distribution, allowing data centres to install private, community, or public clouds for their users. The distribution focuses on simplicity: allowing quick, straightforward installation on commodity hardware as well as easy access by scientists and engineers. Having run a cloud service based on StratusLab for several years, we will present issues that have arisen and provide examples of scientific use of the infrastructure. This will be followed by a live demonstration of the main features of the StratusLab cloud distribution.

(Presentation given at Summer School on Cloud Computing, Evry, France, August 29, 2013.)

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
832
On SlideShare
0
From Embeds
0
Number of Embeds
25
Actions
Shares
0
Downloads
10
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

StratusLab: A IaaS Cloud Distribution Focusing on Simplicity

  1. 1. StratusLab: Darn Simple Cloud Charles (Cal) Loomis LAL, Univ. Paris-Sud, CNRS/IN2P3 & SixSq Sàrl 27 August 2013
  2. 2. 2 In two pages NIST defines:  Essential characteristics  Deployment models  Service models What is a Cloud? http://csrc.nist.gov/publications/ nistpubs/800-145/SP800-145.pdf
  3. 3. 3 On-demand self-service  No human intervention Broad network access  Fast, reliable remote access Rapid elasticity  Scale based on app. needs Resource pooling  Multi-tenant sharing Measured service  Direct or indirect economic model with measured use Essential Characteristics http://csrc.nist.gov/publications/ nistpubs/800-145/SP800-145.pdf
  4. 4. 4 Private  Single administrative domain, limited number of users Community  Different administrative domains with common interests & proc. Public  People outside of institute‟s administrative domain Hybrid  Federation via combination of other deployment models Deployment Models http://csrc.nist.gov/publications/ nistpubs/800-145/SP800-145.pdf
  5. 5. 5 Software as a Service (SaaS)  Direct (scalable) hosting of end user applications Platform as a Service (PaaS)  Framework and infrastructure for creating web applications Infrastructure as a Service (IaaS)  Access to remote virtual machines with root access Service Models http://csrc.nist.gov/publications/ nistpubs/800-145/SP800-145.pdf
  6. 6. 6 Why are cloud technologies useful? Users  Custom environment: no more porting, revalidation of code  Pre-installed and configured applications  Rapid, dynamic provisioning of resources  Complete control over the requested resources Developers  Simple access: use of REST and RPC over HTTP(S)  Elasticity to respond to peaks in demand for applications Administrators  Flexible management: separate mgt. of machines and services  Separation of responsibilities: Hardware / Services / Platforms / Users Resource Providers  Better utilization of shared resources  Federation (outsourcing) possible
  7. 7. 7 State of the Art Commercial Provider: Amazon Web Services (AWS)  Leading and largest IaaS service provider  Improving and adding new services at a phenomenal rate  Providers differentiate based on price, SLAs, location, etc. Commercial Cloud Distribution: VM-ware  Extremely good and complete  Very expensive, except for ESXi hypervisor (free) Open Source Cloud Distributions: Many!  Essentially none in 2007; now easily a dozen different distributions  StratusLab, …, OpenStack, OpenNebula, CloudStack  Very different levels of maturity, stability, scalability, etc. IaaS cloud providers all use similar semantics, but different APIs, etc.
  8. 8. 8 What is it?  Complete IaaS cloud distribution  Open source (Apache 2 license)  Works well for production private and public IaaS clouds Focus: Darn Simple Cloud  Simple to install on commodity hardware  Simple to use, from any client machine  Scales down as well as up! Infrastructure as a Service (IaaS) +Customized environment +Dynamic (scalable) provisioning +Easy access −Variety of APIs and interfaces −Image creation is tedious −Single machine granularity StratusLab
  9. 9. 9 Where did it start? Informal collaboration to investigate running grid services on Amazon EC2 (2007) StratusLab Project (6/2010 to 5/2012) co-funded by EC with 6 partners from 5 countries Open collaboration to continue the development and support of the StratusLab software Website: http://stratuslab.eu Twitter: @StratusLab Support:support@stratuslab.eu Source: http://github.com/StratusLab Identified need for open source cloud distribution. Production dist. with academic & commercial deployments.
  10. 10. 10 Releases Post-Project Releases  V2.1 (16/10): Streamlined release; improved IO perf. with virtio drivers  V2.1.1 (29/11): Bug fixes; storage upload; better Windows support  V13.02 (31/1): Support for CloudInit contextualization and bug fixes  V13.05 (18/6): Initial steps towards new architecture  V13.09 (30/9): CIMI and new architecture Release Policy  Quarterly timed releases (13.02, 13.05, …)  Intermediate bug fix releases as needed  Roadmap (6-month) available describing the StratusLab evolution Support Policy  Best-effort support for all recent releases, emphasis on latest
  11. 11. 11 StratusLab Services
  12. 12. 12 StratusLab Services  Compute: Virtual machine management (currently uses OpenNebula)  Storage: Volume-based storage service  Network: Simple configuration for public, local, and private VM access  Image mgt.: Complete system for trusted sharing of VM images Tools  Python CLI and APIs (Libcloud) to facilitate use of cloud  CLI to facilitate the installation of services
  13. 13. 13 Service Details
  14. 14. 14 Compute Features  Fast provisioning of VMs, with low latency start-up Contextualization  HEPiX&OpenNebula CDROM contextualization by default  CloudInit (disk based) also supported Implementation  API: XML-RPC interface of OpenNebula  OpenNebula (C++, Ruby) with customized hooks  Hooks primarily for caching, snapshots, and storage access
  15. 15. 15 Storage Features  Volume abstraction for storage service  Provide users with persistent storage for data  Serves also as cache of images for VM instances  (No file-based or object-based storage service) Implementation  API: Proprietary REST interface with CRUD actions  Java-based service using MySQL database for state information  Can use iSCSI or shared file system for physical storage  Can use simple files or LVM volumes for disk content
  16. 16. 16 Network Features  Support 3 specific use cases: public service (public), batch system (local), and BOINC-like worker (private)  Dynamic configuration of network switches not needed  Uses usual services for VM network configuration Implementation  No API: manual, static configuration of network  Rec. configuration: VLAN for cloud services separate VLAN for VMs  All classes of IP addresses are optional, can create other classes  Uses DHCP for VM network configuration  Users responsible for protecting their machines
  17. 17. 17 Marketplace & Image Handling Priorities  Mechanism for sharing and trusting images  Possible to distribute fixed, read-only data sets as well  Split the storage of image metadata and image contents  Availability of VM images of common operating systems Implementation  Marketplace API: Proprietary REST API for create, read, search  Marketplace acts as image registry and handles only metadata  Image contents can be located on any public (web) server  „Private‟ images can also be held in cloud storage  CentOS, Ubuntu, OpenSuSE, Debian, Fedora, ScientificLinux images created and supported by StratusLab
  18. 18. 18 Image Handling Workflow
  19. 19. 19 Tools Command Line Client  Administrator: simplifies StratusLab installation  Users: access StratusLab cloud from anywhere Administration  Quarantine for stopped virtual machines  Monitoring of cloud activity and resources Authentication and Authorization  Supports username/password, certificates, cert. proxies  Specification in local file and/or LDAP
  20. 20. 20 Support Information  Web site documentation  Recorded tutorials Mailing List  support@stratuslab.eu Meetings  Live tutorials (usually 2-3 per year)  Workshops (2+ per year)
  21. 21. 21 Priorities for Evolution Interfaces  Adopt CIMI as the standard interface to services  Provide complete browser interface for all services Simplicity, Scalability, & Robustness  Direct use of libvirt as VM manager  Distributed database (Couchbase) as information „bus‟ Better services for system administrators  Improved overview and monitoring of infrastructure  Fine-grained accounting for all resources  Migration control
  22. 22. 22 New Architecture
  23. 23. 23 Running Clouds in Production
  24. 24. 24 StratusLab Deployments Reference Cloud Services  (~)Open infrastructures for using StratusLab and providing feedback  Operated on a first-come, first-serve, best-effort basis  In production 2+ years, with 250+ registered users  Two sites: LAL (Orsay, France) and GRNET (Athens, Greece) Other deployments…  Academic: France, Ireland, UK, Vietnam, South Africa, …  Commercial: Atos, Helix Nebula, … Building on top…  SlipStream from SixSq: cloud based systems deployment and testing
  25. 25. 25 Cloud Experience at LAL Private cloud for laboratory services  Works well, plan to migrate all services including grid worker nodes and experiment-specific servers  Services switched to VMs without users being aware of change  Very different way of working, need to change administrator habits  Have seen some stability issues related to SL6 kernel/virtualization Public cloud open to university  Very positive reaction to cloud; LAL resources nearly 100% used  Fields: biology, software eng., stats, astrophysics, bioinformatics, …  After initial introduction, users require only low level of support  Other labs offering StratusLab training without our direct involvement Majority of problems from machine room & hardware, not software.
  26. 26. 26 Who’s Using the LAL Cloud? Users:  Wide variety of disciplines  Both academic and commercial Current resources:  416 CPU cores  730 GB of RAM  15 TB of persistent storage University Affiliated Users ABgenomica SME providing bioinformatics applications and services AppStat Analysis of machine learning algorithms and their use for scientific discovery CSNSM Providing training infrastructure for NARVAL data acquisition software IGM Running scientific applications in genomics and microbiology STAPS Investigation for using cloud as teaching infrastructure to provide MATLAB, … LAL SI Infrastructure for testing and validation of software
  27. 27. 27 Virtual Data Computing Platform for P2IO  An extensible, modern infrastructure that can support large scale scientific computing.  Designed to meet the needs of P2IO as part of its global scientific computing infrastructure  Will provide both grid and cloud services to P2IO laboratories  Discussing with university to see this become an official, supported scientific platform for UPS. LABEX : Physique des 2 Infinis et des Origines (P2IO) Physique nucléaire Physique des hautes énergies Astrophysique Orsay : CSNSM, IAS, IMNC, IPNO, LAL, LPT Saclay : CEA/Irfu Palaiseau : Polytechnique/LLR ~2000 personnes dont 130 informaticiens
  28. 28. 28 Global Vision  High availability of services Two machine rooms:  Vallée (Orsay, Bât. 206)  Plateau (EcolePolytechnique) Salle Vallée (Avail. 1/10/2013)  ⅓ of final capacity  100 m2 (28 racks), 400 kW IT  Redundant cooling capacity  Single electrical supply Virtual Data Evolution
  29. 29. 29 Federated Clouds
  30. 30. 30 Transparent Federation  Site operators “outsource” to other providers  Completely transparent to end users  Difficult to achieve in practice because of concerns about data protection, network access and performance Federation Models (Hybrid Cloud & “Sky” Computing)
  31. 31. 31 Brokered Federation  Variety of different cloud infrastructures are visible to users  Users choose to place virtual machines in particular locations  Simple clients can handle federation if differences are small  Orchestrators are needed for larger differences between clouds Both Helix Nebula and EGI take the brokered approach Federation Models (Hybrid Cloud & “Sky” Computing)
  32. 32. 32 SlipStream Cloud orchestrator and deployment engine  Facilitates testing, deployment, and maintenance of complex systems  Transparent access to multiple cloud infrastructures  Allows automated multi-cloud deployment of systems
  33. 33. 33 Conclusions StratusLab Cloud Distribution  Supported, stable, and production-quality IaaS cloud distribution  Used for reference cloud service for 2+ years  Other academic and commercial deployments  Defined, ambitious roadmap for the its continued evolution  Frequent administrator and user tutorials and workshops StratusLab Collaboration  New collaborators welcome: developers and documenters!  Weekly phone conference between developers  Biannual StratusLab workshops
  34. 34. 34 Questions and Discussion website http://stratuslab.eu twitter @StratusLab support support@stratuslab.eu StratusLab source http://github.com/StratusLab SlipStream source http://github.com/slipstream
  35. 35. http://stratuslab.eu/ Copyright © 2013, Members of the StratusLab collaboration. This work is licensed under the Creative Commons Attribution 3.0 Unported License (http://creativecommons.org/licenses/by/3.0/).

×