SlideShare a Scribd company logo
1 of 43
Download to read offline
Scientific Cloud Computing: Present & Future
Charles (Cal) Loomis (CNRS/LAL & SixSq Sàrl)
INFN CNAF, Bologna, Italy (22 May 2013)
2
Cloud Marketing
“Cloud” is currently very trendy, used everywhere
 Many definitions that are often incompatible
 Used (often) to market pre-existing (non-cloud) software
CommodityComputing(Sun)
UtilityComputing(IBM,HP,…)
AmazonEC2
AmazonEBS
Mature Virtualization
Simple APIs
Excess Capacity
3
In two pages NIST defines:
 Essential characteristics
 Deployment models
 Service models
What is a Cloud?
http://csrc.nist.gov/publications/
nistpubs/800-145/SP800-145.pdf
4
On-demand self-service
 No human intervention
Broad network access
 Fast, reliable remote access
Rapid elasticity
 Scale based on app. needs
Resource pooling
 Multi-tenant sharing
Measured service
 Direct or indirect economic
model with measured use
Essential Characteristics
http://csrc.nist.gov/publications/
nistpubs/800-145/SP800-145.pdf
5
Private
 Single administrative domain,
limited number of users
Community
 Different administrative domains
with common interests & proc.
Public
 People outside of institute’s
administrative domain
Hybrid
 Federation via combination of
other deployment models
Deployment Models
http://csrc.nist.gov/publications/
nistpubs/800-145/SP800-145.pdf
6
Software as a Service (SaaS)
 Direct (scalable) hosting of end
user applications
Platform as a Service (PaaS)
 Framework and infrastructure
for creating web applications
Infrastructure as a Service (IaaS)
 Access to remote virtual
machines with root access
Service Models
http://csrc.nist.gov/publications/
nistpubs/800-145/SP800-145.pdf
7
Advantages
 No software installation
 Universally accessible
Disadvantages
 Questions about data access,
ownership, reliability, etc.
 Integration of services & novel
uses of data are (often) difficult
Trends
 Social scientific computing
 Service APIs to allow
integration  PaaS
Software as a Service (SaaS)
8
Advantages
 Programmers take advantage of
integrated load balancing,
automatic failover, etc.
Disadvantages
 Restricted number of languages
 Applications strongly locked to a
particular provider
Trends
 Dearth of “pure” PaaS offers
 Encroachment from both SaaS
and IaaS sides
Platform as a Service (PaaS)
9
Advantages
 Customized environment with
“root” access
 Easy access to scalable
resources
Disadvantages
 Variety of APIs and interfaces
 VM image creation is difficult
and time-consuming
Trends
 Lots of specialized cloud
providers appearing
 Orchestration pushing into
PaaS space
Infrastructure as a Service (IaaS)
10
Focus: Infrastructure as a Service
11
State of the Art
Commercial Provider: Amazon Web Services (AWS)
 Leading and largest IaaS service provider
 Improving and adding new services at a phenomenal rate
 Almost all IaaS providers use AWS-like service semantics, but
differentiate based on price, SLAs, location, etc.
Commercial Cloud Distribution: VM-ware
 VM-ware: extremely good and complete, but very expensive
 Provide ESXi virtual machine host for free
Open Source Cloud Distributions
 Essentially none in 2004; now easily a dozen different distributions
 StratusLab, WNoDeS, …, OpenStack, OpenNebula, CloudStack
 Very different levels of maturity, stability, scalability, etc.
12
Why are cloud technologies useful?
For (scientific) users
 Custom environment: no rewriting or porting applications to fit into a
resource provider’s environment
 Simple access: most providers use a REST or RPC API allowing
simple access from all programming languages
 Reasonable cost: only pay for what resources are used, especially
attractive for individuals/groups that do not have large, existing
hardware investment
13
Why are cloud technologies useful?
Separation of responsibilities
 Hardware / Services / Platforms / Users
 People at each layer can focus on their responsibilities with minimal
interactions with people in the other layers.
Resource Providers
 Better utilization of shared resource because wider range of
applications (and disciplines) can use the cloud
 With hybrid cloud infrastructures, providers can outsource excess
demand to other providers
14
Trend for Scientific Computing
Will we all just be users of the Amazon cloud?
Pendulum swinging towards large data centers with “fat” machines
 These can offer elastic cloud services at a reasonable price
 With scientific clouds there is low barrier to entry and users can
maintain administrative control of services and data
 Providing shared resource between scientific disciplines much easier
because of virtualization
Migration will be gradual…
15
Overcoming inertia…
Users
  How to use virtual machines to get my work done?
  How to structure, store, access, and protect data?
  Realize shared infrastructures with customized env. are possible
Application Developers
  How to use cloud techniques to improve my applications?
  … and my development workflows?
  Applications can be services (with assoc. pluses and minuses)
Data Centers
  Reuse existing (commodity) hardware investments
  Take advantage of (and train) existing system administrators
  How to manage/use a (private, community, public) cloud?
Significant benefits from cloud even without large scale elasticity!
16
Challenges
17
Elasticity
Can we have infinite elasticity with limited resources?
“Local” solutions
 Economic models to avoid hitting infrastructure limits?
 Spot instances and/or different service classes?
 Aside: IPv6 addressing is necessary for large (scientific) clouds
Federated solutions
 Hybrid (scale-out) infrastructures?
 Higher-level brokers or orchestrators?
Cannot have elasticity without some kind of accounting!
18
Data Management (Legal)
Transfer and treatment of data across borders
 Differing legal protections in different jurisdictions
 Legal constraints for data locality (banking, medical data)
 Unclear responsibilities for data: guardian, custodian, owner, etc.
 Europe working hard to come to a consistent legal framework
Protection of data in the cloud
 Consistent access controls for all data locations
 Guarantees about data protection from cloud provider personnel
 Reliability of the provider’s storage
 Knowledge about provider’s policies for data protection
19
Data Management (Technical)
Efficient exploitation of large datasets
 Need significant computing next to storage, AND/OR
 High bandwidth remote access
Locality matters
 Sometimes it is inconvenient to transfer raw data away from instrument
 Clouds can be used to reduce data locally before transfer
“Open Data” requirements
 Cloud may help meet such requirements
 Does not remove need for well defined dataset metadata and format
 Need long-term funding for the curation of those datasets
20
Security
How to maintain the security of a cloud infrastructure?
Shifting some responsibility to users:
 Users have root access and must secure services within their VMs
 Users have less security experience  need for education & help
 Dynamic network configurations can help improve security
Changing expectations from administrators:
 Leave firewall policies to users running VMs
 Should not expect to run security software inside of VMs
 Need to enhance monitoring to discover abnormal behavior
21
Image Management
Image metadata
 What does an image contain (OS, services, configuration, etc.)?
 What versions of the kernel, software, etc. are included?
 Who is responsible and/or supports a given image?
 How do I identify a given image?
Creating machine images
 How can an image be created for multiple clouds?
 What do I have to do to create a secure machine image?
Sharing images
 How can I make my images available to others?
 Can I parameterize my images to make them useful to more people?
 Can the images be transported and used efficiently?
22
Vendor Lock-in vs. Federation
Common API
 Fully interoperable API avoids duplication in cloud control software
 Has very limited impact on applications and services in the cloud
 Current (quasi-) standards: EC2, OCCI, CIMI, CDMI, …
Common Semantics
 Semantics determine how apps and services operate in the cloud
 For IaaS, cloud providers have a broadly similar semantics, but…
 File-based and block storage is one difference.
Contextualization
 Can a user run the same validated image on all clouds?
 Can a user share the same parameterized image with others?
 Neglected issue in standardization; CloudInit becoming de facto std.
23
Solutions
24
StratusLab History
Informal collaboration to investigate
running grid services on Amazon
EC2 (2007)
StratusLab Project (6/2010 to
5/2012) co-funded by EC with
6 partners from 5 countries
Open collaboration
to continue the
development and support of
the StratusLab software
Website: http://stratuslab.eu
Twitter: @StratusLab
Support: support@stratuslab.eu
Source: http://github.com/StratusLab
Identified need for open
source cloud distribution.
Production dist. with academic
& commercial deployments.
25
StratusLab
Complete Infrastructure as a Service cloud distribution
 Developed within EU project, software maintained by partners
 Focus: Simple to install and simple to use
Services
 Compute: Virtual machine management (currently uses OpenNebula)
 Storage: Volume-based storage service
 Network: Simple configuration for public, local, and private VM access
 Image mgt.: Complete system for trusted sharing of VM images
 Tools (python CLI) and APIs (Libcloud) to facilitate use of cloud
 Tools to facilitate installation of services
26
SlipStream
Cloud orchestrator and deployment engine
 Facilitates testing, deployment, and maintenance of complex systems
 Transparent access
to multiple cloud
infrastructures
 Allows automated
deployment of systems
in one or more clouds
27
Image Management
28
Image Management
Image metadata
 What does an image contain (OS, services, configuration, etc.)?
 What versions of the kernel, software, etc. are included?
 Who is responsible and/or supports a given image?
 How do I identify a given image?
Creating machine images
 How can an image be created for multiple clouds?
 What do I have to do to create a secure machine image?
Sharing images
 How can I make my images available to others?
 Can I parameterize my images to make them useful to more people?
 Can the images be transported and used efficiently?
29
Image Management Actors
30
Marketplace
Priorities
 Mechanism for sharing and trusting images
 Possible to distribute fixed, read-only data sets as well
 Split the storage of image metadata and image contents
 Define roles for creator, user, administrator, and validator
Implementation
 Marketplace API: Proprietary REST API for create, read, search
 Marketplace acts as image registry and handles only metadata
 Image contents can be located on any public (web) server
 ‘Private’ images can also be held in cloud storage
31
Marketplace: Key service in larger ecosystem
Trust
 Marketplace metadata plays key role in providing information about the
image contents and provenance
Factories
 stratus-create-image facilitates customization of images
 VirtualBox and other local virtual tools
 Bitnami and many similar services for build of standard OS services
 Quattor and other fabric management can be used
Transport (& Storage)
 StratusLab uses simple HTTP(S) for image transport
 Could imagine using ftp, gridftp, bittorrent, etc.
 vm-caster/catcher in EGI federated cloud task force
32
Image Handling Workflow
33
Cloud Federation
34
StratusLab Federated Cloud Infrastructure
Features
 Two sites operating (LAL and GRNET) for ~3 years
 Common user authentication
 Ability to use the same images across resources
 StratusLab client allows easy switching between sites
 StratusLab Libcloud binding allows common view of both sites
Need to go further…
 To sites running different cloud software
 Helix Nebula and EGI Fed. Cloud Task Force active in this area
35
Federation Models
Transparent Federation
 Site operators “outsource” to other providers
 Completely transparent to end users
 Difficult to achieve in practice because of data protection concerns and
network access/performance
Brokered Federation
 Variety of different cloud infrastructures are visible to users
 Users choose to place virtual machines in particular locations
 Simple clients can handle federation if differences are small
 Orchestrators are needed for larger differences between clouds
Both Helix Nebula and EGI take the brokered approach
36
Vendor Lock-in vs. Federation
Common API — minor issue
 Fully interoperable API avoids duplication in cloud control software
 Has very limited impact on applications and services in the cloud
 Current (quasi-) standards: EC2, OCCI, CIMI, CDMI, …
Common Semantics — important issue
 Semantics determine how apps and services operate in the cloud
 For IaaS, cloud providers have a broadly similar semantics, but…
 File-based and block storage is one difference.
Contextualization — critical issue
 Can a user run the same validated image on all clouds?
 Can a user share the same parameterized image with others?
 Neglected in standardization but CloudInit becoming de facto std.
37
“Our” Contributions
StratusLab
 Adopt CIMI as the standard interface to services
 Provide Libcloud (python) driver for StratusLab
 Provide EC2, OCCI, etc. as adaptors to CIMI interface
 Move to CloudInit for contextualization framework
 Flexible authentication system to support different methods
SlipStream
 Chosen as main interface to supported clouds in Helix Nebula
 Already supports multi-cloud deployments
 Support for all clouds within Helix Nebula
38
Running Clouds in Production
39
Cloud Experience at LAL
Private cloud for laboratory services
 Works well, plan to migrate all services including grid worker nodes
and experiment-specific servers
 Services switched to VMs without users being aware of change
 Very different way of working, need to change administrator habits
 Have seen some stability issues related to SL6 kernel/virtualization
Public cloud open to university
 Very positive reaction to cloud; LAL resources nearly 100% used
 Variety of disciplines: biology, software eng., statistics, astrophysics,
bioinformatics, …
 After initial introduction, users require only low level of support
 Other labs offering StratusLab training without our direct involvement
40
Summary
41
Summary
Cloud for scientific computing
 Already a common tool for many scientific disciplines
 Cloud technologies will become more pervasive with time
 Associated swing back to larger, centralized data centers
Many challenges for cloud
 Elasticity with limited resources
 Data management (legal & technical)
 Security
 Image management — unique holistic approach from StratusLab
 Federation — brokered federation with StratusLab & SlipStream
LAL cloud experience
 Very positive feedback from both administrators and users
42
Questions and Discussion
Website: http://stratuslab.eu/
Twitter: @StratusLab
Support: support@stratuslab.eu
Source: http://github.com/StratusLab
SixSq: http://sixsq.com
SlipStream: https://github.com/organizations/slipstream
http://stratuslab.eu/
Copyright © 2013, Members of the StratusLab collaboration.
This work is licensed under the Creative Commons Attribution 3.0
Unported License (http://creativecommons.org/licenses/by/3.0/).

More Related Content

What's hot

Cloud Computing- Proposal (Autosaved)
Cloud Computing- Proposal (Autosaved)Cloud Computing- Proposal (Autosaved)
Cloud Computing- Proposal (Autosaved)Zuhair Haroon khan
 
Cloud computing reference architecture from nist and ibm
Cloud computing reference architecture from nist and ibmCloud computing reference architecture from nist and ibm
Cloud computing reference architecture from nist and ibmRichard Kuo
 
www.iosrjournals.org 57 | Page Latest development of cloud computing technolo...
www.iosrjournals.org 57 | Page Latest development of cloud computing technolo...www.iosrjournals.org 57 | Page Latest development of cloud computing technolo...
www.iosrjournals.org 57 | Page Latest development of cloud computing technolo...Sushil kumar Choudhary
 
International Journal of Engineering Research and Development
International Journal of Engineering Research and DevelopmentInternational Journal of Engineering Research and Development
International Journal of Engineering Research and DevelopmentIJERD Editor
 
Cloud Computing Security
Cloud Computing SecurityCloud Computing Security
Cloud Computing SecurityNinh Nguyen
 
Application Model for Cloud Deployment
Application Model for Cloud DeploymentApplication Model for Cloud Deployment
Application Model for Cloud DeploymentJim Kaskade
 
Public cloud: A Review
Public cloud: A ReviewPublic cloud: A Review
Public cloud: A ReviewAjay844
 
Ignacio design and building of iaa s clouds
Ignacio design and building of iaa s cloudsIgnacio design and building of iaa s clouds
Ignacio design and building of iaa s cloudsEuroCloud
 
Cloud computing applications for e health
Cloud computing applications for e healthCloud computing applications for e health
Cloud computing applications for e healthHector Martin Garcia
 
Cloud computing - Terena 2011
Cloud computing - Terena 2011Cloud computing - Terena 2011
Cloud computing - Terena 2011Jisc
 
Basic Technology - Module 13 cloud computing
Basic Technology - Module 13 cloud computingBasic Technology - Module 13 cloud computing
Basic Technology - Module 13 cloud computingsolarisyougood
 
Cloud computing security and privacy
Cloud computing security and privacyCloud computing security and privacy
Cloud computing security and privacyAdeel Javaid
 
NIST Definition of Cloud Computing v15
NIST Definition of Cloud Computing v15NIST Definition of Cloud Computing v15
NIST Definition of Cloud Computing v15Bill Annibell
 
Cloud Computing Security Issues in Infrastructure as a Service”
Cloud Computing Security Issues in Infrastructure as a Service”Cloud Computing Security Issues in Infrastructure as a Service”
Cloud Computing Security Issues in Infrastructure as a Service”Vivek Maurya
 
Data Security Model Enhancement In Cloud Environment
Data Security Model Enhancement In Cloud EnvironmentData Security Model Enhancement In Cloud Environment
Data Security Model Enhancement In Cloud EnvironmentIOSR Journals
 

What's hot (20)

Cloud Computing- Proposal (Autosaved)
Cloud Computing- Proposal (Autosaved)Cloud Computing- Proposal (Autosaved)
Cloud Computing- Proposal (Autosaved)
 
Cloud computing
Cloud computingCloud computing
Cloud computing
 
Cloud computing reference architecture from nist and ibm
Cloud computing reference architecture from nist and ibmCloud computing reference architecture from nist and ibm
Cloud computing reference architecture from nist and ibm
 
www.iosrjournals.org 57 | Page Latest development of cloud computing technolo...
www.iosrjournals.org 57 | Page Latest development of cloud computing technolo...www.iosrjournals.org 57 | Page Latest development of cloud computing technolo...
www.iosrjournals.org 57 | Page Latest development of cloud computing technolo...
 
Cs6703 grid and cloud computing unit 3
Cs6703 grid and cloud computing unit 3Cs6703 grid and cloud computing unit 3
Cs6703 grid and cloud computing unit 3
 
International Journal of Engineering Research and Development
International Journal of Engineering Research and DevelopmentInternational Journal of Engineering Research and Development
International Journal of Engineering Research and Development
 
CLOUD COMPUTING_proposal
CLOUD COMPUTING_proposalCLOUD COMPUTING_proposal
CLOUD COMPUTING_proposal
 
Cloud Computing Security
Cloud Computing SecurityCloud Computing Security
Cloud Computing Security
 
Cloud Computing Architecture
Cloud Computing ArchitectureCloud Computing Architecture
Cloud Computing Architecture
 
Application Model for Cloud Deployment
Application Model for Cloud DeploymentApplication Model for Cloud Deployment
Application Model for Cloud Deployment
 
Public cloud: A Review
Public cloud: A ReviewPublic cloud: A Review
Public cloud: A Review
 
Ignacio design and building of iaa s clouds
Ignacio design and building of iaa s cloudsIgnacio design and building of iaa s clouds
Ignacio design and building of iaa s clouds
 
Cloud computing applications for e health
Cloud computing applications for e healthCloud computing applications for e health
Cloud computing applications for e health
 
Cloud computing - Terena 2011
Cloud computing - Terena 2011Cloud computing - Terena 2011
Cloud computing - Terena 2011
 
Basic Technology - Module 13 cloud computing
Basic Technology - Module 13 cloud computingBasic Technology - Module 13 cloud computing
Basic Technology - Module 13 cloud computing
 
Cloud computing in healthcare
Cloud computing in healthcareCloud computing in healthcare
Cloud computing in healthcare
 
Cloud computing security and privacy
Cloud computing security and privacyCloud computing security and privacy
Cloud computing security and privacy
 
NIST Definition of Cloud Computing v15
NIST Definition of Cloud Computing v15NIST Definition of Cloud Computing v15
NIST Definition of Cloud Computing v15
 
Cloud Computing Security Issues in Infrastructure as a Service”
Cloud Computing Security Issues in Infrastructure as a Service”Cloud Computing Security Issues in Infrastructure as a Service”
Cloud Computing Security Issues in Infrastructure as a Service”
 
Data Security Model Enhancement In Cloud Environment
Data Security Model Enhancement In Cloud EnvironmentData Security Model Enhancement In Cloud Environment
Data Security Model Enhancement In Cloud Environment
 

Viewers also liked

Cloud computing present
Cloud computing presentCloud computing present
Cloud computing presentJames Sutter
 
Cloud Computing Solutions Past,Present and Future
Cloud Computing Solutions Past,Present and FutureCloud Computing Solutions Past,Present and Future
Cloud Computing Solutions Past,Present and FutureNexiilabs
 
Trend and Future of Cloud Computing
Trend and Future of Cloud ComputingTrend and Future of Cloud Computing
Trend and Future of Cloud Computinghybrid cloud
 
The Inevitable Cloud Outage
The Inevitable Cloud OutageThe Inevitable Cloud Outage
The Inevitable Cloud OutageNewvewm
 
Can we hack open source #cloud platforms to help reduce emissions?
Can we hack open source #cloud platforms to help reduce emissions?Can we hack open source #cloud platforms to help reduce emissions?
Can we hack open source #cloud platforms to help reduce emissions?Tom Raftery
 
Avoiding Cloud Outage
Avoiding Cloud OutageAvoiding Cloud Outage
Avoiding Cloud OutageNati Shalom
 
Summer School Scale Cloud Across the Enterprise
Summer School   Scale Cloud Across the EnterpriseSummer School   Scale Cloud Across the Enterprise
Summer School Scale Cloud Across the EnterpriseWSO2
 
Simplifying The Cloud Top 10 Questions By SMBs
Simplifying The Cloud Top 10 Questions By SMBsSimplifying The Cloud Top 10 Questions By SMBs
Simplifying The Cloud Top 10 Questions By SMBsSun Digital, Inc.
 
Penetrating the Cloud: Opportunities & Challenges for Businesses
Penetrating the Cloud: Opportunities & Challenges for BusinessesPenetrating the Cloud: Opportunities & Challenges for Businesses
Penetrating the Cloud: Opportunities & Challenges for BusinessesCompTIA
 
The Total Cost of Ownership (TCO) of Web Applications in the AWS Cloud - Jine...
The Total Cost of Ownership (TCO) of Web Applications in the AWS Cloud - Jine...The Total Cost of Ownership (TCO) of Web Applications in the AWS Cloud - Jine...
The Total Cost of Ownership (TCO) of Web Applications in the AWS Cloud - Jine...Amazon Web Services
 
Intro to cloud computing — MegaCOMM 2013, Jerusalem
Intro to cloud computing — MegaCOMM 2013, JerusalemIntro to cloud computing — MegaCOMM 2013, Jerusalem
Intro to cloud computing — MegaCOMM 2013, JerusalemReuven Lerner
 
Best Practices for Architecting in the Cloud - Jeff Barr
Best Practices for Architecting in the Cloud - Jeff BarrBest Practices for Architecting in the Cloud - Jeff Barr
Best Practices for Architecting in the Cloud - Jeff BarrAmazon Web Services
 
2013 State of Cloud Survey SMB Results
2013 State of Cloud Survey SMB Results2013 State of Cloud Survey SMB Results
2013 State of Cloud Survey SMB ResultsSymantec
 
Linthicum what is-the-true-future-of-cloud-computing
Linthicum what is-the-true-future-of-cloud-computingLinthicum what is-the-true-future-of-cloud-computing
Linthicum what is-the-true-future-of-cloud-computingDavid Linthicum
 
LinuxFest NW 2013: Hitchhiker's Guide to Open Source Cloud Computing
LinuxFest NW 2013: Hitchhiker's Guide to Open Source Cloud ComputingLinuxFest NW 2013: Hitchhiker's Guide to Open Source Cloud Computing
LinuxFest NW 2013: Hitchhiker's Guide to Open Source Cloud ComputingMark Hinkle
 
Delivering IaaS with Open Source Software
Delivering IaaS with Open Source SoftwareDelivering IaaS with Open Source Software
Delivering IaaS with Open Source SoftwareMark Hinkle
 
2013 Future of Cloud Computing - 3rd Annual Survey Results
2013 Future of Cloud Computing - 3rd Annual Survey Results2013 Future of Cloud Computing - 3rd Annual Survey Results
2013 Future of Cloud Computing - 3rd Annual Survey ResultsMichael Skok
 
Breaking through the Clouds
Breaking through the CloudsBreaking through the Clouds
Breaking through the CloudsAndy Piper
 
Cloud computing simple ppt
Cloud computing simple pptCloud computing simple ppt
Cloud computing simple pptAgarwaljay
 

Viewers also liked (20)

Cloud computing present
Cloud computing presentCloud computing present
Cloud computing present
 
Cloud Computing Solutions Past,Present and Future
Cloud Computing Solutions Past,Present and FutureCloud Computing Solutions Past,Present and Future
Cloud Computing Solutions Past,Present and Future
 
Trend and Future of Cloud Computing
Trend and Future of Cloud ComputingTrend and Future of Cloud Computing
Trend and Future of Cloud Computing
 
The Inevitable Cloud Outage
The Inevitable Cloud OutageThe Inevitable Cloud Outage
The Inevitable Cloud Outage
 
Can we hack open source #cloud platforms to help reduce emissions?
Can we hack open source #cloud platforms to help reduce emissions?Can we hack open source #cloud platforms to help reduce emissions?
Can we hack open source #cloud platforms to help reduce emissions?
 
Avoiding Cloud Outage
Avoiding Cloud OutageAvoiding Cloud Outage
Avoiding Cloud Outage
 
Summer School Scale Cloud Across the Enterprise
Summer School   Scale Cloud Across the EnterpriseSummer School   Scale Cloud Across the Enterprise
Summer School Scale Cloud Across the Enterprise
 
Simplifying The Cloud Top 10 Questions By SMBs
Simplifying The Cloud Top 10 Questions By SMBsSimplifying The Cloud Top 10 Questions By SMBs
Simplifying The Cloud Top 10 Questions By SMBs
 
Penetrating the Cloud: Opportunities & Challenges for Businesses
Penetrating the Cloud: Opportunities & Challenges for BusinessesPenetrating the Cloud: Opportunities & Challenges for Businesses
Penetrating the Cloud: Opportunities & Challenges for Businesses
 
The Total Cost of Ownership (TCO) of Web Applications in the AWS Cloud - Jine...
The Total Cost of Ownership (TCO) of Web Applications in the AWS Cloud - Jine...The Total Cost of Ownership (TCO) of Web Applications in the AWS Cloud - Jine...
The Total Cost of Ownership (TCO) of Web Applications in the AWS Cloud - Jine...
 
Intro to cloud computing — MegaCOMM 2013, Jerusalem
Intro to cloud computing — MegaCOMM 2013, JerusalemIntro to cloud computing — MegaCOMM 2013, Jerusalem
Intro to cloud computing — MegaCOMM 2013, Jerusalem
 
Best Practices for Architecting in the Cloud - Jeff Barr
Best Practices for Architecting in the Cloud - Jeff BarrBest Practices for Architecting in the Cloud - Jeff Barr
Best Practices for Architecting in the Cloud - Jeff Barr
 
2013 State of Cloud Survey SMB Results
2013 State of Cloud Survey SMB Results2013 State of Cloud Survey SMB Results
2013 State of Cloud Survey SMB Results
 
Linthicum what is-the-true-future-of-cloud-computing
Linthicum what is-the-true-future-of-cloud-computingLinthicum what is-the-true-future-of-cloud-computing
Linthicum what is-the-true-future-of-cloud-computing
 
LinuxFest NW 2013: Hitchhiker's Guide to Open Source Cloud Computing
LinuxFest NW 2013: Hitchhiker's Guide to Open Source Cloud ComputingLinuxFest NW 2013: Hitchhiker's Guide to Open Source Cloud Computing
LinuxFest NW 2013: Hitchhiker's Guide to Open Source Cloud Computing
 
Delivering IaaS with Open Source Software
Delivering IaaS with Open Source SoftwareDelivering IaaS with Open Source Software
Delivering IaaS with Open Source Software
 
2013 Future of Cloud Computing - 3rd Annual Survey Results
2013 Future of Cloud Computing - 3rd Annual Survey Results2013 Future of Cloud Computing - 3rd Annual Survey Results
2013 Future of Cloud Computing - 3rd Annual Survey Results
 
Breaking through the Clouds
Breaking through the CloudsBreaking through the Clouds
Breaking through the Clouds
 
Cloud computing ppt
Cloud computing pptCloud computing ppt
Cloud computing ppt
 
Cloud computing simple ppt
Cloud computing simple pptCloud computing simple ppt
Cloud computing simple ppt
 

Similar to Scientific Cloud Computing: Present & Future

Cloud Computing genral for all concepts.pptx
Cloud Computing genral for all concepts.pptxCloud Computing genral for all concepts.pptx
Cloud Computing genral for all concepts.pptxraghavanp4
 
Cloud computing
Cloud computingCloud computing
Cloud computingSyam Lal
 
Cloudmod4
Cloudmod4Cloudmod4
Cloudmod4kongara
 
Introduction to Cloud Computing
Introduction to Cloud ComputingIntroduction to Cloud Computing
Introduction to Cloud ComputingAnimesh Chaturvedi
 
Introduction to aneka cloud
Introduction to aneka cloudIntroduction to aneka cloud
Introduction to aneka cloudssuser84183f
 
Wicsa2011 cloud tutorial
Wicsa2011 cloud tutorialWicsa2011 cloud tutorial
Wicsa2011 cloud tutorialAnna Liu
 
Cloud Computing Networks
Cloud Computing NetworksCloud Computing Networks
Cloud Computing Networksjayapal385
 
Cloud computing and its application in libraries
Cloud computing and its application in librariesCloud computing and its application in libraries
Cloud computing and its application in librariesNabi Hasan
 
Implementing security groups in open stack
Implementing security groups in open stackImplementing security groups in open stack
Implementing security groups in open stackRishabh Agarwal
 
Bob Plumridge - Enabling easier Cloud Solution Deployment (Storage Expo 2010)
Bob Plumridge - Enabling easier Cloud Solution Deployment (Storage Expo 2010)Bob Plumridge - Enabling easier Cloud Solution Deployment (Storage Expo 2010)
Bob Plumridge - Enabling easier Cloud Solution Deployment (Storage Expo 2010)VNU Exhibitions Europe
 
cloudintro-lec01.ppt
cloudintro-lec01.pptcloudintro-lec01.ppt
cloudintro-lec01.pptMunmunSaha7
 

Similar to Scientific Cloud Computing: Present & Future (20)

Cloud Computing genral for all concepts.pptx
Cloud Computing genral for all concepts.pptxCloud Computing genral for all concepts.pptx
Cloud Computing genral for all concepts.pptx
 
Cloud computing
Cloud computingCloud computing
Cloud computing
 
Cloudmod4
Cloudmod4Cloudmod4
Cloudmod4
 
Introduction to Cloud Computing
Introduction to Cloud ComputingIntroduction to Cloud Computing
Introduction to Cloud Computing
 
Cloud computing
Cloud computingCloud computing
Cloud computing
 
Introduction to aneka cloud
Introduction to aneka cloudIntroduction to aneka cloud
Introduction to aneka cloud
 
Wicsa2011 cloud tutorial
Wicsa2011 cloud tutorialWicsa2011 cloud tutorial
Wicsa2011 cloud tutorial
 
cloud computing models
cloud computing modelscloud computing models
cloud computing models
 
Cloud Computing Networks
Cloud Computing NetworksCloud Computing Networks
Cloud Computing Networks
 
Module-1 introductaion cloud computing.pdf
Module-1 introductaion cloud computing.pdfModule-1 introductaion cloud computing.pdf
Module-1 introductaion cloud computing.pdf
 
Cloud Computing
Cloud ComputingCloud Computing
Cloud Computing
 
Cloud computing What Why How
Cloud computing What Why HowCloud computing What Why How
Cloud computing What Why How
 
Securing The Journey To The Cloud
Securing The Journey To The Cloud Securing The Journey To The Cloud
Securing The Journey To The Cloud
 
CLOUD COMPUTING.pptx
CLOUD COMPUTING.pptxCLOUD COMPUTING.pptx
CLOUD COMPUTING.pptx
 
Cloud computing and its application in libraries
Cloud computing and its application in librariesCloud computing and its application in libraries
Cloud computing and its application in libraries
 
Implementing security groups in open stack
Implementing security groups in open stackImplementing security groups in open stack
Implementing security groups in open stack
 
cloud computing
cloud computing cloud computing
cloud computing
 
Computer project
Computer projectComputer project
Computer project
 
Bob Plumridge - Enabling easier Cloud Solution Deployment (Storage Expo 2010)
Bob Plumridge - Enabling easier Cloud Solution Deployment (Storage Expo 2010)Bob Plumridge - Enabling easier Cloud Solution Deployment (Storage Expo 2010)
Bob Plumridge - Enabling easier Cloud Solution Deployment (Storage Expo 2010)
 
cloudintro-lec01.ppt
cloudintro-lec01.pptcloudintro-lec01.ppt
cloudintro-lec01.ppt
 

More from stratuslab

StratusLab: A IaaS Cloud Distribution Focusing on Simplicity
StratusLab: A IaaS Cloud Distribution Focusing on SimplicityStratusLab: A IaaS Cloud Distribution Focusing on Simplicity
StratusLab: A IaaS Cloud Distribution Focusing on Simplicitystratuslab
 
Providing Bioinformatics Services on Cloud
Providing Bioinformatics Services on CloudProviding Bioinformatics Services on Cloud
Providing Bioinformatics Services on Cloudstratuslab
 
IDB-Cloud Providing Bioinformatics Services on Cloud
IDB-Cloud Providing Bioinformatics Services on CloudIDB-Cloud Providing Bioinformatics Services on Cloud
IDB-Cloud Providing Bioinformatics Services on Cloudstratuslab
 
StratusLab: Darn Simple Cloud
StratusLab: Darn Simple CloudStratusLab: Darn Simple Cloud
StratusLab: Darn Simple Cloudstratuslab
 
Feuille de Route pour StratusLab
Feuille de Route pour StratusLabFeuille de Route pour StratusLab
Feuille de Route pour StratusLabstratuslab
 
StratusLab at FOSDEM'13
StratusLab at FOSDEM'13StratusLab at FOSDEM'13
StratusLab at FOSDEM'13stratuslab
 

More from stratuslab (6)

StratusLab: A IaaS Cloud Distribution Focusing on Simplicity
StratusLab: A IaaS Cloud Distribution Focusing on SimplicityStratusLab: A IaaS Cloud Distribution Focusing on Simplicity
StratusLab: A IaaS Cloud Distribution Focusing on Simplicity
 
Providing Bioinformatics Services on Cloud
Providing Bioinformatics Services on CloudProviding Bioinformatics Services on Cloud
Providing Bioinformatics Services on Cloud
 
IDB-Cloud Providing Bioinformatics Services on Cloud
IDB-Cloud Providing Bioinformatics Services on CloudIDB-Cloud Providing Bioinformatics Services on Cloud
IDB-Cloud Providing Bioinformatics Services on Cloud
 
StratusLab: Darn Simple Cloud
StratusLab: Darn Simple CloudStratusLab: Darn Simple Cloud
StratusLab: Darn Simple Cloud
 
Feuille de Route pour StratusLab
Feuille de Route pour StratusLabFeuille de Route pour StratusLab
Feuille de Route pour StratusLab
 
StratusLab at FOSDEM'13
StratusLab at FOSDEM'13StratusLab at FOSDEM'13
StratusLab at FOSDEM'13
 

Recently uploaded

Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxFIDO Alliance
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingScyllaDB
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentationyogeshlabana357357
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...ScyllaDB
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxFIDO Alliance
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch TuesdayIvanti
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!Memoori
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontologyjohnbeverley2021
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxjbellis
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAnitaRaj43
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTopCSSGallery
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMKumar Satyam
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Skynet Technologies
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Paige Cruz
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfdanishmna97
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe中 央社
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...ScyllaDB
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxMarkSteadman7
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxFIDO Alliance
 

Recently uploaded (20)

Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptxHarnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
Harnessing Passkeys in the Battle Against AI-Powered Cyber Threats.pptx
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
Introduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptxIntroduction to FIDO Authentication and Passkeys.pptx
Introduction to FIDO Authentication and Passkeys.pptx
 
2024 May Patch Tuesday
2024 May Patch Tuesday2024 May Patch Tuesday
2024 May Patch Tuesday
 
State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
AI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by AnitarajAI in Action: Real World Use Cases by Anitaraj
AI in Action: Real World Use Cases by Anitaraj
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
Observability Concepts EVERY Developer Should Know (DevOpsDays Seattle)
 
How to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cfHow to Check CNIC Information Online with Pakdata cf
How to Check CNIC Information Online with Pakdata cf
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
Event-Driven Architecture Masterclass: Engineering a Robust, High-performance...
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 

Scientific Cloud Computing: Present & Future

  • 1. Scientific Cloud Computing: Present & Future Charles (Cal) Loomis (CNRS/LAL & SixSq Sàrl) INFN CNAF, Bologna, Italy (22 May 2013)
  • 2. 2 Cloud Marketing “Cloud” is currently very trendy, used everywhere  Many definitions that are often incompatible  Used (often) to market pre-existing (non-cloud) software CommodityComputing(Sun) UtilityComputing(IBM,HP,…) AmazonEC2 AmazonEBS Mature Virtualization Simple APIs Excess Capacity
  • 3. 3 In two pages NIST defines:  Essential characteristics  Deployment models  Service models What is a Cloud? http://csrc.nist.gov/publications/ nistpubs/800-145/SP800-145.pdf
  • 4. 4 On-demand self-service  No human intervention Broad network access  Fast, reliable remote access Rapid elasticity  Scale based on app. needs Resource pooling  Multi-tenant sharing Measured service  Direct or indirect economic model with measured use Essential Characteristics http://csrc.nist.gov/publications/ nistpubs/800-145/SP800-145.pdf
  • 5. 5 Private  Single administrative domain, limited number of users Community  Different administrative domains with common interests & proc. Public  People outside of institute’s administrative domain Hybrid  Federation via combination of other deployment models Deployment Models http://csrc.nist.gov/publications/ nistpubs/800-145/SP800-145.pdf
  • 6. 6 Software as a Service (SaaS)  Direct (scalable) hosting of end user applications Platform as a Service (PaaS)  Framework and infrastructure for creating web applications Infrastructure as a Service (IaaS)  Access to remote virtual machines with root access Service Models http://csrc.nist.gov/publications/ nistpubs/800-145/SP800-145.pdf
  • 7. 7 Advantages  No software installation  Universally accessible Disadvantages  Questions about data access, ownership, reliability, etc.  Integration of services & novel uses of data are (often) difficult Trends  Social scientific computing  Service APIs to allow integration  PaaS Software as a Service (SaaS)
  • 8. 8 Advantages  Programmers take advantage of integrated load balancing, automatic failover, etc. Disadvantages  Restricted number of languages  Applications strongly locked to a particular provider Trends  Dearth of “pure” PaaS offers  Encroachment from both SaaS and IaaS sides Platform as a Service (PaaS)
  • 9. 9 Advantages  Customized environment with “root” access  Easy access to scalable resources Disadvantages  Variety of APIs and interfaces  VM image creation is difficult and time-consuming Trends  Lots of specialized cloud providers appearing  Orchestration pushing into PaaS space Infrastructure as a Service (IaaS)
  • 11. 11 State of the Art Commercial Provider: Amazon Web Services (AWS)  Leading and largest IaaS service provider  Improving and adding new services at a phenomenal rate  Almost all IaaS providers use AWS-like service semantics, but differentiate based on price, SLAs, location, etc. Commercial Cloud Distribution: VM-ware  VM-ware: extremely good and complete, but very expensive  Provide ESXi virtual machine host for free Open Source Cloud Distributions  Essentially none in 2004; now easily a dozen different distributions  StratusLab, WNoDeS, …, OpenStack, OpenNebula, CloudStack  Very different levels of maturity, stability, scalability, etc.
  • 12. 12 Why are cloud technologies useful? For (scientific) users  Custom environment: no rewriting or porting applications to fit into a resource provider’s environment  Simple access: most providers use a REST or RPC API allowing simple access from all programming languages  Reasonable cost: only pay for what resources are used, especially attractive for individuals/groups that do not have large, existing hardware investment
  • 13. 13 Why are cloud technologies useful? Separation of responsibilities  Hardware / Services / Platforms / Users  People at each layer can focus on their responsibilities with minimal interactions with people in the other layers. Resource Providers  Better utilization of shared resource because wider range of applications (and disciplines) can use the cloud  With hybrid cloud infrastructures, providers can outsource excess demand to other providers
  • 14. 14 Trend for Scientific Computing Will we all just be users of the Amazon cloud? Pendulum swinging towards large data centers with “fat” machines  These can offer elastic cloud services at a reasonable price  With scientific clouds there is low barrier to entry and users can maintain administrative control of services and data  Providing shared resource between scientific disciplines much easier because of virtualization Migration will be gradual…
  • 15. 15 Overcoming inertia… Users   How to use virtual machines to get my work done?   How to structure, store, access, and protect data?   Realize shared infrastructures with customized env. are possible Application Developers   How to use cloud techniques to improve my applications?   … and my development workflows?   Applications can be services (with assoc. pluses and minuses) Data Centers   Reuse existing (commodity) hardware investments   Take advantage of (and train) existing system administrators   How to manage/use a (private, community, public) cloud? Significant benefits from cloud even without large scale elasticity!
  • 17. 17 Elasticity Can we have infinite elasticity with limited resources? “Local” solutions  Economic models to avoid hitting infrastructure limits?  Spot instances and/or different service classes?  Aside: IPv6 addressing is necessary for large (scientific) clouds Federated solutions  Hybrid (scale-out) infrastructures?  Higher-level brokers or orchestrators? Cannot have elasticity without some kind of accounting!
  • 18. 18 Data Management (Legal) Transfer and treatment of data across borders  Differing legal protections in different jurisdictions  Legal constraints for data locality (banking, medical data)  Unclear responsibilities for data: guardian, custodian, owner, etc.  Europe working hard to come to a consistent legal framework Protection of data in the cloud  Consistent access controls for all data locations  Guarantees about data protection from cloud provider personnel  Reliability of the provider’s storage  Knowledge about provider’s policies for data protection
  • 19. 19 Data Management (Technical) Efficient exploitation of large datasets  Need significant computing next to storage, AND/OR  High bandwidth remote access Locality matters  Sometimes it is inconvenient to transfer raw data away from instrument  Clouds can be used to reduce data locally before transfer “Open Data” requirements  Cloud may help meet such requirements  Does not remove need for well defined dataset metadata and format  Need long-term funding for the curation of those datasets
  • 20. 20 Security How to maintain the security of a cloud infrastructure? Shifting some responsibility to users:  Users have root access and must secure services within their VMs  Users have less security experience  need for education & help  Dynamic network configurations can help improve security Changing expectations from administrators:  Leave firewall policies to users running VMs  Should not expect to run security software inside of VMs  Need to enhance monitoring to discover abnormal behavior
  • 21. 21 Image Management Image metadata  What does an image contain (OS, services, configuration, etc.)?  What versions of the kernel, software, etc. are included?  Who is responsible and/or supports a given image?  How do I identify a given image? Creating machine images  How can an image be created for multiple clouds?  What do I have to do to create a secure machine image? Sharing images  How can I make my images available to others?  Can I parameterize my images to make them useful to more people?  Can the images be transported and used efficiently?
  • 22. 22 Vendor Lock-in vs. Federation Common API  Fully interoperable API avoids duplication in cloud control software  Has very limited impact on applications and services in the cloud  Current (quasi-) standards: EC2, OCCI, CIMI, CDMI, … Common Semantics  Semantics determine how apps and services operate in the cloud  For IaaS, cloud providers have a broadly similar semantics, but…  File-based and block storage is one difference. Contextualization  Can a user run the same validated image on all clouds?  Can a user share the same parameterized image with others?  Neglected issue in standardization; CloudInit becoming de facto std.
  • 24. 24 StratusLab History Informal collaboration to investigate running grid services on Amazon EC2 (2007) StratusLab Project (6/2010 to 5/2012) co-funded by EC with 6 partners from 5 countries Open collaboration to continue the development and support of the StratusLab software Website: http://stratuslab.eu Twitter: @StratusLab Support: support@stratuslab.eu Source: http://github.com/StratusLab Identified need for open source cloud distribution. Production dist. with academic & commercial deployments.
  • 25. 25 StratusLab Complete Infrastructure as a Service cloud distribution  Developed within EU project, software maintained by partners  Focus: Simple to install and simple to use Services  Compute: Virtual machine management (currently uses OpenNebula)  Storage: Volume-based storage service  Network: Simple configuration for public, local, and private VM access  Image mgt.: Complete system for trusted sharing of VM images  Tools (python CLI) and APIs (Libcloud) to facilitate use of cloud  Tools to facilitate installation of services
  • 26. 26 SlipStream Cloud orchestrator and deployment engine  Facilitates testing, deployment, and maintenance of complex systems  Transparent access to multiple cloud infrastructures  Allows automated deployment of systems in one or more clouds
  • 28. 28 Image Management Image metadata  What does an image contain (OS, services, configuration, etc.)?  What versions of the kernel, software, etc. are included?  Who is responsible and/or supports a given image?  How do I identify a given image? Creating machine images  How can an image be created for multiple clouds?  What do I have to do to create a secure machine image? Sharing images  How can I make my images available to others?  Can I parameterize my images to make them useful to more people?  Can the images be transported and used efficiently?
  • 30. 30 Marketplace Priorities  Mechanism for sharing and trusting images  Possible to distribute fixed, read-only data sets as well  Split the storage of image metadata and image contents  Define roles for creator, user, administrator, and validator Implementation  Marketplace API: Proprietary REST API for create, read, search  Marketplace acts as image registry and handles only metadata  Image contents can be located on any public (web) server  ‘Private’ images can also be held in cloud storage
  • 31. 31 Marketplace: Key service in larger ecosystem Trust  Marketplace metadata plays key role in providing information about the image contents and provenance Factories  stratus-create-image facilitates customization of images  VirtualBox and other local virtual tools  Bitnami and many similar services for build of standard OS services  Quattor and other fabric management can be used Transport (& Storage)  StratusLab uses simple HTTP(S) for image transport  Could imagine using ftp, gridftp, bittorrent, etc.  vm-caster/catcher in EGI federated cloud task force
  • 34. 34 StratusLab Federated Cloud Infrastructure Features  Two sites operating (LAL and GRNET) for ~3 years  Common user authentication  Ability to use the same images across resources  StratusLab client allows easy switching between sites  StratusLab Libcloud binding allows common view of both sites Need to go further…  To sites running different cloud software  Helix Nebula and EGI Fed. Cloud Task Force active in this area
  • 35. 35 Federation Models Transparent Federation  Site operators “outsource” to other providers  Completely transparent to end users  Difficult to achieve in practice because of data protection concerns and network access/performance Brokered Federation  Variety of different cloud infrastructures are visible to users  Users choose to place virtual machines in particular locations  Simple clients can handle federation if differences are small  Orchestrators are needed for larger differences between clouds Both Helix Nebula and EGI take the brokered approach
  • 36. 36 Vendor Lock-in vs. Federation Common API — minor issue  Fully interoperable API avoids duplication in cloud control software  Has very limited impact on applications and services in the cloud  Current (quasi-) standards: EC2, OCCI, CIMI, CDMI, … Common Semantics — important issue  Semantics determine how apps and services operate in the cloud  For IaaS, cloud providers have a broadly similar semantics, but…  File-based and block storage is one difference. Contextualization — critical issue  Can a user run the same validated image on all clouds?  Can a user share the same parameterized image with others?  Neglected in standardization but CloudInit becoming de facto std.
  • 37. 37 “Our” Contributions StratusLab  Adopt CIMI as the standard interface to services  Provide Libcloud (python) driver for StratusLab  Provide EC2, OCCI, etc. as adaptors to CIMI interface  Move to CloudInit for contextualization framework  Flexible authentication system to support different methods SlipStream  Chosen as main interface to supported clouds in Helix Nebula  Already supports multi-cloud deployments  Support for all clouds within Helix Nebula
  • 38. 38 Running Clouds in Production
  • 39. 39 Cloud Experience at LAL Private cloud for laboratory services  Works well, plan to migrate all services including grid worker nodes and experiment-specific servers  Services switched to VMs without users being aware of change  Very different way of working, need to change administrator habits  Have seen some stability issues related to SL6 kernel/virtualization Public cloud open to university  Very positive reaction to cloud; LAL resources nearly 100% used  Variety of disciplines: biology, software eng., statistics, astrophysics, bioinformatics, …  After initial introduction, users require only low level of support  Other labs offering StratusLab training without our direct involvement
  • 41. 41 Summary Cloud for scientific computing  Already a common tool for many scientific disciplines  Cloud technologies will become more pervasive with time  Associated swing back to larger, centralized data centers Many challenges for cloud  Elasticity with limited resources  Data management (legal & technical)  Security  Image management — unique holistic approach from StratusLab  Federation — brokered federation with StratusLab & SlipStream LAL cloud experience  Very positive feedback from both administrators and users
  • 42. 42 Questions and Discussion Website: http://stratuslab.eu/ Twitter: @StratusLab Support: support@stratuslab.eu Source: http://github.com/StratusLab SixSq: http://sixsq.com SlipStream: https://github.com/organizations/slipstream
  • 43. http://stratuslab.eu/ Copyright © 2013, Members of the StratusLab collaboration. This work is licensed under the Creative Commons Attribution 3.0 Unported License (http://creativecommons.org/licenses/by/3.0/).