• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
e-Clouds: a SaaS Marketplace for Scientific Computing
 

e-Clouds: a SaaS Marketplace for Scientific Computing

on

  • 3,707 views

Cloud computing promises to offer great opportunities for research groups; however when researchers want to execute applications in cloud infrastructures many complex processes must be accomplished. ...

Cloud computing promises to offer great opportunities for research groups; however when researchers want to execute applications in cloud infrastructures many complex processes must be accomplished. In this presentation we present the e-Clouds project which will allow researchers to easily execute many applications on public Infrastructure as a Service (IaaS) solutions. Designed for being a Software as a Service (SaaS) marketplace for scientific applications, e-Clouds allows researchers to submit jobs which are transparently executed on public IaaS platforms, such as Amazon Web Services (AWS). e-Clouds manages the on-demand provisioning and configuration of computing instances, storage, applications, schedulers, jobs, and data. The architectural design and how a first application has been supported on e-Clouds are presented. e-Clouds will allow researchers to easily share and execute applications in the cloud at low TCO (Total Cost of Ownership) and without the complexities associated with details of IT configurations and management. e-Clouds provides new opportunities for research groups with low or none budget for dedicated cluster or grid solutions, providing on-demand access to ready-to-use applications and accelerating the result generation of e-Science projects.

Statistics

Views

Total Views
3,707
Views on SlideShare
909
Embed Views
2,798

Actions

Likes
0
Downloads
0
Comments
0

8 Embeds 2,798

http://157.253.238.194 2001
http://ecloudshomepage.herokuapp.com 588
http://www.scoop.it 172
http://murmuring-woodland-6136.herokuapp.com 28
http://www.linkedin.com 6
http://twitter.com 1
https://twitter.com 1
http://hootsuite.scoop.it 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    e-Clouds: a SaaS Marketplace for Scientific Computing e-Clouds: a SaaS Marketplace for Scientific Computing Presentation Transcript

    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USAe-Clouds: A SaaS Marketplace for Scientific Computing Mario Villamizar, Harold Castro, David Mendez Department of Systems and Computing Engineering Universidad de los Andes Bogotá, Colombia
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA THE PROBLEM RELATED WORKSOLUTION REQUIREMENTSSOLUTION ARCHITECTURE IMPLEMENTATION FUTURE WORK
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA THE PROBLEMWhen Small and Medium research groups have projects that requires largeprocessing capabilities they regularly have to use dedicated cluster and/or grid computing infrastructures. However …
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA THE PROBLEM 1) The total cost of ownership (TCO). 2) Complex processes related to IT management. 3) There are different models to parallelize applications. 4) Researchers require large computing capabilities during peak periods.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA THE PROBLEMResearch groups of different sizes are adopting IaaS solutions.The adoption of cloud computing is a complex process, where non-ITresearchers have to face similar or more complex processes as those found incluster or grid infrastructures.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA THE PROBLEMAt enterprise level, the Software as a Service (SaaS) model allows that smallbusinesses can easily adopt complex and ready-to-use applications.Problems associated to the management and the configuration of theunderlying (PaaS, IaaS) infrastructure are abstracted.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA RELATED WORK – e-Clouds GOAL e-Clouds: A SaaS marketplace ofWe believe that cloud scientific applications,computing may provide more where research groups canopportunities for a large access to on-demand andnumber of small and medium ready-to-use applicationsresearch groups. via a Web Browser using a pay-per-use model.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA RELATED WORKSome commercial scientific SaaS marketplaces such as Cyclone and CloudNumbers are offering scientific SaaS application on public IaaS.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA SOLUTION REQUIREMENTSIn most in-house HPC/HTC environments, clusters are the core element usedto execute scientific applications.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA SOLUTION REQUIREMENTS Public Enterprise In-house Public Scientific SaaS Platforms Cluster SaaS (e-Clouds) End-user interfaces End-user interfaces End-user interfaces Web Applications Batch Applications Batch Applications Distributed File Cloud Storage Data Systems Systems Runtime External Libraries External Libraries Middleware Cluster middleware Cluster middleware Operating Systems Operating Systems Operating Systems Virtualization Virtualization Virtualization Servers Servers Servers Storage Storage Storage Networking Networking NetworkingA multi-tenant SaaS solution for scientific computing should allow, to eachresearcher, to control the on-demand deployment of virtual servers, theexecution of jobs of preconfigured scientific applications, and the movementof data from and to the cloud; we call these four elements the workspace.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA SOLUTION REQUIREMENTSPerformance penalties when a common cluster is migrated to the cloud. Scheduling Blob Storage App Management/Installation
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA SOLUTION ARCHITECTURE Virtual cluster Application Data management management management Job execution and Pricing Security monitoringe-Clouds should be based in a new architecture that abstract the aboveproblems, allowing researchers to focus their efforts in result analysis andexperimental designs, and not on technical IT issues.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA SOLUTION ARCHITECTUREThe job executions are transparent tothe user; the user only have to createthe VC, send the jobs, and downloadthe results using the WP. e-Clouds is in charge of interacting with the public IaaS in all steps.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USASOLUTION ARCHITECTURE The e-Clouds architecture for a SaaS marketplace is mainly based in the fact that servers, applications, jobs and user data must be isolated and configured on-demand. Researchers (e-Clouds users) do not have to pay anything if they do not use e-Clouds and they can execute jobs of scientific applications without spending time in complex IT tasks. Operational costs of the e-Clouds Web Portal, e-Clouds Resource Manager and e-Clouds Database are assumed by the e-Clouds team.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USAIMPLEMENTATION Amazon Web Services (AWS) J2EE (Glassfish container) The RM was developed as a Java background process. MySQL EC2, S3 e-Clouds Agent (J2SE) BLAST application
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA EXPERIMENTAL TESTS – EXECUTION TIMES Average Concurrent Network Storage Used in Instance Type Execution Jobs Use (GB) S3 (GB) Time (Hours) 1 Small 0.226 0.226 3.90 5 Small 1.130 1.130 4.57 10 Small 2.260 2.260 3.98 1 Medium 0.226 0.226 2.50 5 Medium 1.130 1.130 1.99 10 Medium 2.260 2.260 2.05 BLAST with two input files: Database (226MB) y Query File (1MB)Small (1 EC2 Unit, 1.7 GB RAM) and medium (2 EC2 Units, 3.75 GB RAM)
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA EXPERIMENTAL TESTS – COSTS Storage TotalConcurrent Instance Networking Processing Cost Cost Jobs Type Cost (USD) Cost (USD) (USD) (USD) 1 Small 0,125 0 0,32 0,445 5 Small 0,625 0 1,92 2,545 10 Small 1,250 0 3,60 4,850 1 Medium 0,125 0 0,48 0,605 5 Medium 0,625 0 2,00 2,625 10 Medium 1,250 0 4,16 5,410 More tests need to be performed.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA IMPLEMENTATIONThe first tests with NCBI BLAST executed on e-Clouds show that through e-Clouds researchers can focus their efforts analyzing and designing thecomputational simulations.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA IMPLEMENTATIONe-Clouds allows that virtual servers, (BoT) applications, data, and jobs can beeasily managed by e-Clouds admins and researchers. The first implementationshows that e-Clouds may be easily extended to include new applications andresearch groups, which should be reflected in new research results.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA FUTURE WORK Usability tests with more research groups (at least 5). Large scale performance tests. Auto scaling and queues mechanisms. More Bag of Tasks applications. e-Clouds as an Open Source project. Shared workspaces. Different public IaaS. Types of applications (MPI, GPUs). Stronger security mechanisms.
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USATHANKS FOR YOUR ATTENTION!
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA CONTACT INFORMATIONmj.villamizar24@uniandes.edu.co mjvc007@hotmail.com @mariocloud http://linkedin.com/in/mariojosevillamizarcano Mario José Villamizar Cano
    • UCC2012: IEEE/ACM Fifth International Conference on Utility and Cloud Computing, Chicago - USA MAGELLAN REPORT (DEC 2011)Cases where Private and Commercial Clouds may be Cost Effective (Magellan R.):Unknown Demand. In the case of a new project or a new application area wherethe potential demand is still poorly understood.Sporadic Demand. One of the more common cases for using commercial cloudofferings is when the demand is highly variable, especially if there are also timesensitive requirements for the service.Facility Constrained. Some sites are severely infrastructure limited.