Appscale: Open-Source Platform-Level Cloud Computing
Upcoming SlideShare
Loading in...5
×
 

Appscale: Open-Source Platform-Level Cloud Computing

on

  • 1,797 views

 

Statistics

Views

Total Views
1,797
Views on SlideShare
1,795
Embed Views
2

Actions

Likes
0
Downloads
52
Comments
0

2 Embeds 2

http://saaspartners.blogspot.com 1
http://saaspartners.blogspot.jp 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Requests are accepted and resources granted via web services Accountability is e-commerce based
  • Scalable services: mapreduce, relational db, VM management, job queue support Rapid uptake in the commercial sector
  • Data APIs: JDO/JPA Structured data
  • Datastore - Flat file database Memcache - In-memory hash table URL Fetch - Works fine, basically a curl / wget Mail - Prints to standard out / error instead of sending mail XMPP (IM) - Same as Mail Images - Works fine once python-imaging library is installed (otherwise fails) Google Accounts - Allows you to log in as anyone with no password Task Queues - Puts tasks on the admin console for later execution Cron - Same as Task Queues Blobstore - Uses URL Fetch to post large files to a URL which then stores it in the flat-file DB
  • Data APIs: JDO/JPA Structured data
  • Data APIs: JDO/JPA Structured data

Appscale: Open-Source Platform-Level Cloud Computing Appscale: Open-Source Platform-Level Cloud Computing Presentation Transcript

  • appscale: open-source platform-level cloud computing Chandra Krintz Computer Science Dept. Univ. of California, Santa Barbara OpenFabrics Alliance, Sonoma Workshop March 15, 2010
  • cloud computing
    • Software systems for accessing easily and transparently scalable CPU/storage/network resources via a network connection or web interface – “ as-a-service”
      • Culmination of grid/cluster/utility/elastic computing
      • Advances in processor, virtualization, systems technology
    • Remote access to distributed and shared cluster resources
      • Has experienced a rapid uptake in the commercial sector
        • Public clouds –your software/apps on others’ systems
        • Users rent a small fraction of vast resource pools
          • Advertised service-level-agreements (SLAs)
          • Resources are opaque and isolated
      • Offer high availability, fault tolerance, and extreme scale
  • cloud computing
    • 3 types: as-a-Service (aaS)
      • Infrastructure: Amazon Web Services (EC2, S3, EBS)
        • Virtualized, isolated (CPU, Network, Storage) systems on which users execute entire runtime stacks
          • Fully customer self-service
        • Open APIs (IaaS standard), scalable services
      • Platform: Google App Engine, Microsoft Azure
        • Scalable program-level abstractions via well-defined interfaces
        • Enable construction of network-accessible applications
        • Process-level (sandbox) isolation, complete software stack
      • Software: Salesforce.com
        • Applications provided to thin clients over a network
        • Customizable
  • many public cloud features useful on-premise
    • Ease of use – of local cluster resources
      • Development, configuration, deployment
      • Broaden participation
    • Scale, performance and resource isolation
    • Concerns
      • Vendor “lock-in”, pay-per-use, and availability reliance
      • Privacy of code and data
      • Resource constraints: storage, CPU/memory, communication
    • Potential for hybrid and customized approaches
      • One size may not fit all
  • many public cloud features useful on-premise
    • Ease of use – of local cluster resources
      • Development, configuration, deployment
      • Broaden participation
    • Difficult to test / try things out in public-only setting
      • Developers visualize the system / have an intellectual model
    • How to support other application domains: scale/availability
      • HPC, data-intensive
    • Clouds are opaque
      • Open APIs, closed implementations
      • What is your application/workload doing?
  • many public cloud features useful on-premise
    • Ease of use – of local cluster resources
      • Development, configuration, deployment
      • Broaden participation
    • Difficult to test / try things out in public-only setting
      • Developers visualize the system / have an intellectual model
    • How to support other application domains: scale/availability
      • HPC, data-intensive
    • Clouds are opaque
      • Open APIs, closed implementations
      • What is your application/workload doing?
      • If everyone does their own…
      • many non-standard APIs!
  • cloud computing fabrics from UCSB
    • Goal : Bring popular cloud fabrics to “on-premise” clusters that are easy to use and are transparent
      • By emulating key cloud layers from the commercial sector
        • Engender user community, access to real applications/users
          • “ Standard” APIs (Amazon AWS, Google App Engine, MapReduce)
        • Leverage extant software technologies
  • cloud computing fabrics from UCSB
    • Goal : Bring popular cloud fabrics to “on-premise” clusters that are easy to use and are transparent
      • By emulating key cloud layers from the commercial sector
        • Engender user community, access to real applications/users
          • “ Standard” APIs (Amazon AWS, Google App Engine, MapReduce)
        • Leverage extant software technologies
      • To facilitate investigation of
        • Next-generation distributed / cloud computing software
          • Services, underlying device technology, support technologies
          • Customization (availability, performance, application behavior)
        • Hybrid cloud solutions (public and on-premise)
      • Not a replacement technology for any Public Cloud service
  • cloud computing fabrics from UCSB
    • Goal : Bring popular cloud fabrics to “on-premise” clusters that are easy to use and are transparent
      • By emulating key cloud layers from the commercial sector
        • Engender user community, access to real applications/users
          • “ Standard” APIs (Amazon AWS, Google App Engine, MapReduce)
        • Leverage extant software technologies
    • Platform-as-a-service (PaaS)
      • Open-source implementation of Google App Engine APIs
      • Pluggable (services), scalable, fault tolerant
      • Runs over virtualization or IaaS layer: AWS, Eucalyptus
      • appscale.cs.ucsb.edu
  • google app engine GAE Application (your code here) private, enterprise data, Google apps Images IM Memcache Mail Users URL Fetch Administrator Console Data Store Protobuf Data APIs SDC Google App Engine (GAE) Services Cron Tasks MyApp.appspot.com Blob store
  • sandbox restrictions GAE Application (your code here) Google App Engine (GAE) MyApp.appspot.com
        • Pure Python or Java only, white list of library calls to framework
        • No thread/subprocess spawning, system calls
        • No writes to file system, reads only to static files uploaded w/app
        • Storage using key-value, schema-free datastore (Bigtable-based)
        • HTTP/S communication only, CGI to handle page requests
        • Limit on number of datastore elements accessed per request
        • Limit on response duration, task frequency, request rate
        • Enforced quotas (BW, CPU, requests/s, files, app size, …)
  • google app engine: upload to google GAE Application (Python, Java) Google App Engine (GAE) SDC Administrator Console Free w/ quotas Pay for additional scale: CPU, BW, emails, data BigTable: Automatic scaling High availability … GAE app users via the Internet MyApp.appspot.com private, enterprise data, Google apps Images IM Mem Cache Mail Users URL Fetch Cron Tasks Data store Blob Store
  • from gae to appscale GAE Application (your code here) Open-source Google App Engine Software Development Kit (SDK) AppScale Images IM Mem Cache Mail Users URL Fetch Cron Tasks Data store Blob Store
  • from gae to appscale
    • GAE SDK extensions
      • Pluggable using open-source distributed database technologies
        • HBase, Hypertable, Cassandra, Voldemort, MongoDB, MemcacheDB, MySQL
      • MemcacheD library
      • From console or as background thread (automatically)
        • Interface to Hadoop (MapReduce)
          • Multi-language support: Python, Java, Ruby, Perl, soon: X10
      • Translator to Linux Cron job, similar to Tasks
      • Pluggable: built-in cloud-wide authentication via Rails, support for Eucalyptus and EC2 credentials
    Mem Cache Users Cron Tasks Data store
  • appscale’s offerings
    • Use of the Google APIs as a unifying interface
      • Datastore/database backends, services
        • Can grow this to include other standard APIs
    • Single framework that executes extant GAE applications
      • Disparate API implementations automatically deploy/compare
    • No restrictions on resource usage, library support
      • Multiple high-level language front-ends and components
      • Optimized library support in the back-end
  • appscale’s offerings
    • Use of the Google APIs as a unifying interface
      • Datastore/database backends, services
        • Can grow this to include other standard APIs
    • Single framework that executes extant GAE applications
      • Disparate API implementations automatically deploy/compare
    • No restrictions on resource usage, library support
      • Multiple high-level language front-ends and components
      • Optimized library support in the back-end
    • Open-source international user community
    • Potential for integration with infrastructure level (IaaS)
      • Adaptive SLA re-negotation
      • Growth/Shrinkage of the cloud
      • Application or workload specific customization of the platform
  • appscale
    • Distributed system with four key components
    • AppLoadBalancer (ALB) Database Master/Peer (DB M/P)
    • AppServer (AS) Database Slave/Peer (DB S/P)
    • Automatic deployment over Eucalyptus and EC2
      • Released as a single Xen or KVM image
      • Adaptive role configuration
    GAE App Developer (AppScale Admin) GAE App Users HTTPS App Controller ALB DB S/P AS GAE App Users GAE App Users AppScale Cloud Compute tasks (e.g. Map Reduce) DB M/P AppScale tools
  • appscale team foci
    • Full system monitoring (low-overhead HPM/OS & profiling)
      • Feedback driven adaptation, grow/shrink the cloud
        • Online workload characterization, behavior pattern recognition
        • Identify critical tasks, applications, components
    • Shared-memory support for co-located components
    • PaaS customization, PaaS-IaaS coordination, adaptation
    GAE App Developer (AppScale Admin) GAE App Users HTTPS App Controller ALB DB S/P AS GAE App Users GAE App Users AppScale Cloud Compute tasks (e.g. Map Reduce) DB M/P AppScale tools
  • google app engine GAE Application (your code here) private, enterprise data, Google apps Images IM Memcache Mail Users URL Fetch Administrator Console Data Store Protobuf Data APIs SDC Google App Engine (GAE) Services Cron Tasks MyApp.appspot.com Blob store
  • customized platform-as-a-service GAE, HPC, Data Intensive Application private, enterprise data, other apps public cloud fabrics GPU Map Reduce Collaboration Vector support Auth MPI Administrator Console Data Store Protobuf Data APIs SDC
    • Specialized platform access to
    • standard APIs
    • Platforms for other app domains
    Services Stream support Tasks MyApp.your.url.gov Custom Libs Messaging
  • appscale and RDMA
    • Shared-memory support for co-located components
    • PaaS-IaaS coordination, adaptation, scheduling
    • Compute/data intensive workloads (HPC, messages, streams)
    • Database system support for different workloads
    • Programmatic interface and tools for applications developers
      • Improve ease of use access
    GAE App Developer (AppScale Admin) GAE App Users HTTPS App Controller ALB DB S/P AS GAE App Users GAE App Users AppScale Cloud Compute tasks (e.g. Map Reduce) DB M/P AppScale tools
  • appscale http://appscale.cs.ucsb.edu
    • Thanks!
      • Leads: Chris Bunch, Navraj Chohan
      • Development and research team: Jovan Chohan, Nupur Garg, Matt Hubert, Jonathan Kupferman, Puneet Lakhina, Yiming Li, Nagy Mostafa, Yoshihide Nomura (Fujitsu), Michal Weigel
      • Support
        • Google, IBM Research, National Science Foundation
      • OpenFabrics Alliance