• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
AppScale @ LA.rb
 

AppScale @ LA.rb

on

  • 1,214 views

These are my slides from my talk at LA.rb, covering research at UCSB on the AppScale project. This is a condensed version of the talk I gave at SBonRails - see that talk for about twice as much ...

These are my slides from my talk at LA.rb, covering research at UCSB on the AppScale project. This is a condensed version of the talk I gave at SBonRails - see that talk for about twice as much material on these topics.

Statistics

Views

Total Views
1,214
Views on SlideShare
1,206
Embed Views
8

Actions

Likes
1
Downloads
19
Comments
0

2 Embeds 8

http://www.byzantinereality.com 6
http://localhost 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    AppScale @ LA.rb AppScale @ LA.rb Presentation Transcript

    • The AppScale Project Presented by Chris Bunch (on behalf of the AppScale team) April 14, 2011 @ LA.rb
    • Overview• Google App Engine• AppScale - now with 50% Ruby!• Neptune - A Ruby DSL for the cloud
    • Google App Engine• A web framework introduced in 2008• Python and Java supported• Offers a Platform-as-a-Service: Use Google’s APIs to achieve scale • Upload your app to Google
    • Quotas
    • Data Model• Not relational - semi-structured schema• Compared to models in Rails• Exposes a get / put / delete / query interface
    • Storing Data• Datastore API - Persistent storage• Memcache API - Transient storage • User can set expiration times• Blobstore API - Store large files • need to enable billing to use it
    • Be Social!• Mail API - Send and receive e-mail• XMPP API - Send and receive IMs• Channel API - Creating persistent connections via XMPP • Use for chat rooms, games, etc.
    • Background Tasks• Cron API - Access a URL periodically • Descriptive language: “every 5 minutes”, “every 1st Sun of Jan, Mar, Dec”, etc. • Uses a separate cron.yaml file• TaskQueue API - Within your app, fire off tasks to be done later
    • Dealing with Users• Users API: Uses Google Accounts • Don’t write that ‘forgot password’ page ever again!• Authorization: via app.yaml: • anyone, must login, or admin only
    • When Services Fail• Originally: failures throw exceptions • Just catch them all!• Capabilities API: Check if a service is available • Datastore, Memcache, and so on
    • Deploying Your App• Develop locally on SDK • Stub implementations of most APIs• Then deploy to Google
    • How to Scale• Limitations on the programming model: • No filesystem interaction • 30 second limit per web request • Language libraries must be on whitelist • Sandboxed execution
    • Enter AppScale• App Engine is easy to use • but we really want to tinker with the internals!• Need an open platform to experiment on • test API implementations • add new APIs
    • Enter AppScale• Lots of NoSQL DBs out there • Hard to compare DBs• Configuration and deployment can be complex• Need one-button deployment
    • Deploying Your App• run-instances: Start AppScale• describe-instances:View cloud metadata• upload-app: Deploy an App Engine app• remove-app: Un-deploy an App Engine app• terminate-instances: Stop AppScale
    • Deployment Models• Cloud deployment: Amazon EC2 or Eucalyptus (the open source implementation of the EC2 APIs) • Just specify how many machines you need• Non-cloud deployment via Xen or KVM
    • AppController• The brains of the outfit• Runs on every node• Handles configuration and deployment of all services (including other AppControllers)• Written in Ruby
    • Load balancer• Routes users to their app via nginx• haproxy makes sure app servers are live• Can’t assume the user has DNS: • Thus we wrote the AppLoadBalancer • Rails app that routes users to apps • Performs authentication as well
    • App Server• We modified the App Engine SDK• Easier for Python (source included)• Harder for Java (had to decompile)• Removed non-scalable API implementations • Goal: Use open source whenever possible
    • A Common Feature Request
    • Database Options• Open source / open APIs / proprietary• Master / slave v. peer-to-peer• Differences in query languages• Data model (key/val, semi-structured)• In-memory or persistent• Data consistency model• Interfaces - REST / Thrift / libraries
    • Neptune• Need a simple way to run compute-intensive jobs• We have the code from the ‘net• We have the resources - the cloud• But the average user does not have the know how• Our solution: create a domain specific language for configuring cloud apps• Based on Ruby
    • Syntax• It’s as easy as: neptune :type => “mpi”, :code => “MpiNQueens”, :nodes_to_use => 8, :output => “/mpi/output-1.txt”
    • Neptune Supports:• Message Passing Interface (MPI)• MapReduce• Unified Parallel C (UPC)• X10• Erlang
    • Extensibility• Experts can add support for other computational jobs• Biochemists can run simulations via DFSP and dwSSA • Embarassingly parallel Monte Carlo simulations
    • Wrapping It Up• Thanks to the AppScale team, especially: • Co-lead Navraj Chohan and advisor Chandra Krintz• Check us out on the web: • http://appscale.cs.ucsb.edu • http://neptune-lang.org