AppScale @ LA.rb
Upcoming SlideShare
Loading in...5

AppScale @ LA.rb



These are my slides from my talk at LA.rb, covering research at UCSB on the AppScale project. This is a condensed version of the talk I gave at SBonRails - see that talk for about twice as much ...

These are my slides from my talk at LA.rb, covering research at UCSB on the AppScale project. This is a condensed version of the talk I gave at SBonRails - see that talk for about twice as much material on these topics.



Total Views
Views on SlideShare
Embed Views



2 Embeds 8 6
http://localhost 2



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

AppScale @ LA.rb AppScale @ LA.rb Presentation Transcript

  • The AppScale Project Presented by Chris Bunch (on behalf of the AppScale team) April 14, 2011 @ LA.rb
  • Overview• Google App Engine• AppScale - now with 50% Ruby!• Neptune - A Ruby DSL for the cloud
  • Google App Engine• A web framework introduced in 2008• Python and Java supported• Offers a Platform-as-a-Service: Use Google’s APIs to achieve scale • Upload your app to Google
  • Quotas
  • Data Model• Not relational - semi-structured schema• Compared to models in Rails• Exposes a get / put / delete / query interface
  • Storing Data• Datastore API - Persistent storage• Memcache API - Transient storage • User can set expiration times• Blobstore API - Store large files • need to enable billing to use it
  • Be Social!• Mail API - Send and receive e-mail• XMPP API - Send and receive IMs• Channel API - Creating persistent connections via XMPP • Use for chat rooms, games, etc.
  • Background Tasks• Cron API - Access a URL periodically • Descriptive language: “every 5 minutes”, “every 1st Sun of Jan, Mar, Dec”, etc. • Uses a separate cron.yaml file• TaskQueue API - Within your app, fire off tasks to be done later
  • Dealing with Users• Users API: Uses Google Accounts • Don’t write that ‘forgot password’ page ever again!• Authorization: via app.yaml: • anyone, must login, or admin only
  • When Services Fail• Originally: failures throw exceptions • Just catch them all!• Capabilities API: Check if a service is available • Datastore, Memcache, and so on
  • Deploying Your App• Develop locally on SDK • Stub implementations of most APIs• Then deploy to Google
  • How to Scale• Limitations on the programming model: • No filesystem interaction • 30 second limit per web request • Language libraries must be on whitelist • Sandboxed execution
  • Enter AppScale• App Engine is easy to use • but we really want to tinker with the internals!• Need an open platform to experiment on • test API implementations • add new APIs
  • Enter AppScale• Lots of NoSQL DBs out there • Hard to compare DBs• Configuration and deployment can be complex• Need one-button deployment
  • Deploying Your App• run-instances: Start AppScale• describe-instances:View cloud metadata• upload-app: Deploy an App Engine app• remove-app: Un-deploy an App Engine app• terminate-instances: Stop AppScale
  • Deployment Models• Cloud deployment: Amazon EC2 or Eucalyptus (the open source implementation of the EC2 APIs) • Just specify how many machines you need• Non-cloud deployment via Xen or KVM
  • AppController• The brains of the outfit• Runs on every node• Handles configuration and deployment of all services (including other AppControllers)• Written in Ruby
  • Load balancer• Routes users to their app via nginx• haproxy makes sure app servers are live• Can’t assume the user has DNS: • Thus we wrote the AppLoadBalancer • Rails app that routes users to apps • Performs authentication as well
  • App Server• We modified the App Engine SDK• Easier for Python (source included)• Harder for Java (had to decompile)• Removed non-scalable API implementations • Goal: Use open source whenever possible
  • A Common Feature Request
  • Database Options• Open source / open APIs / proprietary• Master / slave v. peer-to-peer• Differences in query languages• Data model (key/val, semi-structured)• In-memory or persistent• Data consistency model• Interfaces - REST / Thrift / libraries
  • Neptune• Need a simple way to run compute-intensive jobs• We have the code from the ‘net• We have the resources - the cloud• But the average user does not have the know how• Our solution: create a domain specific language for configuring cloud apps• Based on Ruby
  • Syntax• It’s as easy as: neptune :type => “mpi”, :code => “MpiNQueens”, :nodes_to_use => 8, :output => “/mpi/output-1.txt”
  • Neptune Supports:• Message Passing Interface (MPI)• MapReduce• Unified Parallel C (UPC)• X10• Erlang
  • Extensibility• Experts can add support for other computational jobs• Biochemists can run simulations via DFSP and dwSSA • Embarassingly parallel Monte Carlo simulations
  • Wrapping It Up• Thanks to the AppScale team, especially: • Co-lead Navraj Chohan and advisor Chandra Krintz• Check us out on the web: • •