Your SlideShare is downloading. ×
AppScale @ LA.rb
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

AppScale @ LA.rb


Published on

These are my slides from my talk at LA.rb, covering research at UCSB on the AppScale project. This is a condensed version of the talk I gave at SBonRails - see that talk for about twice as much …

These are my slides from my talk at LA.rb, covering research at UCSB on the AppScale project. This is a condensed version of the talk I gave at SBonRails - see that talk for about twice as much material on these topics.

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. The AppScale Project Presented by Chris Bunch (on behalf of the AppScale team) April 14, 2011 @ LA.rb
  • 2. Overview• Google App Engine• AppScale - now with 50% Ruby!• Neptune - A Ruby DSL for the cloud
  • 3. Google App Engine• A web framework introduced in 2008• Python and Java supported• Offers a Platform-as-a-Service: Use Google’s APIs to achieve scale • Upload your app to Google
  • 4. Quotas
  • 5. Data Model• Not relational - semi-structured schema• Compared to models in Rails• Exposes a get / put / delete / query interface
  • 6. Storing Data• Datastore API - Persistent storage• Memcache API - Transient storage • User can set expiration times• Blobstore API - Store large files • need to enable billing to use it
  • 7. Be Social!• Mail API - Send and receive e-mail• XMPP API - Send and receive IMs• Channel API - Creating persistent connections via XMPP • Use for chat rooms, games, etc.
  • 8. Background Tasks• Cron API - Access a URL periodically • Descriptive language: “every 5 minutes”, “every 1st Sun of Jan, Mar, Dec”, etc. • Uses a separate cron.yaml file• TaskQueue API - Within your app, fire off tasks to be done later
  • 9. Dealing with Users• Users API: Uses Google Accounts • Don’t write that ‘forgot password’ page ever again!• Authorization: via app.yaml: • anyone, must login, or admin only
  • 10. When Services Fail• Originally: failures throw exceptions • Just catch them all!• Capabilities API: Check if a service is available • Datastore, Memcache, and so on
  • 11. Deploying Your App• Develop locally on SDK • Stub implementations of most APIs• Then deploy to Google
  • 12. How to Scale• Limitations on the programming model: • No filesystem interaction • 30 second limit per web request • Language libraries must be on whitelist • Sandboxed execution
  • 13. Enter AppScale• App Engine is easy to use • but we really want to tinker with the internals!• Need an open platform to experiment on • test API implementations • add new APIs
  • 14. Enter AppScale• Lots of NoSQL DBs out there • Hard to compare DBs• Configuration and deployment can be complex• Need one-button deployment
  • 15. Deploying Your App• run-instances: Start AppScale• describe-instances:View cloud metadata• upload-app: Deploy an App Engine app• remove-app: Un-deploy an App Engine app• terminate-instances: Stop AppScale
  • 16. Deployment Models• Cloud deployment: Amazon EC2 or Eucalyptus (the open source implementation of the EC2 APIs) • Just specify how many machines you need• Non-cloud deployment via Xen or KVM
  • 17. AppController• The brains of the outfit• Runs on every node• Handles configuration and deployment of all services (including other AppControllers)• Written in Ruby
  • 18. Load balancer• Routes users to their app via nginx• haproxy makes sure app servers are live• Can’t assume the user has DNS: • Thus we wrote the AppLoadBalancer • Rails app that routes users to apps • Performs authentication as well
  • 19. App Server• We modified the App Engine SDK• Easier for Python (source included)• Harder for Java (had to decompile)• Removed non-scalable API implementations • Goal: Use open source whenever possible
  • 20. A Common Feature Request
  • 21. Database Options• Open source / open APIs / proprietary• Master / slave v. peer-to-peer• Differences in query languages• Data model (key/val, semi-structured)• In-memory or persistent• Data consistency model• Interfaces - REST / Thrift / libraries
  • 22. Neptune• Need a simple way to run compute-intensive jobs• We have the code from the ‘net• We have the resources - the cloud• But the average user does not have the know how• Our solution: create a domain specific language for configuring cloud apps• Based on Ruby
  • 23. Syntax• It’s as easy as: neptune :type => “mpi”, :code => “MpiNQueens”, :nodes_to_use => 8, :output => “/mpi/output-1.txt”
  • 24. Neptune Supports:• Message Passing Interface (MPI)• MapReduce• Unified Parallel C (UPC)• X10• Erlang
  • 25. Extensibility• Experts can add support for other computational jobs• Biochemists can run simulations via DFSP and dwSSA • Embarassingly parallel Monte Carlo simulations
  • 26. Wrapping It Up• Thanks to the AppScale team, especially: • Co-lead Navraj Chohan and advisor Chandra Krintz• Check us out on the web: • •