AppScale @ LA.rb

  • 1,042 views
Uploaded on

These are my slides from my talk at LA.rb, covering research at UCSB on the AppScale project. This is a condensed version of the talk I gave at SBonRails - see that talk for about twice as much …

These are my slides from my talk at LA.rb, covering research at UCSB on the AppScale project. This is a condensed version of the talk I gave at SBonRails - see that talk for about twice as much material on these topics.

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,042
On Slideshare
0
From Embeds
0
Number of Embeds
2

Actions

Shares
Downloads
20
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. The AppScale Project Presented by Chris Bunch (on behalf of the AppScale team) April 14, 2011 @ LA.rb
  • 2. Overview• Google App Engine• AppScale - now with 50% Ruby!• Neptune - A Ruby DSL for the cloud
  • 3. Google App Engine• A web framework introduced in 2008• Python and Java supported• Offers a Platform-as-a-Service: Use Google’s APIs to achieve scale • Upload your app to Google
  • 4. Quotas
  • 5. Data Model• Not relational - semi-structured schema• Compared to models in Rails• Exposes a get / put / delete / query interface
  • 6. Storing Data• Datastore API - Persistent storage• Memcache API - Transient storage • User can set expiration times• Blobstore API - Store large files • need to enable billing to use it
  • 7. Be Social!• Mail API - Send and receive e-mail• XMPP API - Send and receive IMs• Channel API - Creating persistent connections via XMPP • Use for chat rooms, games, etc.
  • 8. Background Tasks• Cron API - Access a URL periodically • Descriptive language: “every 5 minutes”, “every 1st Sun of Jan, Mar, Dec”, etc. • Uses a separate cron.yaml file• TaskQueue API - Within your app, fire off tasks to be done later
  • 9. Dealing with Users• Users API: Uses Google Accounts • Don’t write that ‘forgot password’ page ever again!• Authorization: via app.yaml: • anyone, must login, or admin only
  • 10. When Services Fail• Originally: failures throw exceptions • Just catch them all!• Capabilities API: Check if a service is available • Datastore, Memcache, and so on
  • 11. Deploying Your App• Develop locally on SDK • Stub implementations of most APIs• Then deploy to Google
  • 12. How to Scale• Limitations on the programming model: • No filesystem interaction • 30 second limit per web request • Language libraries must be on whitelist • Sandboxed execution
  • 13. Enter AppScale• App Engine is easy to use • but we really want to tinker with the internals!• Need an open platform to experiment on • test API implementations • add new APIs
  • 14. Enter AppScale• Lots of NoSQL DBs out there • Hard to compare DBs• Configuration and deployment can be complex• Need one-button deployment
  • 15. Deploying Your App• run-instances: Start AppScale• describe-instances:View cloud metadata• upload-app: Deploy an App Engine app• remove-app: Un-deploy an App Engine app• terminate-instances: Stop AppScale
  • 16. Deployment Models• Cloud deployment: Amazon EC2 or Eucalyptus (the open source implementation of the EC2 APIs) • Just specify how many machines you need• Non-cloud deployment via Xen or KVM
  • 17. AppController• The brains of the outfit• Runs on every node• Handles configuration and deployment of all services (including other AppControllers)• Written in Ruby
  • 18. Load balancer• Routes users to their app via nginx• haproxy makes sure app servers are live• Can’t assume the user has DNS: • Thus we wrote the AppLoadBalancer • Rails app that routes users to apps • Performs authentication as well
  • 19. App Server• We modified the App Engine SDK• Easier for Python (source included)• Harder for Java (had to decompile)• Removed non-scalable API implementations • Goal: Use open source whenever possible
  • 20. A Common Feature Request
  • 21. Database Options• Open source / open APIs / proprietary• Master / slave v. peer-to-peer• Differences in query languages• Data model (key/val, semi-structured)• In-memory or persistent• Data consistency model• Interfaces - REST / Thrift / libraries
  • 22. Neptune• Need a simple way to run compute-intensive jobs• We have the code from the ‘net• We have the resources - the cloud• But the average user does not have the know how• Our solution: create a domain specific language for configuring cloud apps• Based on Ruby
  • 23. Syntax• It’s as easy as: neptune :type => “mpi”, :code => “MpiNQueens”, :nodes_to_use => 8, :output => “/mpi/output-1.txt”
  • 24. Neptune Supports:• Message Passing Interface (MPI)• MapReduce• Unified Parallel C (UPC)• X10• Erlang
  • 25. Extensibility• Experts can add support for other computational jobs• Biochemists can run simulations via DFSP and dwSSA • Embarassingly parallel Monte Carlo simulations
  • 26. Wrapping It Up• Thanks to the AppScale team, especially: • Co-lead Navraj Chohan and advisor Chandra Krintz• Check us out on the web: • http://appscale.cs.ucsb.edu • http://neptune-lang.org