Designing the Call of Cthulhu app with Google App Engine
Upcoming SlideShare
Loading in...5

Designing the Call of Cthulhu app with Google App Engine



These are slides from a talk I gave at UCSB to the Senior Capstone class on 02/10/10 on how I developed the Call of Cthulhu application using Google App Engine.

These are slides from a talk I gave at UCSB to the Senior Capstone class on 02/10/10 on how I developed the Call of Cthulhu application using Google App Engine.



Total Views
Views on SlideShare
Embed Views



2 Embeds 37 21 16



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Designing the Call of Cthulhu app with Google App Engine Designing the Call of Cthulhu app with Google App Engine Presentation Transcript

  • Designing the Call of Cthulhu app with Google App Engine
    • Presented by Chris Bunch at UCSB
    • CS 189A / 172 : Capstone Project / Software Engineering
    • February 10, 2010
  • Motivation
    • I am what you would call a “nerd”
    • Plays nerdy board games
    • Would like to upload stats from games and analyze them
    • Could do other cool things as well
  • Specifically
  • Arkham Horror
    • Lots of variables that can be analyzed
      • Co-op game: all v. the board
      • Mix and match expansions
      • Many different winning / losing conditions
    • But fundamentally boils down to:
      • Save the world from total destruction
  • App Requirements
    • Must be able to upload game data
    • Should be able to predict a win or loss given some preliminary info
      • Can experiment with prediction algorithms
    • Should be able to summarize game data
  • App Requirements
    • Allows users to log in to upload their data
    • Allows users to share data with each other
    • Handles users in a sane fashion
      • Harder than it sounds
    • Display a top score list for users to view
      • Game provides scoring metric
  • Enter the Cloud
    • Why go the cloud route?
      • Simple answer: You likely don’t have your own hardware to host this app on, so just use somebody else’s
    • But cloud computing is confusing and not well defined yet, so...
  • Defining Cloud Computing
    • Three layers, each building on the last
    • Infrastructure:
      • Programmer gets a virtual machine and / or reliable storage
      • Most control of environment
      • Must scale manually
      • Amazon EC2, Eucalyptus
  • Defining Cloud Computing
    • Three layers, each building on the last
    • Platform:
      • Programmer gets a scalable API
      • Less control of environment
      • Scaling done automatically
      • Google App Engine, Microsoft Azure
  • Defining Cloud Computing
    • Three layers, each building on the last
    • Software:
      • Programmer gets to “skin” an existing app
      • No control of environment
      • Scaling done automatically
      • SalesForce, GMail
  • My Choice
    • Clearly I want the platform!
    • Platform:
      • Scalable API = great!
      • Less control of environment = ok
      • Programability depends on APIs offered
      • Have personal exp. w/ Google App Engine
  • What is Google App Engine?
    • Scalable web framework first offered in 2008
    • Users program apps in:
      • Python
      • Java
      • Unofficially: JRuby, Groovy, JavaScript
  • Sandboxed Environment
    • Requests limited to 30 seconds
    • Only white-listed libraries can be used
    • Anything non-trivial is quota-ed
    • No reading / writing to the filesystem
    • Web communication over HTTP(S) only
  • Datastore API
    • A giant hash table
    • Primitives:
    • Get, put, delete - same as regular HT
    • Query, count - new operations
    • Transactions supported (row-level only)
  • Datastore API
    • GQL queries offered - subset of SQL Select statement, no JOINs
    • Tip: Queries are the most expensive operation, use sparingly
    • If possible, perform it asynchronously
  • Memcache API
    • Equivalent to memcached
    • Provides access to low latency LRU cache
    • Roughly 100x faster than Datastore
      • But unreliable - remember it’s a cache!
    • Exposes get, set, delete operations
    • Exposes atomic increment / decrement
  • URL Fetch API
    • Equivalent to curl
    • Used to grab web content from other sites
      • Only allows for HTTP(S) data to be read
    • Can also be done asynchronously
    • Typically grabs images or mashup data
  • Mail API
    • Equivalent to sendmail
    • Can send or receive mail asynchronously
    • Can also add attachments
      • Not all types allowed - no .doc, .xls, or .ppt in Python SDK
    • Equivalent to jabber
    • Send and receive instant messages to
    • App can’t join group chat yet
    • Use as a notifier or alternate UI to app
  • Images API
    • Equivalent to python-imaging
    • Exposes image rotation, flipping, cropping, and histogram
    • Image is stored internally as a blob, then retrieved and manipulated
  • Users API
    • Equivalent to Google Accounts
    • Provides for simple authentication
    • Allows app to force login or admin on certain pages
    • Also exposes URL to optionally login to
  • Blobstore API
    • Allows for large file uploads (<50 MB)
    • Users can upload videos, datasets, or large music files for later retrieval
    • Upload must be done via form post
  • Cron API
    • Equivalent to cron
    • User specifies how often a web page should be accessed (e.g., daily, every Monday)
    • Task is then run in the background
    • Useful for running expensive data analysis
  • Task Queue API
    • Spawns a new thread in the background
      • But still must visit your app
    • Configuration file specifies queues
    • Useful for short to medium length tasks
  • To summarize:
    • Many, many APIs available for what we need to do!
    • Use Datastore to store / retrieve user data
    • Use Users to authenticate users
      • Gets around a lot of boilerplate authentication code
  • To summarize:
    • Use Mail to e-mail users if their score is beat
    • Use XMPP to allow users to add new data
    • Use Cron to run compute-intensive code in the background
  • To summarize:
    • Think about paid functionality:
      • Maybe use Blobstore to allow users to upload videos explaining gameplay
      • Handle greater traffic if site becomes popular
  • Let’s Get to Business
    • Data Modeling
    • Upload Stats Page
    • Game Stats Page
    • Game Prediction Page
  • Data Modeling
    • class Game(db.Model):
    • player = db.UserProperty()
    • goo = db.StringProperty()
    • comments = dbStringProperty(multiline=True)
    • date = db.DateTimeProperty(auto_now_add=True)
  • Data Modeling
    • num_of_players = db.IntegerProperty()
    • won_game = db.BooleanProperty()
    • doom_counters = db.IntegerProperty()
    • terror_level = db.IntegerProperty()
    • expansions_used = dbListProperty(str)
  • Uploading Stats
    • class AddGames(webapp.RequestHandler):
    • def get(self):
    • user = users.get_current_user()
    • if user: # means they’re logged in
    • # write html form for getting data
    • else:
    • self.redirect(users.create_login_url(self.request.uri))
  • Uploading Stats
    • def post(self):
    • new_goo = self.request.get(‘goo’)
    • # validate goo, make sure data type is right
    • game = Game()
    • = new_goo
    • game.put()
  • Game Stats
    • games = db.GqlQuery (“SELECT * FROM Game ORDER BY date desc”)
    • for game in games:
    • # find the goo w/ highest player win percentage
    • # can also just display the info here
  • Game Prediction
    • def get(self):
    • # user specifies the game they will play
    • def post(self):
    • # calculate how likely they are to win
    • # right now we only care about the goo
  • Routing
    • application = webapp.WSGIApplication([
    • (‘/upload’, AddGames),
    • (‘/stats’, GameStats),
    • (‘/predict’, PredictGame)])
  • Thankfully
    • Most requirements have very little state shared between URLs (all saved in DB)
    • Thus each class is one of two forms:
      • get / post: Grab data from user and do something with it
      • get: Just display or synthesize some data
  • Rendering Web Pages
    • Two main options available:
    • self.response.out.write(“Hello World!”)
    • self.response.out.write(template.render(‘index.html’, {‘my_title: ‘baz’, ‘data’: some_variable}))
    • Use the second for any non-trivial app
  • Rendering Web Pages
    • Now in your index.html:
    • <title> {{ my_title }} </title>
    • <div> {{ data }} </div>
  • Tips for Your Programs
    • Separate out your code into files as needed
    • app.yaml specifies this:
    • handlers:
    • - url: /one
    • script:
    • - url: /two/(.*)/(.*)
    • script: /two/2/
  • Tips for Your Programs
    • Make pages with sensitive data ‘secure’
      • All access done over HTTPS
    • Optional - HTTP and HTTPS work
    • Never - HTTPS redirects to HTTP
    • Always - HTTP redirects to HTTPS
  • Tips for Your Programs
    • Specify static files in app.yaml
    • - url: /images
    • static_dir: static/images
    • Can also tag ‘secure’ if desired
  • Tips for Your Programs
    • Be creative!
    • The webapp framework supports:
      • get, post, head, options, put, delete, trace
    • Map user pages to ~name?
  • Summary
    • Google App Engine is more than flexible enough to handle your web app desires
    • And it’s only getting better!
    • More than enough possibilities to create a web app: you just need a good idea!