Designing the Call of Cthulhu app with Google App Engine <ul><li>Presented by Chris Bunch at UCSB </li></ul><ul><li>CS 189...
Motivation <ul><li>I am what you would call a “nerd” </li></ul><ul><li>Plays nerdy board games </li></ul><ul><li>Would lik...
Specifically
Arkham Horror <ul><li>Lots of variables that can be analyzed </li></ul><ul><ul><li>Co-op game: all v. the board </li></ul>...
App Requirements <ul><li>Must be able to upload game data </li></ul><ul><li>Should be able to predict a win or loss given ...
App Requirements <ul><li>Allows users to log in to upload their data </li></ul><ul><li>Allows users to share data with eac...
Enter the Cloud <ul><li>Why go the cloud route? </li></ul><ul><ul><li>Simple answer: You likely don’t have your own hardwa...
Defining Cloud Computing <ul><li>Three layers, each building on the last </li></ul><ul><li>Infrastructure: </li></ul><ul><...
Defining Cloud Computing <ul><li>Three layers, each building on the last </li></ul><ul><li>Platform: </li></ul><ul><ul><li...
Defining Cloud Computing <ul><li>Three layers, each building on the last </li></ul><ul><li>Software: </li></ul><ul><ul><li...
My Choice <ul><li>Clearly I want the platform! </li></ul><ul><li>Platform: </li></ul><ul><ul><li>Scalable API = great! </l...
What is Google App Engine? <ul><li>Scalable web framework first offered in 2008 </li></ul><ul><li>Users program apps in: <...
Sandboxed Environment <ul><li>Requests limited to 30 seconds </li></ul><ul><li>Only white-listed libraries can be used </l...
Datastore API <ul><li>A giant hash table </li></ul><ul><li>Primitives: </li></ul><ul><li>Get, put, delete - same as regula...
Datastore API <ul><li>GQL queries offered - subset of SQL Select statement, no JOINs </li></ul><ul><li>Tip: Queries are th...
Memcache API <ul><li>Equivalent to memcached </li></ul><ul><li>Provides access to low latency LRU cache </li></ul><ul><li>...
URL Fetch API <ul><li>Equivalent to curl </li></ul><ul><li>Used to grab web content from other sites </li></ul><ul><ul><li...
Mail API <ul><li>Equivalent to sendmail </li></ul><ul><li>Can send or receive mail asynchronously </li></ul><ul><li>Can al...
XMPP API <ul><li>Equivalent to jabber </li></ul><ul><li>Send and receive instant messages to </li></ul><ul><ul><li>appname...
Images API <ul><li>Equivalent to python-imaging </li></ul><ul><li>Exposes image rotation, flipping, cropping, and histogra...
Users API <ul><li>Equivalent to Google Accounts </li></ul><ul><li>Provides for simple authentication </li></ul><ul><li>All...
Blobstore API <ul><li>Allows for large file uploads (<50 MB) </li></ul><ul><li>Users can upload videos, datasets, or large...
Cron API <ul><li>Equivalent to cron </li></ul><ul><li>User specifies how often a web page should be accessed (e.g., daily,...
Task Queue API <ul><li>Spawns a new thread in the background </li></ul><ul><ul><li>But still must visit your app </li></ul...
To summarize: <ul><li>Many, many APIs available for what we need to do! </li></ul><ul><li>Use Datastore to store / retriev...
To summarize: <ul><li>Use Mail to e-mail users if their score is beat </li></ul><ul><li>Use XMPP to allow users to add new...
To summarize: <ul><li>Think about paid functionality: </li></ul><ul><ul><li>Maybe use Blobstore to allow users to upload v...
Let’s Get to Business <ul><li>Data Modeling </li></ul><ul><li>Upload Stats Page </li></ul><ul><li>Game Stats Page </li></u...
Data Modeling <ul><li>class Game(db.Model): </li></ul><ul><li>player = db.UserProperty() </li></ul><ul><li>goo = db.String...
Data Modeling <ul><li>num_of_players = db.IntegerProperty() </li></ul><ul><li>won_game = db.BooleanProperty() </li></ul><u...
Uploading Stats <ul><li>class AddGames(webapp.RequestHandler): </li></ul><ul><li>def get(self): </li></ul><ul><li>user = u...
Uploading Stats <ul><li>def post(self): </li></ul><ul><li>new_goo = self.request.get(‘goo’) </li></ul><ul><li># validate g...
Game Stats <ul><li>games = db.GqlQuery (“SELECT * FROM Game ORDER BY date desc”) </li></ul><ul><li>for game in games: </li...
Game Prediction <ul><li>def get(self): </li></ul><ul><li># user specifies the game they will play </li></ul><ul><li>def po...
Routing <ul><li>application = webapp.WSGIApplication([ </li></ul><ul><li>(‘/upload’, AddGames), </li></ul><ul><li>(‘/stats...
Thankfully <ul><li>Most requirements have very little state shared between URLs (all saved in DB) </li></ul><ul><li>Thus e...
Rendering Web Pages <ul><li>Two main options available: </li></ul><ul><li>self.response.out.write(“Hello World!”) </li></u...
Rendering Web Pages <ul><li>Now in your index.html: </li></ul><ul><li><title> {{ my_title }} </title> </li></ul><ul><li><d...
Tips for Your Programs <ul><li>Separate out your code into files as needed </li></ul><ul><li>app.yaml specifies this: </li...
Tips for Your Programs <ul><li>Make pages with sensitive data ‘secure’ </li></ul><ul><ul><li>All access done over HTTPS </...
Tips for Your Programs <ul><li>Specify static files in app.yaml </li></ul><ul><li>- url: /images </li></ul><ul><li>static_...
Tips for Your Programs <ul><li>Be creative! </li></ul><ul><li>The webapp framework supports: </li></ul><ul><ul><li>get, po...
Summary <ul><li>Google App Engine is more than flexible enough to handle your web app desires </li></ul><ul><li>And it’s o...
 
Upcoming SlideShare
Loading in...5
×

Designing the Call of Cthulhu app with Google App Engine

1,853

Published on

These are slides from a talk I gave at UCSB to the Senior Capstone class on 02/10/10 on how I developed the Call of Cthulhu application using Google App Engine.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,853
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Designing the Call of Cthulhu app with Google App Engine

  1. 1. Designing the Call of Cthulhu app with Google App Engine <ul><li>Presented by Chris Bunch at UCSB </li></ul><ul><li>CS 189A / 172 : Capstone Project / Software Engineering </li></ul><ul><li>February 10, 2010 </li></ul><ul><li>http://cs.ucsb.edu/~cgb </li></ul>
  2. 2. Motivation <ul><li>I am what you would call a “nerd” </li></ul><ul><li>Plays nerdy board games </li></ul><ul><li>Would like to upload stats from games and analyze them </li></ul><ul><li>Could do other cool things as well </li></ul>
  3. 3. Specifically
  4. 4. Arkham Horror <ul><li>Lots of variables that can be analyzed </li></ul><ul><ul><li>Co-op game: all v. the board </li></ul></ul><ul><ul><li>Mix and match expansions </li></ul></ul><ul><ul><li>Many different winning / losing conditions </li></ul></ul><ul><li>But fundamentally boils down to: </li></ul><ul><ul><li>Save the world from total destruction </li></ul></ul>
  5. 5. App Requirements <ul><li>Must be able to upload game data </li></ul><ul><li>Should be able to predict a win or loss given some preliminary info </li></ul><ul><ul><li>Can experiment with prediction algorithms </li></ul></ul><ul><li>Should be able to summarize game data </li></ul>
  6. 6. App Requirements <ul><li>Allows users to log in to upload their data </li></ul><ul><li>Allows users to share data with each other </li></ul><ul><li>Handles users in a sane fashion </li></ul><ul><ul><li>Harder than it sounds </li></ul></ul><ul><li>Display a top score list for users to view </li></ul><ul><ul><li>Game provides scoring metric </li></ul></ul>
  7. 7. Enter the Cloud <ul><li>Why go the cloud route? </li></ul><ul><ul><li>Simple answer: You likely don’t have your own hardware to host this app on, so just use somebody else’s </li></ul></ul><ul><li>But cloud computing is confusing and not well defined yet, so... </li></ul>
  8. 8. Defining Cloud Computing <ul><li>Three layers, each building on the last </li></ul><ul><li>Infrastructure: </li></ul><ul><ul><li>Programmer gets a virtual machine and / or reliable storage </li></ul></ul><ul><ul><li>Most control of environment </li></ul></ul><ul><ul><li>Must scale manually </li></ul></ul><ul><ul><li>Amazon EC2, Eucalyptus </li></ul></ul>
  9. 9. Defining Cloud Computing <ul><li>Three layers, each building on the last </li></ul><ul><li>Platform: </li></ul><ul><ul><li>Programmer gets a scalable API </li></ul></ul><ul><ul><li>Less control of environment </li></ul></ul><ul><ul><li>Scaling done automatically </li></ul></ul><ul><ul><li>Google App Engine, Microsoft Azure </li></ul></ul>
  10. 10. Defining Cloud Computing <ul><li>Three layers, each building on the last </li></ul><ul><li>Software: </li></ul><ul><ul><li>Programmer gets to “skin” an existing app </li></ul></ul><ul><ul><li>No control of environment </li></ul></ul><ul><ul><li>Scaling done automatically </li></ul></ul><ul><ul><li>SalesForce, GMail </li></ul></ul>
  11. 11. My Choice <ul><li>Clearly I want the platform! </li></ul><ul><li>Platform: </li></ul><ul><ul><li>Scalable API = great! </li></ul></ul><ul><ul><li>Less control of environment = ok </li></ul></ul><ul><ul><li>Programability depends on APIs offered </li></ul></ul><ul><ul><li>Have personal exp. w/ Google App Engine </li></ul></ul>
  12. 12. What is Google App Engine? <ul><li>Scalable web framework first offered in 2008 </li></ul><ul><li>Users program apps in: </li></ul><ul><ul><li>Python </li></ul></ul><ul><ul><li>Java </li></ul></ul><ul><ul><li>Unofficially: JRuby, Groovy, JavaScript </li></ul></ul>
  13. 13. Sandboxed Environment <ul><li>Requests limited to 30 seconds </li></ul><ul><li>Only white-listed libraries can be used </li></ul><ul><li>Anything non-trivial is quota-ed </li></ul><ul><li>No reading / writing to the filesystem </li></ul><ul><li>Web communication over HTTP(S) only </li></ul>
  14. 14. Datastore API <ul><li>A giant hash table </li></ul><ul><li>Primitives: </li></ul><ul><li>Get, put, delete - same as regular HT </li></ul><ul><li>Query, count - new operations </li></ul><ul><li>Transactions supported (row-level only) </li></ul>
  15. 15. Datastore API <ul><li>GQL queries offered - subset of SQL Select statement, no JOINs </li></ul><ul><li>Tip: Queries are the most expensive operation, use sparingly </li></ul><ul><li>If possible, perform it asynchronously </li></ul>
  16. 16. Memcache API <ul><li>Equivalent to memcached </li></ul><ul><li>Provides access to low latency LRU cache </li></ul><ul><li>Roughly 100x faster than Datastore </li></ul><ul><ul><li>But unreliable - remember it’s a cache! </li></ul></ul><ul><li>Exposes get, set, delete operations </li></ul><ul><li>Exposes atomic increment / decrement </li></ul>
  17. 17. URL Fetch API <ul><li>Equivalent to curl </li></ul><ul><li>Used to grab web content from other sites </li></ul><ul><ul><li>Only allows for HTTP(S) data to be read </li></ul></ul><ul><li>Can also be done asynchronously </li></ul><ul><li>Typically grabs images or mashup data </li></ul>
  18. 18. Mail API <ul><li>Equivalent to sendmail </li></ul><ul><li>Can send or receive mail asynchronously </li></ul><ul><li>Can also add attachments </li></ul><ul><ul><li>Not all types allowed - no .doc, .xls, or .ppt in Python SDK </li></ul></ul>
  19. 19. XMPP API <ul><li>Equivalent to jabber </li></ul><ul><li>Send and receive instant messages to </li></ul><ul><ul><li>appname.appspotchat.com </li></ul></ul><ul><li>App can’t join group chat yet </li></ul><ul><li>Use as a notifier or alternate UI to app </li></ul>
  20. 20. Images API <ul><li>Equivalent to python-imaging </li></ul><ul><li>Exposes image rotation, flipping, cropping, and histogram </li></ul><ul><li>Image is stored internally as a blob, then retrieved and manipulated </li></ul>
  21. 21. Users API <ul><li>Equivalent to Google Accounts </li></ul><ul><li>Provides for simple authentication </li></ul><ul><li>Allows app to force login or admin on certain pages </li></ul><ul><li>Also exposes URL to optionally login to </li></ul>
  22. 22. Blobstore API <ul><li>Allows for large file uploads (<50 MB) </li></ul><ul><li>Users can upload videos, datasets, or large music files for later retrieval </li></ul><ul><li>Upload must be done via form post </li></ul>
  23. 23. Cron API <ul><li>Equivalent to cron </li></ul><ul><li>User specifies how often a web page should be accessed (e.g., daily, every Monday) </li></ul><ul><li>Task is then run in the background </li></ul><ul><li>Useful for running expensive data analysis </li></ul>
  24. 24. Task Queue API <ul><li>Spawns a new thread in the background </li></ul><ul><ul><li>But still must visit your app </li></ul></ul><ul><li>Configuration file specifies queues </li></ul><ul><li>Useful for short to medium length tasks </li></ul>
  25. 25. To summarize: <ul><li>Many, many APIs available for what we need to do! </li></ul><ul><li>Use Datastore to store / retrieve user data </li></ul><ul><li>Use Users to authenticate users </li></ul><ul><ul><li>Gets around a lot of boilerplate authentication code </li></ul></ul>
  26. 26. To summarize: <ul><li>Use Mail to e-mail users if their score is beat </li></ul><ul><li>Use XMPP to allow users to add new data </li></ul><ul><li>Use Cron to run compute-intensive code in the background </li></ul>
  27. 27. To summarize: <ul><li>Think about paid functionality: </li></ul><ul><ul><li>Maybe use Blobstore to allow users to upload videos explaining gameplay </li></ul></ul><ul><ul><li>Handle greater traffic if site becomes popular </li></ul></ul>
  28. 28. Let’s Get to Business <ul><li>Data Modeling </li></ul><ul><li>Upload Stats Page </li></ul><ul><li>Game Stats Page </li></ul><ul><li>Game Prediction Page </li></ul>
  29. 29. Data Modeling <ul><li>class Game(db.Model): </li></ul><ul><li>player = db.UserProperty() </li></ul><ul><li>goo = db.StringProperty() </li></ul><ul><li>comments = dbStringProperty(multiline=True) </li></ul><ul><li>date = db.DateTimeProperty(auto_now_add=True) </li></ul>
  30. 30. Data Modeling <ul><li>num_of_players = db.IntegerProperty() </li></ul><ul><li>won_game = db.BooleanProperty() </li></ul><ul><li>doom_counters = db.IntegerProperty() </li></ul><ul><li>terror_level = db.IntegerProperty() </li></ul><ul><li>expansions_used = dbListProperty(str) </li></ul>
  31. 31. Uploading Stats <ul><li>class AddGames(webapp.RequestHandler): </li></ul><ul><li>def get(self): </li></ul><ul><li>user = users.get_current_user() </li></ul><ul><li>if user: # means they’re logged in </li></ul><ul><li># write html form for getting data </li></ul><ul><li>else: </li></ul><ul><li>self.redirect(users.create_login_url(self.request.uri)) </li></ul>
  32. 32. Uploading Stats <ul><li>def post(self): </li></ul><ul><li>new_goo = self.request.get(‘goo’) </li></ul><ul><li># validate goo, make sure data type is right </li></ul><ul><li>game = Game() </li></ul><ul><li>game.goo = new_goo </li></ul><ul><li>game.put() </li></ul>
  33. 33. Game Stats <ul><li>games = db.GqlQuery (“SELECT * FROM Game ORDER BY date desc”) </li></ul><ul><li>for game in games: </li></ul><ul><li># find the goo w/ highest player win percentage </li></ul><ul><li># can also just display the info here </li></ul>
  34. 34. Game Prediction <ul><li>def get(self): </li></ul><ul><li># user specifies the game they will play </li></ul><ul><li>def post(self): </li></ul><ul><li># calculate how likely they are to win </li></ul><ul><li># right now we only care about the goo </li></ul>
  35. 35. Routing <ul><li>application = webapp.WSGIApplication([ </li></ul><ul><li>(‘/upload’, AddGames), </li></ul><ul><li>(‘/stats’, GameStats), </li></ul><ul><li>(‘/predict’, PredictGame)]) </li></ul>
  36. 36. Thankfully <ul><li>Most requirements have very little state shared between URLs (all saved in DB) </li></ul><ul><li>Thus each class is one of two forms: </li></ul><ul><ul><li>get / post: Grab data from user and do something with it </li></ul></ul><ul><ul><li>get: Just display or synthesize some data </li></ul></ul>
  37. 37. Rendering Web Pages <ul><li>Two main options available: </li></ul><ul><li>self.response.out.write(“Hello World!”) </li></ul><ul><li>self.response.out.write(template.render(‘index.html’, {‘my_title: ‘baz’, ‘data’: some_variable})) </li></ul><ul><li>Use the second for any non-trivial app </li></ul>
  38. 38. Rendering Web Pages <ul><li>Now in your index.html: </li></ul><ul><li><title> {{ my_title }} </title> </li></ul><ul><li><div> {{ data }} </div> </li></ul>
  39. 39. Tips for Your Programs <ul><li>Separate out your code into files as needed </li></ul><ul><li>app.yaml specifies this: </li></ul><ul><li>handlers: </li></ul><ul><li>- url: /one </li></ul><ul><li>script: one.py </li></ul><ul><li>- url: /two/(.*)/(.*) </li></ul><ul><li>script: /two/2/1.py </li></ul>
  40. 40. Tips for Your Programs <ul><li>Make pages with sensitive data ‘secure’ </li></ul><ul><ul><li>All access done over HTTPS </li></ul></ul><ul><li>Optional - HTTP and HTTPS work </li></ul><ul><li>Never - HTTPS redirects to HTTP </li></ul><ul><li>Always - HTTP redirects to HTTPS </li></ul>
  41. 41. Tips for Your Programs <ul><li>Specify static files in app.yaml </li></ul><ul><li>- url: /images </li></ul><ul><li>static_dir: static/images </li></ul><ul><li>Can also tag ‘secure’ if desired </li></ul>
  42. 42. Tips for Your Programs <ul><li>Be creative! </li></ul><ul><li>The webapp framework supports: </li></ul><ul><ul><li>get, post, head, options, put, delete, trace </li></ul></ul><ul><li>Map user pages to ~name? </li></ul>
  43. 43. Summary <ul><li>Google App Engine is more than flexible enough to handle your web app desires </li></ul><ul><li>And it’s only getting better! </li></ul><ul><li>More than enough possibilities to create a web app: you just need a good idea! </li></ul>
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×