• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
App Engine Dev Days DC 20091026
 

App Engine Dev Days DC 20091026

on

  • 2,001 views

Jonathan Blocksom's talk on Google App Engine at Stack Overflow Dev Days DC in 2009.

Jonathan Blocksom's talk on Google App Engine at Stack Overflow Dev Days DC in 2009.

Statistics

Views

Total Views
2,001
Views on SlideShare
1,977
Embed Views
24

Actions

Likes
0
Downloads
27
Comments
0

3 Embeds 24

http://gae-devdays09.appspot.com 22
http://localhost:8090 1
http://www.slideshare.net 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    App Engine Dev Days DC 20091026 App Engine Dev Days DC 20091026 Presentation Transcript

    • Google App Engine Jonathan Blocksom Stack Overflow DevDays DC, 10/26/09
    • Submit a Question: http://tinyurl.com/gae-dd09 Attention Conservation Notice • Jonathan Blocksom jblocksom@google.com @jblocksom Software Engineer Google Public Sector Projects Team
    • Submit a Question: http://tinyurl.com/gae-dd09 We’re lonely Open positions at Google in DC
    • Submit a Question: http://tinyurl.com/gae-dd09 App Engine! • Launched April 2008 • Paid Use Feb 2009 • Java support May 2009 • Latest Release (1.2.7) October 2009 • Incoming Email • Previous Release (1.2.5) September 2009 • XMPP Support
    • Submit a Question: http://tinyurl.com/gae-dd09 Your Scalable Website? • Brad Fitzpatrick describes LiveJournal in ’07: net. LiveJournal Backend: 2007 Today (Roughly.) BIG-IP perlbal (httpd/proxy) Global Database bigip1 mod_perl bigip2 proxy1 master_a master_b web1 proxy2 web2 proxy3 Memcached web3 slave1 slave2 ... slave5 djabberd proxy4 mc1 web4 djabberd proxy5 ... mc2 User DB Cluster 1 djabberd webN mc3 uc1a uc1b mc4 User DB Cluster 2 ... uc2a uc2b gearmand Mogile Storage Nodes gearmand1 mcN User DB Cluster 3 sto1 sto2 gearmandN uc3a uc3b Mogile Trackers ... sto8 tracker1 tracker3 User DB Cluster N ucNa ucNb MogileFS Database “workers” gearwrkN Job Queues (xN) mog_a mog_b theschwkN jqNa jqNb slave1 slaveN http://danga.com/words/ 3
    • Submit a Question: http://tinyurl.com/gae-dd09 App Engine Scalable Website Request App Engine Response
    • Submit a Question: http://tinyurl.com/gae-dd09 What’s Scalable? • Google Moderator on App Engineused in March 2009 for White House Town Hall meeting
    • Submit a Question: http://tinyurl.com/gae-dd09 Scalable
    • Submit a Question: http://tinyurl.com/gae-dd09 Google Moderator team didn’t... • Provision a server • Install and configure an OS • Install and configure a web server • Install and configure a web framework • Install and configure a database • Install and configure a load balancer • Create user accounts • Crash in the middle of any of the above
    • Submit a Question: http://tinyurl.com/gae-dd09 App Engine Architecture
    • Submit a Question: http://tinyurl.com/gae-dd09 Static Content • Served by Google Static Content Servers • Very Efficient
    • Submit a Question: http://tinyurl.com/gae-dd09 Dynamic Content • Java xor Python • Sandboxed • No file writes • No sockets • No spawning threads/processes • Smart about caching • Fairly standard CGI Framework • Measured and Instrumented
    • Submit a Question: http://tinyurl.com/gae-dd09 Datastore 1 / 3 • Based on BigTable, transitioning to MegaStore • ACID Compliant • 2008 Talk on Implementation: http://sites.google.com/site/io/under-the- covers-of-the-google-app-engine-datastore
    • Submit a Question: http://tinyurl.com/gae-dd09 Datastore 2 / 3 • Typical types: Int, String, DateTime, GeoPt, PhoneNumber, PostalAddress, Email, Link, Blob...
    • Submit a Question: http://tinyurl.com/gae-dd09 Datastore 3 / 3 • GQL: Google Query Language • If it’s not SQL, it must be good! SELECT [* | __key__] FROM <kind> [WHERE <condition> [AND <condition> ...]] [ORDER BY <property> [ASC | DESC] [, <property> [ASC | DESC] ...]] [LIMIT [<offset>,]<count>] [OFFSET <offset>] <condition> := <property> {< | <= | > | >= | = | != } <value> <condition> := <property> IN <list> <condition> := ANCESTOR IS <entity or key>
    • Submit a Question: http://tinyurl.com/gae-dd09 Memcache • Arbitrary key/value cache • Similar API to memcached • get, set • get_multi, set_multi for speed • Namespace Support • 1MB limit per entry • Free Quota 8.6M/day, 48K/min (800/sec)
    • Submit a Question: http://tinyurl.com/gae-dd09 Users & Authentication • @gmail.com addresses • Apps for Domain • Admin Privileges
    • Submit a Question: http://tinyurl.com/gae-dd09 URL Fetch • Load URL • Asynchronous support • HTTP / HTTPS • Max 10 second response (default 5) • Max 1MB data
    • Submit a Question: http://tinyurl.com/gae-dd09 Even More... • Scheduled Tasks • Task Queue • Email • Image Manipulation cron jobs • XMPP email • ... XMPP
    • Submit a Question: http://tinyurl.com/gae-dd09 Web Hooks Pattern • Incoming data / operations to perform are URLs with POST data • Handled same as web request Scheduled Tasks Task Queue tasks Incoming Email XMPP Messages
    • Submit a Question: http://tinyurl.com/gae-dd09 Templates? • Treetop Wants to know
    • Submit a Question: http://tinyurl.com/gae-dd09 Templates! • Django Templates built in Variable substitution, function calls, loops, etc • Can add your own Python temlpate lib
    • Submit a Question: http://tinyurl.com/gae-dd09 Let’s Review!
    • Submit a Question: http://tinyurl.com/gae-dd09 Developing w/ App Engine • Local test server • Write your code, html, etc on local machine • Send code to App Engine • Monitor in App Engine Admin Console
    • Submit a Question: http://tinyurl.com/gae-dd09 Dashboard Demo • http://appengine.google.com/
    • Submit a Question: http://tinyurl.com/gae-dd09 What have we done? • Served static web pages • Served dynamic web pages • Stored data (cached & persistent) • Ran a cron Job • Authenticated users • We could have... • Scheduled a task, sent or received email, loaded a URL, modified an image, rocked out with XMPP
    • Submit a Question: http://tinyurl.com/gae-dd09 Quotas • 34 global quotas in Dashboard • Individual quotas: request size time amount of data • Exceptions thrown when hit • Free and Paid
    • Submit a Question: http://tinyurl.com/gae-dd09 Quota Rule of Thumb • Less than 1000 of everything • The Zen of Quotas: Meet the quotas and you will be scalable
    • Submit a Question: http://tinyurl.com/gae-dd09 All For Good • Volunteering Opportunity Search Engine • Started March 2009 in cooperation with White House, CNCS, Craigslist, Google... • Released June 2009 • Open source, released under Apache License http://code.google.com/p/allforgood/ • Front end at http://allforgood.org API powers search results on serve.gov
    • Submit a Question: http://tinyurl.com/gae-dd09 Tech Overview • Partners provide listings in XML or Google Spreadsheets • AARP, United Way, AmeriCorp, HandsOnNetwork, United Way • AFG indexes feeds, stores outside GAE • Initially Google Base • Now SOLR on private server • GAE front end • API for external apps, partners • Integration with Google Maps, FriendConnect, Facebook
    • Submit a Question: http://tinyurl.com/gae-dd09 Tech Overview User or API Search Queries Backend Volunteering Opportunities (Base or App Engine s SOLR) lt su Volunteering Opportunities Re memcache Listings ... Volunteering Opportunities Crawler Static Content Google Code SVN
    • Submit a Question: http://tinyurl.com/gae-dd09 Why A Different Back End? • No full text search in App Engine Datastore
    • Submit a Question: http://tinyurl.com/gae-dd09 On The Promiscuous Nature of GAE • GAE plays well with others • Do full text search with something else! • Serve large files with S3! • Serve videos with YouTube!
    • Submit a Question: http://tinyurl.com/gae-dd09 Challenges with AFG • Parsing questionable data • Integration w/ Social Networks • Google Base is an Odd Duck • I Was New And Had No Clue What Was Going • App Engine Outages
    • Submit a Question: http://tinyurl.com/gae-dd09 Q &A
    • Submit a Question: http://tinyurl.com/gae-dd09 More Information • App Engine Documentation • Google I/O 2008 & 2009 • 2008 for intro, 2009 for deep dives / Java • http://code.google.com/events/io/2009/ sessions.html#appengine “App Engine Nitty-Gritty” “Spark Plug to Drive Train” • Google Group • Samples http://code.google.com/p/google-app-engine-samples/