0
Google App Engine
         Jonathan Blocksom
Stack Overflow DevDays DC, 10/26/09
Submit a Question: http://tinyurl.com/gae-dd09


              Attention Conservation Notice
                  • Jonathan ...
Submit a Question: http://tinyurl.com/gae-dd09


                     We’re lonely




                     Open positions...
Submit a Question: http://tinyurl.com/gae-dd09


              App Engine!
                  • Launched April 2008
       ...
Submit a Question: http://tinyurl.com/gae-dd09


              Your Scalable Website?
                  • Brad Fitzpatrick...
Submit a Question: http://tinyurl.com/gae-dd09


              App Engine Scalable Website



               Request      ...
Submit a Question: http://tinyurl.com/gae-dd09


              What’s Scalable?
                  • Google Moderator on Ap...
Submit a Question: http://tinyurl.com/gae-dd09


              Scalable
Submit a Question: http://tinyurl.com/gae-dd09


              Google Moderator team didn’t...
                  • Provisi...
Submit a Question: http://tinyurl.com/gae-dd09


              App Engine Architecture
Submit a Question: http://tinyurl.com/gae-dd09


              Static Content
                  • Served by Google
       ...
Submit a Question: http://tinyurl.com/gae-dd09


              Dynamic Content
                  • Java xor Python
       ...
Submit a Question: http://tinyurl.com/gae-dd09


              Datastore 1 / 3
                  • Based on BigTable,
    ...
Submit a Question: http://tinyurl.com/gae-dd09


              Datastore 2 / 3
                  • Typical types:
        ...
Submit a Question: http://tinyurl.com/gae-dd09


              Datastore 3 / 3
                  • GQL:
                  ...
Submit a Question: http://tinyurl.com/gae-dd09


              Memcache
                  • Arbitrary key/value cache
    ...
Submit a Question: http://tinyurl.com/gae-dd09


              Users & Authentication
                  • @gmail.com addre...
Submit a Question: http://tinyurl.com/gae-dd09


              URL Fetch
                  • Load URL
                  • ...
Submit a Question: http://tinyurl.com/gae-dd09


              Even More...
                  • Scheduled Tasks
          ...
Submit a Question: http://tinyurl.com/gae-dd09


              Web Hooks Pattern
                  • Incoming data / opera...
Submit a Question: http://tinyurl.com/gae-dd09


              Templates?
                  • Treetop Wants to know
Submit a Question: http://tinyurl.com/gae-dd09


              Templates!
                  • Django Templates built in
  ...
Submit a Question: http://tinyurl.com/gae-dd09


              Let’s Review!
Submit a Question: http://tinyurl.com/gae-dd09


              Developing w/ App Engine
                  • Local test ser...
Submit a Question: http://tinyurl.com/gae-dd09


              Dashboard Demo
                  • http://appengine.google....
Submit a Question: http://tinyurl.com/gae-dd09


              What have we done?
                  • Served static web pa...
Submit a Question: http://tinyurl.com/gae-dd09


              Quotas
                  • 34 global quotas
               ...
Submit a Question: http://tinyurl.com/gae-dd09


              Quota Rule of Thumb
                  • Less than 1000 of e...
Submit a Question: http://tinyurl.com/gae-dd09


              All For Good
                  • Volunteering Opportunity S...
Submit a Question: http://tinyurl.com/gae-dd09


              Tech Overview
                  •      Partners provide lis...
Submit a Question: http://tinyurl.com/gae-dd09


              Tech Overview

User or
 API
                               ...
Submit a Question: http://tinyurl.com/gae-dd09


              Why A Different Back End?
                  • No full text ...
Submit a Question: http://tinyurl.com/gae-dd09


              On The Promiscuous Nature of GAE
                  • GAE pl...
Submit a Question: http://tinyurl.com/gae-dd09


              Challenges with AFG
                  • Parsing questionabl...
Submit a Question: http://tinyurl.com/gae-dd09


              Q &A
Submit a Question: http://tinyurl.com/gae-dd09


              More Information
                  • App Engine Documentati...
Upcoming SlideShare
Loading in...5
×

App Engine Dev Days DC 20091026

2,005

Published on

Jonathan Blocksom's talk on Google App Engine at Stack Overflow Dev Days DC in 2009.

Published in: Technology, News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
2,005
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
28
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "App Engine Dev Days DC 20091026"

  1. 1. Google App Engine Jonathan Blocksom Stack Overflow DevDays DC, 10/26/09
  2. 2. Submit a Question: http://tinyurl.com/gae-dd09 Attention Conservation Notice • Jonathan Blocksom jblocksom@google.com @jblocksom Software Engineer Google Public Sector Projects Team
  3. 3. Submit a Question: http://tinyurl.com/gae-dd09 We’re lonely Open positions at Google in DC
  4. 4. Submit a Question: http://tinyurl.com/gae-dd09 App Engine! • Launched April 2008 • Paid Use Feb 2009 • Java support May 2009 • Latest Release (1.2.7) October 2009 • Incoming Email • Previous Release (1.2.5) September 2009 • XMPP Support
  5. 5. Submit a Question: http://tinyurl.com/gae-dd09 Your Scalable Website? • Brad Fitzpatrick describes LiveJournal in ’07: net. LiveJournal Backend: 2007 Today (Roughly.) BIG-IP perlbal (httpd/proxy) Global Database bigip1 mod_perl bigip2 proxy1 master_a master_b web1 proxy2 web2 proxy3 Memcached web3 slave1 slave2 ... slave5 djabberd proxy4 mc1 web4 djabberd proxy5 ... mc2 User DB Cluster 1 djabberd webN mc3 uc1a uc1b mc4 User DB Cluster 2 ... uc2a uc2b gearmand Mogile Storage Nodes gearmand1 mcN User DB Cluster 3 sto1 sto2 gearmandN uc3a uc3b Mogile Trackers ... sto8 tracker1 tracker3 User DB Cluster N ucNa ucNb MogileFS Database “workers” gearwrkN Job Queues (xN) mog_a mog_b theschwkN jqNa jqNb slave1 slaveN http://danga.com/words/ 3
  6. 6. Submit a Question: http://tinyurl.com/gae-dd09 App Engine Scalable Website Request App Engine Response
  7. 7. Submit a Question: http://tinyurl.com/gae-dd09 What’s Scalable? • Google Moderator on App Engineused in March 2009 for White House Town Hall meeting
  8. 8. Submit a Question: http://tinyurl.com/gae-dd09 Scalable
  9. 9. Submit a Question: http://tinyurl.com/gae-dd09 Google Moderator team didn’t... • Provision a server • Install and configure an OS • Install and configure a web server • Install and configure a web framework • Install and configure a database • Install and configure a load balancer • Create user accounts • Crash in the middle of any of the above
  10. 10. Submit a Question: http://tinyurl.com/gae-dd09 App Engine Architecture
  11. 11. Submit a Question: http://tinyurl.com/gae-dd09 Static Content • Served by Google Static Content Servers • Very Efficient
  12. 12. Submit a Question: http://tinyurl.com/gae-dd09 Dynamic Content • Java xor Python • Sandboxed • No file writes • No sockets • No spawning threads/processes • Smart about caching • Fairly standard CGI Framework • Measured and Instrumented
  13. 13. Submit a Question: http://tinyurl.com/gae-dd09 Datastore 1 / 3 • Based on BigTable, transitioning to MegaStore • ACID Compliant • 2008 Talk on Implementation: http://sites.google.com/site/io/under-the- covers-of-the-google-app-engine-datastore
  14. 14. Submit a Question: http://tinyurl.com/gae-dd09 Datastore 2 / 3 • Typical types: Int, String, DateTime, GeoPt, PhoneNumber, PostalAddress, Email, Link, Blob...
  15. 15. Submit a Question: http://tinyurl.com/gae-dd09 Datastore 3 / 3 • GQL: Google Query Language • If it’s not SQL, it must be good! SELECT [* | __key__] FROM <kind> [WHERE <condition> [AND <condition> ...]] [ORDER BY <property> [ASC | DESC] [, <property> [ASC | DESC] ...]] [LIMIT [<offset>,]<count>] [OFFSET <offset>] <condition> := <property> {< | <= | > | >= | = | != } <value> <condition> := <property> IN <list> <condition> := ANCESTOR IS <entity or key>
  16. 16. Submit a Question: http://tinyurl.com/gae-dd09 Memcache • Arbitrary key/value cache • Similar API to memcached • get, set • get_multi, set_multi for speed • Namespace Support • 1MB limit per entry • Free Quota 8.6M/day, 48K/min (800/sec)
  17. 17. Submit a Question: http://tinyurl.com/gae-dd09 Users & Authentication • @gmail.com addresses • Apps for Domain • Admin Privileges
  18. 18. Submit a Question: http://tinyurl.com/gae-dd09 URL Fetch • Load URL • Asynchronous support • HTTP / HTTPS • Max 10 second response (default 5) • Max 1MB data
  19. 19. Submit a Question: http://tinyurl.com/gae-dd09 Even More... • Scheduled Tasks • Task Queue • Email • Image Manipulation cron jobs • XMPP email • ... XMPP
  20. 20. Submit a Question: http://tinyurl.com/gae-dd09 Web Hooks Pattern • Incoming data / operations to perform are URLs with POST data • Handled same as web request Scheduled Tasks Task Queue tasks Incoming Email XMPP Messages
  21. 21. Submit a Question: http://tinyurl.com/gae-dd09 Templates? • Treetop Wants to know
  22. 22. Submit a Question: http://tinyurl.com/gae-dd09 Templates! • Django Templates built in Variable substitution, function calls, loops, etc • Can add your own Python temlpate lib
  23. 23. Submit a Question: http://tinyurl.com/gae-dd09 Let’s Review!
  24. 24. Submit a Question: http://tinyurl.com/gae-dd09 Developing w/ App Engine • Local test server • Write your code, html, etc on local machine • Send code to App Engine • Monitor in App Engine Admin Console
  25. 25. Submit a Question: http://tinyurl.com/gae-dd09 Dashboard Demo • http://appengine.google.com/
  26. 26. Submit a Question: http://tinyurl.com/gae-dd09 What have we done? • Served static web pages • Served dynamic web pages • Stored data (cached & persistent) • Ran a cron Job • Authenticated users • We could have... • Scheduled a task, sent or received email, loaded a URL, modified an image, rocked out with XMPP
  27. 27. Submit a Question: http://tinyurl.com/gae-dd09 Quotas • 34 global quotas in Dashboard • Individual quotas: request size time amount of data • Exceptions thrown when hit • Free and Paid
  28. 28. Submit a Question: http://tinyurl.com/gae-dd09 Quota Rule of Thumb • Less than 1000 of everything • The Zen of Quotas: Meet the quotas and you will be scalable
  29. 29. Submit a Question: http://tinyurl.com/gae-dd09 All For Good • Volunteering Opportunity Search Engine • Started March 2009 in cooperation with White House, CNCS, Craigslist, Google... • Released June 2009 • Open source, released under Apache License http://code.google.com/p/allforgood/ • Front end at http://allforgood.org API powers search results on serve.gov
  30. 30. Submit a Question: http://tinyurl.com/gae-dd09 Tech Overview • Partners provide listings in XML or Google Spreadsheets • AARP, United Way, AmeriCorp, HandsOnNetwork, United Way • AFG indexes feeds, stores outside GAE • Initially Google Base • Now SOLR on private server • GAE front end • API for external apps, partners • Integration with Google Maps, FriendConnect, Facebook
  31. 31. Submit a Question: http://tinyurl.com/gae-dd09 Tech Overview User or API Search Queries Backend Volunteering Opportunities (Base or App Engine s SOLR) lt su Volunteering Opportunities Re memcache Listings ... Volunteering Opportunities Crawler Static Content Google Code SVN
  32. 32. Submit a Question: http://tinyurl.com/gae-dd09 Why A Different Back End? • No full text search in App Engine Datastore
  33. 33. Submit a Question: http://tinyurl.com/gae-dd09 On The Promiscuous Nature of GAE • GAE plays well with others • Do full text search with something else! • Serve large files with S3! • Serve videos with YouTube!
  34. 34. Submit a Question: http://tinyurl.com/gae-dd09 Challenges with AFG • Parsing questionable data • Integration w/ Social Networks • Google Base is an Odd Duck • I Was New And Had No Clue What Was Going • App Engine Outages
  35. 35. Submit a Question: http://tinyurl.com/gae-dd09 Q &A
  36. 36. Submit a Question: http://tinyurl.com/gae-dd09 More Information • App Engine Documentation • Google I/O 2008 & 2009 • 2008 for intro, 2009 for deep dives / Java • http://code.google.com/events/io/2009/ sessions.html#appengine “App Engine Nitty-Gritty” “Spark Plug to Drive Train” • Google Group • Samples http://code.google.com/p/google-app-engine-samples/
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×