Your SlideShare is downloading. ×
Introduction toGoogle App Engine      Luke Francl
Scaling is  hard.
He wrote Python
RequestsServers
RequestsServers
RequestsServers
Server   Virtual Server   Process   Request
Server   Virtual Server   Process   Request
App Engine            Restrictions•   No access to servers       •   No C extensions•   Read-only file system       •   No ...
EC2 versus App             EngineApp Engine            EC2   Instance   $0.02 - $0.68 (15                                 ...
Heroku vs App            EngineApp Engine          Heroku      Free     750 instance                                  28 i...
Totally awesomethings about App     Engine
http://<version>.<app‐id>.appspot.com
Getting Started                      or   dev_appserver.pyhttp://code.google.com/appengine/downloads.html
from
google.appengine.ext
import
webapp
class
UserHandler(webapp.RequestHandler):

def
get(self):



key_name
=
self.request.get(user_id)



user
=
User.get_by_ke...
webapp = BARE   BONES!
Substrate
The Datastore                photo:
class
User(db.Model):



email



=
db.EmailProperty(required=True)



password
=
db.StringProperty(required=True)



age
...
key
=
db.Key.from_path(User,
foobar,






















Preferences,
123)prefs
=
Preferences.get(key)user
=
User.get_by...
def
txn(user,
items_per_page,
bg_color):

user.put()

prefs
=
Preferences(parent=user,





















items_per_page...
Queryingquery
=
User.all()query.filter("age
>=",
42)query.order("age")#
get
list
of
up
to
100
usersusers
=
query.fetch(100...
Custom Index            Query#
requires
custom
indexquery
=
User.all()query.filter("age
>=",
20)query.filter("website
=",
...
Invalid Queriesquery
=
User.all()query.filter("age
>=",
42)query.filter("email
>=",
"abc")query
=
User.all()query.filter("...
sputter
UserPost          Follow
following
=
Follow.all().filter("user
=",
current_user)Post.all().filter("user
in",
following).order("‐created_at")
following
=
Follow.all().filter("user
=",
current_user)Post.all().filter("user
in",
following).order("‐created_at")
User                TimelinePosPost   Follow                     t
TimelinePost.all().filter("delivered_to",
current_user).order("‐created_at")
Follow along at:http://bitbucket.org/look/sputter
App EngineWeaknesses
...but it is moving                  fast           ≈ 1 release/month•   Backends                       •   Bulk import/ex...
On the road map• SSL for your own domain• Full-text search• Python 2.7• Hosted SQL
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Introduction to Google App Engine
Upcoming SlideShare
Loading in...5
×

Introduction to Google App Engine

1,361

Published on

Published in: Technology, News & Politics
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,361
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • About me: primarily think of myself as a Rails developer; brought into Best Buy to join a team looking at porting a Rails app to GAE; now have done GAE development for 1.5 years\n
  • Why App Engine?\n\n \n \nThe promise of app engine is that if you write your application the App Engine way, you can outsource the management and scaling of your application to Google.\n
  • In short, scaling is hard so...\n
  • Let these guys do it for you.\n
  • One problem with scaling is that you have to over provision in order to get acceptable user performance. \n
  • Of course scaling&apos;s not so neat. Traffic goes up and down. You either end up scrambling to add capacity or over-provisioned. Probably both.\n
  • Google App engine takes care of that for you. By running on Google&apos;s infrastructure and following their restrictions, App Engine can scale up and down automatically.\n
  • Server: physical hard ware - most bang for the buck, but inflexible. Example: colo\nVirtual server: dedicated access to a fraction of a server&apos;s resources. Example: VMWare, AWS\nProcess: managed hosting environment. Example: Heroku\nRequest: Spin up and down resources on demand, automatically. Example: App Engine\n
  • GAE went from an amazing deal to merely a good deal for free hosting. It is no comparable to Heroku in pricing.\n
  • \n \n http://www.flickr.com/photos/kash_if/2539073569/\n \n \n
  • EC2 is MUCH more flexible in what you can do -- you can run anything. &quot;Build your own cloud&quot;\n\n \n \nApp Engine is more expensive but provides a managed environment that means you do zero sysadmin.\n
  • Heroku and App Engine are very similar.\n\n \n \nHeroku&apos;s main advantage (now that App Engine pricing has increased) is its support for 3rd party add ons, plus you can use EC2 APIs with less latency.\n
  • Admin Console (development server)\nAdmin Console (production server)\nApp Stats (extreme visualization of requests)\nRemote API &amp; remote shell\n
  • \n \n
  • \n \n
  • Datastore viewer\n
  • And you can drill down and view/edit all the details of an existing entity.\n
  • Permissions settings are really cool, and you can easily add more owners to the app.\n
  • Versions! All run at the same time. Use for testing, data migrations, even deploy a Java or Go runtime to a different version and access the same datastore!\n
  • \n \n
  • Getting started with App Engine: step 1: download the SDK\n\n \n \nThe SDK is how you run your app locally. It comes with a launcher, or you can use use dev_appserver.py on the command line.\n\n \n \nIt is important to note: the SDK EMULATES App Engine production environment. The implementation of the APIs is different, and the server itself is single threaded. AND your computer is probably way faster than an App Engine instance, so some things that work locally just *won&apos;t* on App Engine.\n
  • Once you&apos;ve got the SDK installed, it&apos;s time to dive in.\n
  • Here I am outputting HTML directly for simplicities&apos; sake, but webapp can use Django templates (also built into App Engine).\n
  • As you can see, webapp is pretty bare-bones. It doesn&apos;t do much for you.\n\n \n \nApp Engine also comes packaged with Django, but for the longest time the version supported was 0.98 or 1.1 (both super old). Adding your own was hard, and Django isn&apos;t optimized for App Engine anyway. It&apos;s slow, and a lot of what makes Django &quot;Django&quot; doesn&apos;t work on App Engine. You can&apos;t use the automatic admin or Django Models.\n
  • So people keep coming up with frameworks optimized for App Engine.\n\n \n \nApp Engine Frameworks tend to die.\n\n \n \nI think the number of interested developers is just too low.\n
  • So people keep coming up with frameworks optimized for App Engine.\n\n \n \nApp Engine Frameworks tend to die.\n\n \n \nI think the number of interested developers is just too low.\n
  • So people keep coming up with frameworks optimized for App Engine.\n\n \n \nApp Engine Frameworks tend to die.\n\n \n \nI think the number of interested developers is just too low.\n
  • So people keep coming up with frameworks optimized for App Engine.\n\n \n \nApp Engine Frameworks tend to die.\n\n \n \nI think the number of interested developers is just too low.\n
  • So people keep coming up with frameworks optimized for App Engine.\n\n \n \nApp Engine Frameworks tend to die.\n\n \n \nI think the number of interested developers is just too low.\n
  • So people keep coming up with frameworks optimized for App Engine.\n\n \n \nApp Engine Frameworks tend to die.\n\n \n \nI think the number of interested developers is just too low.\n
  • webapp2 by the author of tipfy. Very simple, adds some nicities to WebApp, like routes\n
  • So, we wrote Substrate.\n\n \n \nIt&apos;s not a framework. It just packages up some useful libraries (including some we&apos;ve written ourselves) and provides a base application to build off of.\n
  • Google built the Datastore for App Engine on top of the famous BigTable. It is a distributed data store that supports querying and transactions in limited circumstances. It does not support joins, though it does have references. If you have used MongoDB, it sort of reminds me of that (though Mongo&apos;s querying is more powerful).\n\n \n \nIn my experience I would compare it most closely to MongoDB.\n\n \n http://www.flickr.com/photos/redjar/113152393/\n \n \n \n\n \n
  • Code example: defining a model\nProblem: finding a good key \n- key is the only way to guarantee uniqueness\n\n \n \nExample of several of the property types\n
  • Data modeling: Keys\nThe Key encodes the type of Entity and its unique ID or Name. Keys are universally unique in your application. IDs are assigned automatically, but you can assign names.\n\n \n \nYou can use a key to fetch an Entity. This is the fastest way to look up an entity\n
  • Code Example: Transactions\n\n \n \nTransactions are a special case on App Engine. All entities in a transaction must be in the same &quot;Entity Group&quot;, which means they have the same root key. By default, each entity is in its own entity group.\n\n \n \nAs keys cannot be changed, once an entity is assigned to an entity group, this cannot be changed.\n
  • Code Example: querying\n\n \n \nOf course, most of the time you can&apos;t look everything up by key. \n
  • Querying with Custom indexes\n
  • App Engine is limited in the kinds of queries it can do\n
  • \n \n
  • \n \n
  • This is NOT a good idea, because IN query translates to n queries that are OR&apos;ed together.\n
  • So, here&apos;s another idea: make an &quot;inbox&quot; of posts from the users a user follows when they get created. We can do this in the background after a status is posted. All the fields necessary to display a post will be cached in the TimelinePost, which means we don&apos;t have to fetch any other records to display it.\n
  • \n \n
  • \n \n
  • &amp;#x201C;Cold start&amp;#x201D; time - especially for Java, especially for bulky frameworks or JVM langs\nNo SSL for your own domain\n&amp;#x201C;Naked&amp;#x201D; domains don&amp;#x2019;t work\nPython 2.5 (2006!!!) --- Python 2.7 coming soon\nNo full-text search (wtf, Google)\nVendor lock-in\n
  • \n \n http://www.flickr.com/photos/stignygaard/2228400937/\n \n \n
  • \n \n http://www.flickr.com/photos/ocarchives/3401996925/\n \n \n \n\n \n \nI would joke about Duke Nukem, but that already came out.\n
  • Despite the pricing controversy, I think the future for App Engine is bright. Leaving &quot;preview&quot; status and becoming self-sustaining for Google. Google will continue to devote resources to improving GAE. Every month a new release comes out with more features and increases the amount you can do with it.\n\n \n \nIt is awesome for apps that stay inside its sweet spot: read-heavy apps that respond quickly. Not so great for heavy number crunching. EC2 is a better choice there; GAE is optimized for serving HTTP and probably always will be.\n\n \n \n\n \n \nConclusion - for hobby projects GAE is still a great option. Most projects can stay under the 28 hour/day limit, but if your app gets popular, you can smoothly scale up to hundreds of thousands of hits per day for a reasonable price.\n
  • Transcript of "Introduction to Google App Engine"

    1. 1. Introduction toGoogle App Engine Luke Francl
    2. 2. Scaling is hard.
    3. 3. He wrote Python
    4. 4. RequestsServers
    5. 5. RequestsServers
    6. 6. RequestsServers
    7. 7. Server Virtual Server Process Request
    8. 8. Server Virtual Server Process Request
    9. 9. App Engine Restrictions• No access to servers • No C extensions• Read-only file system • No sockets• 30 second response limit • Python, Java, Go• Non-relational datastore photo: Kash_if
    10. 10. EC2 versus App EngineApp Engine EC2 Instance $0.02 - $0.68 (15 $0.08 Hour GB RAM, 8x1GHz CPU) $0.08 - $0.64 (1G RAM,Background same as above 4.8GHz CPU)Autoscaling with effort automatic Storage SimpleDB, RDS, Datastore, Blobstore, Google EBS, S3 Storage (S3 clone) Python, Java, Go (written for Runs Anything! GAEs API) APIs S3, SQS, SES, Storage, BigQuery, Prediction many more API
    11. 11. Heroku vs App EngineApp Engine Heroku Free 750 instance 28 instance hours/dayProcessing hours/mo $0.05/hour tasks (run on web instances)Background (no autoscaling) Backends: $0.08 - $0.68/ 5 MB free, ≈$70/ hour Memcache lots! GB Ruby, Clojure, Runs Java, Python, Go Java, Node.js, Python N/A SSL $5-$100/mo (site.appspot.com only) Deploy git push appcfg.py upload
    12. 12. Totally awesomethings about App Engine
    13. 13. http://<version>.<app‐id>.appspot.com
    14. 14. Getting Started or dev_appserver.pyhttp://code.google.com/appengine/downloads.html
    15. 15. from
google.appengine.ext
import
webapp
    16. 16. class
UserHandler(webapp.RequestHandler):

def
get(self):



key_name
=
self.request.get(user_id)



user
=
User.get_by_key_name(key_name)



html
=
"<html><body><p>hello,
%s</p></body></html>"



self.response.out.write(html
%
user.email)app
=
webapp.WSGIApplication([(/user,































UserHandler)])if
__name__
==
__main__:





util.run_wsgi_app(app)
    17. 17. webapp = BARE BONES!
    18. 18. Substrate
    19. 19. The Datastore photo:
    20. 20. class
User(db.Model):



email



=
db.EmailProperty(required=True)



password
=
db.StringProperty(required=True)



age





=
db.IntegerProperty()



website

=
db.LinkProperty()



notes



=
db.TextProperty()user
=
User(key_name=foobar,












email=look@example.com,











password=encrypt(foobar,
salt),











website=http://example.com)user.put()
    21. 21. key
=
db.Key.from_path(User,
foobar,






















Preferences,
123)prefs
=
Preferences.get(key)user
=
User.get_by_key_name(foobar)prefs
=
Preferences.get_by_id(123,





























parent=user)
    22. 22. def
txn(user,
items_per_page,
bg_color):

user.put()

prefs
=
Preferences(parent=user,





















items_per_page=items_per_page,





















bg_color=bg_color)

prefs.put()user
=
User(key_name=example,











email=user@example.com)db.run_in_transaction(txn,
user,
100,
ff0000)
    23. 23. Queryingquery
=
User.all()query.filter("age
>=",
42)query.order("age")#
get
list
of
up
to
100
usersusers
=
query.fetch(100)#
iterate
over
all
users
(until
you
run
out
of
time)for
user
in
query:

#
do
stuff
with
user
    24. 24. Custom Index Query#
requires
custom
indexquery
=
User.all()query.filter("age
>=",
20)query.filter("website
=",
"http://example.com")#
index.yamlindexes:‐
kind:
User

properties:

‐
name:
age

‐
name:
email



direction:
asc
    25. 25. Invalid Queriesquery
=
User.all()query.filter("age
>=",
42)query.filter("email
>=",
"abc")query
=
User.all()query.filter("age
>=",
42)query.order("email")
    26. 26. sputter
    27. 27. UserPost Follow
    28. 28. following
=
Follow.all().filter("user
=",
current_user)Post.all().filter("user
in",
following).order("‐created_at")
    29. 29. following
=
Follow.all().filter("user
=",
current_user)Post.all().filter("user
in",
following).order("‐created_at")
    30. 30. User TimelinePosPost Follow t
    31. 31. TimelinePost.all().filter("delivered_to",
current_user).order("‐created_at")
    32. 32. Follow along at:http://bitbucket.org/look/sputter
    33. 33. App EngineWeaknesses
    34. 34. ...but it is moving fast ≈ 1 release/month• Backends • Bulk import/export• Always on instances • High replication datastore• Blobstore • MapReduce API• Image serving & thumbnailing • Browser-push "Channel" API• Remove fetch/count limits • Namespaces/Multitenancy photo: Stig Nygaard
    35. 35. On the road map• SSL for your own domain• Full-text search• Python 2.7• Hosted SQL

    ×