Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Ā
I've (probably) been using Google App Engine for a week longer than you have
1. Iāve (probably) been using
Google App Engine
for a week longer than you have
Simon Willison - http://simonwillison.net/
BarCamp London 4
31st May 2008
2. Except you have to re-write
your whole application
If you totally rethink the
way you use a database
3. What it can do
ā¢ Serve static ļ¬les
ā¢ Serve dynamic requests
ā¢ Store data
ā¢ Call web services (sort of)
ā¢ Authenticate against Googleās user database
ā¢ Send e-mail, process images, use memcache
4. The dev environment
is really, really nice
ā¢ Download the (open source) SDK
ā¢ a full simulation of the App Engine environment
ā¢ dev_appserver.py myapp for a local webserver
ā¢ appcfg.py update myapp to deploy to the cloud
5. Options
ā¢ You have to use Python
ā¢ You can choose how you use it:
ā¢ CGI-style scripts
ā¢ WSGI applications
ā¢ Googleās webapp framework
ā¢ Django (0.96 provided, or install your own)
7. With webapp and WSGI
import wsgiref.handlers
from google.appengine.ext import webapp
class MainPage(webapp.RequestHandler):
Ā def get(self):
Ā Ā self.response.headers['Content-Type'] = 'text/html'
Ā Ā self.response.out.write('Hello, webapp World!')
def main():
Ā application = webapp.WSGIApplication(
[('/', MainPage)], debug=True)
Ā wsgiref.handlers.CGIHandler().run(application)
if __name__ == quot;__main__quot;:
Ā main()
8. With Django
from django.conf.urls.defaults import *
from django.http import HttpResponse
def hello(request):
return HttpResponse(quot;Hello, World!quot;)
urlpatterns = patterns('',
('^$', hello),
)
(And django_dispatch.py for boilerplate)
9. ā¢ Don't use CGI: it requires reloading for every hit
ā¢ Why use Django over webapp?
ā¢ Django has easy cookies and custom 500 errors
ā¢ Django is less verbose
ā¢ Django middleware is really handy
ā¢ You can use other WSGI frameworks if you like
12. āBigtable is a distributed storage system for
managing structured data that is designed
to scale to a very large size: petabytes of
data across thousands of commodity
servers. Many projects at Google store data
in Bigtable, including web indexing, Google
Earth, and Google Finance.ā
13. The App Engine datastore
ā¢ Apparently based on BigTable
ā¢ Absolutely not a relational database
ā¢ No joins (they do have āreference ļ¬eldsā)
ā¢ No aggregate queries - not even count()!
ā¢ Hierarchy affects sharding and transactions
ā¢ All queries must run against an existing index
14. Models and entities
ā¢ Data is stored as entities
ā¢ Entities have properties - key/value pairs
ā¢ An entity has a unique key
ā¢ Entities live in a hierarchy, and siblings exist in
the same entity group - these are actually really
important for transactions and performance
ā¢ A model is kind of like a class; it lets you deļ¬ne
a type of entity
15. AppEngine Models
from google.appengine.ext import db
class Account(db.Model):
slug = db.StringProperty(required=True)
owner = db.UserProperty()
onlyme = db.BooleanProperty()
referrers = db.StringListProperty()
(There is a ReferenceProperty, but I havenāt used it yet)
18. BUT...
ā¢ All queries must run against an existing index
ā¢ Filtering or sorting on a property requires that
the property exists
ā¢ Inequality ļ¬lters are allowed on one property only
ā¢ Properties in inequality ļ¬lters must be sorted
before other sort orders
ā¢ ... and various other rules
ā¢ Thankfully the dev server creates most indexes
for you automatically based on usage
19. How indexes are used
1. The datastore identiļ¬es the index that
corresponds with the queryās kind, ļ¬lter
properties, ļ¬lter operators, and sort orders.
2. The datastore starts scanning the index at the
ļ¬rst entity that meets all of the ļ¬lter conditions
using the queryās ļ¬lter values.
3. The datastore continues to scan the index,
returning each entity, until it ļ¬nds the next entity
that does not meet the ļ¬lter conditions, or until
it reaches the end of the index.
20. Further limitations
ā¢ If you create a new index and push it live,
you have to wait for it to rebuilt
ā¢ This can take hours, and apparently can go
wrong
ā¢ You canāt safely grab more than about 500
records at once - App Engine times out
ā¢ You canāt delete in bulk
21. Other random notes
ā¢ You have to use the URL Fetch API to do
HTTP requests (e.g. for web services) - and it
times out agressively at about 5 seconds
ā¢ The Google accounts Users API is ridiculously
easy to use, but...
ā¢ no permanent unique identiļ¬er; if the user
changes their e-mail address youāre screwed
ā¢ The new image and memcache APIs are neat
22. Final thoughts
ā¢ Itās really nice not to have to worry about hosting
ā¢ But... the lack of aggregate queries and ad-hoc
queries really hurts
ā¢ Perfect for small projects you donāt want to
worry about and big things which youāre sure will
have to scale
ā¢ Pricing is comparable to S3 - i.e. stupidly cheap
23. Pricing
ā¢ $0.10 - $0.12 per CPU core-hour
ā¢ $0.15 - $0.18 per GB-month of storage
ā¢ $0.11 - $0.13 per GB outgoing bandwidth
ā¢ $0.09 - $0.11 per GB incoming bandwidth