Writing Apps the Google-y Way (Brisbane)

WRITING APPS THE GOOGLE-Y WAY Pamela Fox, YOW! Australia 2010 (Brisbane)

Who am I? twitter.com/pamelafox [email_address] pamelafox.org you get the idea...

Who am I? Google Maps API Google Wave API 2006 2010 2008 Google App Engine

Who am I? wave side projects 92 apps

Who am I? Java pYthon

What is App Engine?
“ Google App Engine enables you to build and host web apps on the same systems that power Google applications.”
http://code.google.com/appengine

What is a “web app”?

Static vs. Dynamic

Anonymous vs. Users

Intranet vs. Internet ~2 billion Hundreds - Thousands

What is a “web app”?

Some Google web apps

Some Google App Engine web apps www.gifttag.com www.buddypoke.com

Google apps on App Engine panoramio.com pubsubhubbub.appspot.com

How does App Engine work?
You upload application code & resources to Google.
Google serves your application from scalable infrastructure.
You pay for only the resources that Google used in serving the application.

Demo: Guestbook awesomest-app.appspot.com http://code.google.com/p/google-app-engine-samples/source/browse/trunk/guestbook appengine.google.com localhost build deploy monitor

App Engine architecture

App Engine architecture user task

App Engine architecture

App Engine architecture LIMIT CPU LIMIT Memory LIMIT Time

App Engine architecture hardware ports globals file system Groovy, JRuby, Mirah, Clojure, Scala

App Engine architecture 141,241,791 calls 1 GB data $0.15 GB/month 45,000,000 calls 657,000 - 46,000,000 calls *Always check docs for latest quotas. 192,672,000 calls 558 GB data $0.15 GB/month 7,000 - 1,700,000 calls $0.0001 per mail sent 46,000,000 calls 1,046 GB data sent 100,000 - 20,000,000 calls

The tricky bits

Datastore Entity Properties Key Entity Entity Entity Entity Path Kind Name/ID

Example: Speaker Entities Key Path Kind ID First Name Last Name Speaker1 - Speaker 1 Rod Johnson Key Path Kind ID First Name Last Name Middle Name Suffix Speaker2 - Speaker 2 Guy Steele L Jr.

Modeling Speaker Entities
class Speaker(db.model):
firstname = db.StringProperty(required=True)
lastname = db.StringProperty(required=True)
middlename = db.StringProperty()
namesuffix = db.StringProperty()
website = db.StringProperty()
keynote = db.BooleanProperty(default=False)

Saving Speaker Entities
rod = Speaker(firstname="Rod", lastname="Johnson")
guy = Speaker(firstname="Guy", lastname="Steele",
middlename="L", namesuffix="Jr.")
rod.put()
guy.put()

Updating Speaker Entities
rod = Speaker.get_by_id(1)
guy = Speaker.get_by_id(2)
rod.website = "http://www.sexyspring.com"
rod.keynote = True
guy.website = "http://www.lusciouslisp.com"
guy.keynote = True
db.put(rod, guy)
LIMIT! (size/# of batch ops)

Queries & Indexes Query Index Index Index Index Query Query Query Query Query

Queries & Indexes SELECT * from Speaker ORDER BY lastname LIMIT! (# of results) key lastname Speaker3 Fox Speaker4 Hohpe Speaker1 Johnson Speaker2 Steele

Queries & Indexes SELECT * from Speaker ORDER by middlename key middlename Speaker2 L

Queries & Indexes SELECT * from Speaker WHERE keynote = True key keynote Speaker1 True Speaker2 True Speaker3 False Speaker4 False

Queries & Indexes SELECT * from Speaker WHERE keynote = False key keynote Speaker1 True Speaker2 True Speaker3 False Speaker4 False

Queries
allspeakers = Speaker.all().order('lastname)
for speaker in allspeakers:
print speaker.firstname + '' + speaker.lastname + '' + speaker.website
keynotespeakers = Speaker.all().filter('keynote = ', True)
notspecialspeakers = Speaker.all().filter('keynote = ', False)
LIMIT! (size of results)

Custom Indexes
speakers = Speaker.all().order('lastname')
.order('keynote')
SELECT * from Speaker ORDER BY lastname, keynote key lastname keynote Speaker3 Fox false Speaker4 Hohpe false Speaker1 Johnson true Speaker2 Steele true

Custom Indexes
speakers = Speaker.all().order('lastname')
.filter('keynote =', True)
SELECT * from Speaker WHERE lastname > 'Johnson' and keynote = true key lastname keynote Speaker3 Fox false Speaker4 Hohpe false Speaker1 Johnson true Speaker2 Steele true

Impossible Indexes SELECT * from Speaker WHERE lastname < 'Steele' and firstname > 'Gregory' ...not in subsequent rows! key lastname firstname Speaker3 Fox Pamela Speaker4 Hohpe Gregory Speaker1 Johnson Rod Speaker2 Steele Guy

Impossible Indexes SELECT * from Speaker WHERE lastname > 'Fox' ORDER BY firstname ...not in the correct order! key lastname firstname Speaker3 Fox Pamela Speaker4 Hohpe Gregory Speaker1 Johnson Rod Speaker2 Steele Guy

Queries with Offset
speakers = Speaker.all().fetch(2, 2)
SELECT * from Speaker LIMIT 2 OFFSET 2 1 2 ...slow! LIMIT! (# of offset) key lastname Speaker3 Fox Speaker4 Hohpe Speaker1 Johnson Speaker2 Steele

Queries with Cursors
query = db.Query(Speaker)
speakers = q.fetch(1000)
cursor = q.cursor()
memcache.set('speaker_cursor', cursor)
...
last_cursor = memcache.get('speaker_cursor')
q.with_cursor(last_cursor)
speakers = q.fetch(1000)

More Properties class Talk(db.Model): title = db.StringProperty(required=True) abstract = db.TextProperty(required=True) speaker = db.ReferenceProperty(Speaker) tags = db.StringListProperty() pamela = Speaker.all().filter('firstname = ', 'Pamela').get() talk = Talk('Writing Apps the Googley Way', 'Bla bla bla', pamela, ['App Engine', 'Python']) talk.put() talk = Talk('Wonders of the Onesie', 'Bluh bluh bluh', pamela, ['Pajamas', 'Onesies']) talk.put()

Back-References pamela = Speaker.all().filter('firstname = ', 'Pamela').get() for talk in pamela.talk_set: print talk.title SELECT * from Talk WHERE speaker = Speaker3 key speaker Talk6 Speaker2 Talk1 Speaker3 Talk2 Speaker3 Talk5 Speaker4

Searching List Properties talks = Talk.all().filter('tags = ', 'python') .fetch(10) SELECT * from Talk WHERE tags = 'Python' LIMIT! (# of index rows) key lastname Talk1 App Engine Talk2 Pajamas Talk1 Python Talk2 Onesies

Update Transactions commit journal apply entities apply indexes A B

Entity Groups pamela = Speaker.all().filter('firstname = ', 'Pamela').get() talk1 = Talk('Writing Apps the Googley Way', 'Bla bla bla', pamela, ['App Engine', 'Python'], parent=pamela) talk2 = Talk('Wonders of the Onesie', 'Bluh bluh bluh', pamela, ['Pajamas', 'Onesies'], parent=pamela) db.put(talk1, talk2) def update_talks(): talk1.title = 'Writing Apps the Microsoft Way' talk2.title = 'Wonders of the Windows' db.put(talk1, talk2) db.run_in_transaction(update_talks)

Common Features

Counters 1 2 3 4 5 people have done something.

RageTube: Global Stats ragetube.net http://github.com/pamelafox/ragetube

RageTube: Global Stats SongStat yaycount viewcount title artist Key Path Kind Name (song) naycount mehcount

RageTube: Global Stats viewcount viewcount viewcount datastore memcache

RageTube: Global Stats class Song(db.Model): viewcount = db.IntegerProperty(default=0) title = db.StringProperty() artist = db.StringProperty() def get_viewcount(self): viewcount = self.viewcount cached_viewcount = memcache.get('viewcount-' + self.key().name(), self.key().kind()) if cached_viewcount: viewcount += cached_viewcount return viewcount @classmethod def flush_viewcount(cls, name): song = cls.get_by_key_name(name) value = memcache.get('viewcount-' + name, cls.kind()) memcache.decr('viewcount-' + name, value, cls.kind()) song.viewcount += value song.put() @classmethod def incr_viewcount(cls, name, interval=5, value=1): memcache.incr('viewcount-' + name, value, cls.kind()) interval_num = get_interval_number(datetime.now(), interval) task_name = '-'.join([cls.kind(), name.replace(' ', '-'), 'viewcount', str(interval), str(interval_num)]) deferred.defer(cls.flush_viewcount, name, _name=task_name) LIMIT! (# of tasks)

Ratings Rated by 500 users.

App Gallery: Ratings google.com/analytics/apps/

App Gallery: Ratings Comment Application total_ratings sum_ratings avg_rating rated_index comment_count rating

App Gallery: Ratings def UpdateAppCommentData(self, rating, operation): def UpdateCommentData(self, rating, operation): self.comment_count += 1 * operation self.sum_ratings += rating * operation self.total_ratings += 1 * operation self.avg_rating = int(round(self.sum_ratings / self.total_ratings)) self.rated_index = '%d:%d:%d' % (self.avg_rating, self.total_ratings, self.index) self.put() db.run_in_transaction(UpdateCommentData, self, rating, operation) app.UpdateAppCommentData(rating, db_models.Comment.ADD) comment = db_models.Comment() comment.application = app comment.rating = rating comment.put() query.order('-avg_rating').order('-rated_index')

Geospatial Queries

City-Go-Round: Agencies citygoround.org https://github.com/walkscore/City-Go-Round

City-Go-Round: Geo Queries Agency GeoModel location (GeoPt) location_geocells (StringListProperty)

City-Go-Round: Geo Queries def fetch_agencies_near(lat, long, bbox_side_in_miles): query = Agency.all() bbox = bbox_centered_at(lat, long, bbox_side_in_miles) return Agency.bounding_box_fetch(query, bbox, max_results = 50) def bounding_box_fetch(query, bbox, max_results=1000,): results = [] query_geocells = geocell.best_bbox_search_cells(bbox) for entity in query.filter('location_geocells IN', query_geocells): if len(results) == max_results: break if (entity.location.lat >= bbox.south and entity.location.lat <= bbox.north and entity.location.lon >= bbox.west and entity.location.lon <= bbox.east): results.append(entity) return results

Full Text Search
pizza
Search Thingy It's like pizza, but in the cloud. Other Thingy This will make you smell as delicious as pizza.

Disclosed.ca: Search https://github.com/nurey/disclosed disclosed.ca

Disclosed.ca: Search Contract agency_name vendor_name description comments uri

Disclosed.ca: Search from search.core import SearchIndexProperty, porter_stemmer class Contract(db.Model): uri = db.StringProperty(required=True) agency_name = db.StringProperty(required=True) vendor_name = db.StringProperty(required=True) description = db.StringProperty() comments = db.TextProperty() search_index = SearchIndexProperty(('agency_name', 'vendor_name', 'description', 'comments'), indexer=porter_stemmer) results = Contract.search_index.search(sheep').fetch(20)

Disclosed.ca: Search Contract agency_name vendor_name description comments uri search_index (StringListProperty) SearchIndex

Disclosed.ca: Search SELECT FROM ContractSearch WHERE search_index = "sheep" key search_index ContractSearch1 charter ContractSearch1 june ContractSearch1 sheep ContractSearch2 sheep ContractSearch1 wood

More Learning http://ae-book.appspot.com http://code.google.com/appengine http://blog.notdot.net/

AppEngine: Now & Later "Run your web apps on Google's infrastructure. Easy to build, easy to maintain, easy to scale."
Roadmap:
App Engine (Standard):
MapReduce
Bulk Import/Export
Channel API
Datastore Options
App Engine for Business:
SQL
SLA + Support

Thanks for coming!

Writing Apps the Google-y Way (Brisbane)

by Pamela Fox

Accessibility

Categories

Upload Details

Usage Rights

Statistics

Writing Apps the Google-y Way (Brisbane) Presentation Transcript