• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Writing Apps the Google-y Way (Brisbane)
 

Writing Apps the Google-y Way (Brisbane)

on

  • 2,728 views

Talk from Pamela Fox (me) at YOW 2010 in Brisbane. Covers App Engine and the datastore, with Python examples.

Talk from Pamela Fox (me) at YOW 2010 in Brisbane. Covers App Engine and the datastore, with Python examples.

Statistics

Views

Total Views
2,728
Views on SlideShare
2,728
Embed Views
0

Actions

Likes
5
Downloads
12
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Writing Apps the Google-y Way (Brisbane) Writing Apps the Google-y Way (Brisbane) Presentation Transcript

    • WRITING APPS THE GOOGLE-Y WAY Pamela Fox, YOW! Australia 2010 (Brisbane)
    • Who am I? twitter.com/pamelafox [email_address] pamelafox.org you get the idea...
    • Who am I? Google Maps API Google Wave API 2006 2010 2008 Google App Engine
    • Who am I? wave side projects 92 apps
    • Who am I? Java pYthon
    • What is App Engine?
      • “ Google App Engine enables you to build and host web apps on the same systems that power Google applications.”
      • http://code.google.com/appengine
    • What is a “web app”?
    • Static vs. Dynamic
    • Anonymous vs. Users
    • Intranet vs. Internet ~2 billion Hundreds - Thousands
    • What is a “web app”?
    • Some Google web apps
    • Some Google App Engine web apps www.gifttag.com www.buddypoke.com
    • Google apps on App Engine panoramio.com pubsubhubbub.appspot.com
    • How does App Engine work?
      • You upload application code & resources to Google.
      • Google serves your application from scalable infrastructure.
      • You pay for only the resources that Google used in serving the application.
    • Demo: Guestbook awesomest-app.appspot.com http://code.google.com/p/google-app-engine-samples/source/browse/trunk/guestbook appengine.google.com localhost build deploy monitor
    • App Engine architecture
    • App Engine architecture user task
    • App Engine architecture
    • App Engine architecture LIMIT CPU LIMIT Memory LIMIT Time
    • App Engine architecture hardware ports globals file system Groovy, JRuby, Mirah, Clojure, Scala
    • App Engine architecture 141,241,791 calls 1 GB data $0.15 GB/month 45,000,000 calls 657,000 - 46,000,000 calls *Always check docs for latest quotas. 192,672,000 calls 558 GB data $0.15 GB/month 7,000 - 1,700,000 calls $0.0001 per mail sent 46,000,000 calls 1,046 GB data sent 100,000 - 20,000,000 calls
    • The tricky bits
    • Datastore Entity Properties Key Entity Entity Entity Entity Path Kind Name/ID
    • Example: Speaker Entities Key Path Kind ID First Name Last Name Speaker1 - Speaker 1 Rod Johnson Key Path Kind ID First Name Last Name Middle Name Suffix Speaker2 - Speaker 2 Guy Steele L Jr.
    • Modeling Speaker Entities
      • class Speaker(db.model):
      • firstname = db.StringProperty(required=True)
      • lastname = db.StringProperty(required=True)
      • middlename = db.StringProperty()
      • namesuffix = db.StringProperty()
      • website = db.StringProperty()
      • keynote = db.BooleanProperty(default=False)
    • Saving Speaker Entities
      • rod = Speaker(firstname="Rod", lastname="Johnson")
      • guy = Speaker(firstname="Guy", lastname="Steele",
      • middlename="L", namesuffix="Jr.")
      • rod.put()
      • guy.put()
    • Updating Speaker Entities
      • rod = Speaker.get_by_id(1)
      • guy = Speaker.get_by_id(2)
      • rod.website = "http://www.sexyspring.com"
      • rod.keynote = True
      • guy.website = "http://www.lusciouslisp.com"
      • guy.keynote = True
      • db.put(rod, guy)
      LIMIT! (size/# of batch ops)
    • Queries & Indexes Query Index Index Index Index Query Query Query Query Query
    • Queries & Indexes SELECT * from Speaker ORDER BY lastname LIMIT! (# of results) key lastname Speaker3 Fox Speaker4 Hohpe Speaker1 Johnson Speaker2 Steele
    • Queries & Indexes SELECT * from Speaker ORDER by middlename key middlename Speaker2 L
    • Queries & Indexes SELECT * from Speaker WHERE keynote = True key keynote Speaker1 True Speaker2 True Speaker3 False Speaker4 False
    • Queries & Indexes SELECT * from Speaker WHERE keynote = False key keynote Speaker1 True Speaker2 True Speaker3 False Speaker4 False
    • Queries
      • allspeakers = Speaker.all().order('lastname)
      • for speaker in allspeakers:
      • print speaker.firstname + '' + speaker.lastname + '' + speaker.website
      • keynotespeakers = Speaker.all().filter('keynote = ', True)
      • notspecialspeakers = Speaker.all().filter('keynote = ', False)
      LIMIT! (size of results)
    • Custom Indexes
      • speakers = Speaker.all().order('lastname')
      • .order('keynote')
      SELECT * from Speaker ORDER BY lastname, keynote key lastname keynote Speaker3 Fox false Speaker4 Hohpe false Speaker1 Johnson true Speaker2 Steele true
    • Custom Indexes
      • speakers = Speaker.all().order('lastname')
      • .filter('keynote =', True)
      SELECT * from Speaker WHERE lastname > 'Johnson' and keynote = true key lastname keynote Speaker3 Fox false Speaker4 Hohpe false Speaker1 Johnson true Speaker2 Steele true
    • Impossible Indexes SELECT * from Speaker WHERE lastname < 'Steele' and firstname > 'Gregory' ...not in subsequent rows! key lastname firstname Speaker3 Fox Pamela Speaker4 Hohpe Gregory Speaker1 Johnson Rod Speaker2 Steele Guy
    • Impossible Indexes SELECT * from Speaker WHERE lastname > 'Fox' ORDER BY firstname ...not in the correct order! key lastname firstname Speaker3 Fox Pamela Speaker4 Hohpe Gregory Speaker1 Johnson Rod Speaker2 Steele Guy
    • Queries with Offset
      • speakers = Speaker.all().fetch(2, 2)
      SELECT * from Speaker LIMIT 2 OFFSET 2 1 2 ...slow! LIMIT! (# of offset) key lastname Speaker3 Fox Speaker4 Hohpe Speaker1 Johnson Speaker2 Steele
    • Queries with Cursors
      • query = db.Query(Speaker)
      • speakers = q.fetch(1000)
      • cursor = q.cursor()
      • memcache.set('speaker_cursor', cursor)
      • ...
      • last_cursor = memcache.get('speaker_cursor')
      • q.with_cursor(last_cursor)
      • speakers = q.fetch(1000)
    • More Properties class Talk(db.Model): title = db.StringProperty(required=True) abstract = db.TextProperty(required=True) speaker = db.ReferenceProperty(Speaker) tags = db.StringListProperty() pamela = Speaker.all().filter('firstname = ', 'Pamela').get() talk = Talk('Writing Apps the Googley Way', 'Bla bla bla', pamela, ['App Engine', 'Python']) talk.put() talk = Talk('Wonders of the Onesie', 'Bluh bluh bluh', pamela, ['Pajamas', 'Onesies']) talk.put()
    • Back-References pamela = Speaker.all().filter('firstname = ', 'Pamela').get() for talk in pamela.talk_set: print talk.title SELECT * from Talk WHERE speaker = Speaker3 key speaker Talk6 Speaker2 Talk1 Speaker3 Talk2 Speaker3 Talk5 Speaker4
    • Searching List Properties talks = Talk.all().filter('tags = ', 'python') .fetch(10) SELECT * from Talk WHERE tags = 'Python' LIMIT! (# of index rows) key lastname Talk1 App Engine Talk2 Pajamas Talk1 Python Talk2 Onesies
    • Update Transactions commit journal apply entities apply indexes A B
    • Entity Groups pamela = Speaker.all().filter('firstname = ', 'Pamela').get() talk1 = Talk('Writing Apps the Googley Way', 'Bla bla bla', pamela, ['App Engine', 'Python'], parent=pamela) talk2 = Talk('Wonders of the Onesie', 'Bluh bluh bluh', pamela, ['Pajamas', 'Onesies'], parent=pamela) db.put(talk1, talk2) def update_talks(): talk1.title = 'Writing Apps the Microsoft Way' talk2.title = 'Wonders of the Windows' db.put(talk1, talk2) db.run_in_transaction(update_talks)
    • Common Features
    • Counters 1 2 3 4 5 people have done something.
    • RageTube: Global Stats ragetube.net http://github.com/pamelafox/ragetube
    • RageTube: Global Stats SongStat yaycount viewcount title artist Key Path Kind Name (song) naycount mehcount
    • RageTube: Global Stats viewcount viewcount viewcount datastore memcache
    • RageTube: Global Stats class Song(db.Model): viewcount = db.IntegerProperty(default=0) title = db.StringProperty() artist = db.StringProperty() def get_viewcount(self): viewcount = self.viewcount cached_viewcount = memcache.get('viewcount-' + self.key().name(), self.key().kind()) if cached_viewcount: viewcount += cached_viewcount return viewcount @classmethod def flush_viewcount(cls, name): song = cls.get_by_key_name(name) value = memcache.get('viewcount-' + name, cls.kind()) memcache.decr('viewcount-' + name, value, cls.kind()) song.viewcount += value song.put() @classmethod def incr_viewcount(cls, name, interval=5, value=1): memcache.incr('viewcount-' + name, value, cls.kind()) interval_num = get_interval_number(datetime.now(), interval) task_name = '-'.join([cls.kind(), name.replace(' ', '-'), 'viewcount', str(interval), str(interval_num)]) deferred.defer(cls.flush_viewcount, name, _name=task_name) LIMIT! (# of tasks)
    • Ratings Rated by 500 users.
    • App Gallery: Ratings google.com/analytics/apps/
    • App Gallery: Ratings Comment Application total_ratings sum_ratings avg_rating rated_index comment_count rating
    • App Gallery: Ratings def UpdateAppCommentData(self, rating, operation): def UpdateCommentData(self, rating, operation): self.comment_count += 1 * operation self.sum_ratings += rating * operation self.total_ratings += 1 * operation self.avg_rating = int(round(self.sum_ratings / self.total_ratings)) self.rated_index = '%d:%d:%d' % (self.avg_rating, self.total_ratings, self.index) self.put() db.run_in_transaction(UpdateCommentData, self, rating, operation) app.UpdateAppCommentData(rating, db_models.Comment.ADD) comment = db_models.Comment() comment.application = app comment.rating = rating comment.put() query.order('-avg_rating').order('-rated_index')
    • Geospatial Queries
    • City-Go-Round: Agencies citygoround.org https://github.com/walkscore/City-Go-Round
    • City-Go-Round: Geo Queries Agency GeoModel location (GeoPt) location_geocells (StringListProperty)
    • City-Go-Round: Geo Queries def fetch_agencies_near(lat, long, bbox_side_in_miles): query = Agency.all() bbox = bbox_centered_at(lat, long, bbox_side_in_miles) return Agency.bounding_box_fetch(query, bbox, max_results = 50) def bounding_box_fetch(query, bbox, max_results=1000,): results = [] query_geocells = geocell.best_bbox_search_cells(bbox) for entity in query.filter('location_geocells IN', query_geocells): if len(results) == max_results: break if (entity.location.lat >= bbox.south and entity.location.lat <= bbox.north and entity.location.lon >= bbox.west and entity.location.lon <= bbox.east): results.append(entity) return results
    • Full Text Search
      • pizza
      Search Thingy It's like pizza, but in the cloud. Other Thingy This will make you smell as delicious as pizza.
    • Disclosed.ca: Search https://github.com/nurey/disclosed disclosed.ca
    • Disclosed.ca: Search Contract agency_name vendor_name description comments uri
    • Disclosed.ca: Search from search.core import SearchIndexProperty, porter_stemmer class Contract(db.Model): uri = db.StringProperty(required=True) agency_name = db.StringProperty(required=True) vendor_name = db.StringProperty(required=True) description = db.StringProperty() comments = db.TextProperty() search_index = SearchIndexProperty(('agency_name', 'vendor_name', 'description', 'comments'), indexer=porter_stemmer) results = Contract.search_index.search(sheep').fetch(20)
    • Disclosed.ca: Search Contract agency_name vendor_name description comments uri search_index (StringListProperty) SearchIndex
    • Disclosed.ca: Search SELECT FROM ContractSearch WHERE search_index = &quot;sheep&quot; key search_index ContractSearch1 charter ContractSearch1 june ContractSearch1 sheep ContractSearch2 sheep ContractSearch1 wood
    • More Learning http://ae-book.appspot.com http://code.google.com/appengine http://blog.notdot.net/
    • AppEngine: Now & Later &quot;Run your web apps on Google's infrastructure. Easy to build, easy to maintain, easy to scale.&quot;
      • Roadmap:
      • App Engine (Standard):
      • MapReduce
      • Bulk Import/Export
      • Channel API
      • Datastore Options
      • App Engine for Business:
      • SQL
      • SLA + Support
    • Thanks for coming!