Boredom comes to_those_who_wait
Upcoming SlideShare
Loading in...5
×
 

Boredom comes to_those_who_wait

on

  • 711 views

English version of the presentation on Google's ndb API for the App Engine datastore given at PythonBrasil 2011, on September 29th

English version of the presentation on Google's ndb API for the App Engine datastore given at PythonBrasil 2011, on September 29th

Statistics

Views

Total Views
711
Views on SlideShare
673
Embed Views
38

Actions

Likes
0
Downloads
1
Comments
0

3 Embeds 38

http://coderwall.com 31
http://www.linkedin.com 6
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Boredom comes to_those_who_wait Boredom comes to_those_who_wait Presentation Transcript

    • Boredom comes to thosewho waitAsynchronous calls to Datastore Plus
    • What you should know•toplevels/tasklets•futures•yield (you are going to build lots of generators)•youll use yield instead of return (most of the time)•dont think about threads•..._async•the query API changes•an insanely great use for comparator overrides
    • How it used to be doneclass Fact(model.Model):    text = model.TextProperty()    rating = model.FloatProperty(default = 400.)    random_index = model.ComputedProperty(        lambda self : random.randint(0,            sys.maxint))(...)for i in range(10):    Fact(text = Fact %d % i).put()
    • Digression: _ah/stats•A good way to understand the performance of your apps•If you are doing something wrong, itll become obviousEasy to enable on app.yaml:builtins:(...)- appstats: on
    • How the first examplebehaves locally
    • … and on the server
    • The asynchronous wayfutures = []for i in range(10):    futures.append(Fact(text = Fact %d %         i).put_async())[ f.get_result() for f in futures ] Gives the opportunity to aggregate puts into one large datastoreoperation (and we dont have to worry about it)
    • _ah/stats
    • On the server
    • Better: toplevel/tasklet@context.toplevel(decorating something – usually a request handler – that will call the following tasklet) @tasklets.taskletdef init_facts()  futures = []  for i in range(10):    futures.append(Fact(text = Fact %d %       i).put_async())  yield futures Yield allows the toplevel event loop to manage other generators making asynchronous calls
    • … even better@context.toplevel(decorating your handler)@tasklets.taskletdef init_facts() Futures = [] for i in range(10): futures.append(Fact(text = Fact %d % i).put_async()) raise tasklets.Return(futures)Because its considered polite to raise things when a generator has nothing else to generate
    • ab -n 10000 -c 50 (synchronous)Connection Times (ms) min  mean[+/-sd] median   maxConnect: 140 159 69.1 145 976Processing: 338 7408 5452.2 6231    46247Waiting: 338 7407 5452.2 6230    46247Total: 482 7567 5442.4 6377   46401Percentage of the requests served within a certain time (ms)  50%   6377  66%   8540  75%  10131  80%  11068  90%  13419  95%  16077  98%  23883  99%  30173 100%  46401 (longest request)
    • ab -n 10000 -c 50 (asynchronous)Connection Times (ms) min mean[+/-sd] median   maxConnect: 140 669 1375.6 151 21193Processing: 189 338 300.0 256 15320Waiting: 189 335 243.7 255 4143Total: 332 1007 1407.6 438 21450Percentage of the requests served within a certain time (ms)  50%    438  66%    565  75%    732  80%   1272  90%   3372  95%   3456  98%   3762  99%   9366 100%  21450 (longest request)
    • Whats the difference?
    • Wrongfor f in Fact.query(): f.rating = random.normalvariate(400, 20) f.put()
    • Why?
    • Right: map_async@tasklets.taskletdef randomize_rating(f):  f.rating = random.normalvariate(400, 20)  raise tasklets.Return(f.put_async())@context.topleveldef randomize_all():  Fact.query().map_async(randomize_rating)
    • Right: map_async
    • What else should I know?•context and its event loop•caches•new datatypes•new names for old types•repeated = True•StructuredProperty, LocalStructuredProperty•compress•shorter response times and more efficient instance usage
    • Where do I find it?•official builds•http://code.google.com/p/appengine-ndb-experiment/downloads/list•"Bleeding" edge•hg clone https://code.google.com/p/appengine-ndb-experiment/•version 0.7 released yesterday
    • To know more•documentation: http://code.google.com/p/appengine-ndb-experiment/•Google group: http://groups.google.com/group/appengine-ndb-discuss/
    • ThanksThanks to the fine people who hang out on the appengine-ndb-discussgroup, in special to Guido and Vladimir, whose suggestions pointed me onthe right direction.