Boredom comes to thosewho waitAsynchronous calls to Datastore Plus
What you should know•toplevels/tasklets•futures•yield (you are going to build lots of generators)•youll use yield instead ...
How it used to be doneclass Fact(model.Model):    text = model.TextProperty()    rating = model.FloatProperty(default = 40...
Digression: _ah/stats•A good way to understand the performance of your apps•If you are doing something wrong, itll become ...
How the first examplebehaves locally
… and on the server
The asynchronous wayfutures = []for i in range(10):    futures.append(Fact(text = Fact %d %         i).put_async())[ f.get...
_ah/stats
On the server
Better: toplevel/tasklet@context.toplevel(decorating something – usually a request handler – that will call the following ...
… even better@context.toplevel(decorating your handler)@tasklets.taskletdef init_facts()     Futures = []     for i in ran...
ab -n 10000 -c 50 (synchronous)Connection Times (ms)              min  mean[+/-sd] median   maxConnect:      140   159   6...
ab -n 10000 -c 50 (asynchronous)Connection Times (ms)              min mean[+/-sd]    median   maxConnect:      140   669 ...
Whats the difference?
Wrongfor f in Fact.query():   f.rating = random.normalvariate(400, 20)   f.put()
Why?
Right: map_async@tasklets.taskletdef randomize_rating(f):  f.rating = random.normalvariate(400, 20)  raise tasklets.Return...
Right: map_async
What else should I know?•context and its event loop•caches•new datatypes•new names for old types•repeated = True•Structure...
Where do I find it?•official builds•http://code.google.com/p/appengine-ndb-experiment/downloads/list•"Bleeding" edge•hg cl...
To know more•documentation: http://code.google.com/p/appengine-ndb-experiment/•Google group: http://groups.google.com/grou...
ThanksThanks to the fine people who hang out on the appengine-ndb-discussgroup, in special to Guido and Vladimir, whose su...
Upcoming SlideShare
Loading in …5
×

Boredom comes to_those_who_wait

832 views

Published on

English version of the presentation on Google's ndb API for the App Engine datastore given at PythonBrasil 2011, on September 29th

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
832
On SlideShare
0
From Embeds
0
Number of Embeds
65
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Boredom comes to_those_who_wait

  1. 1. Boredom comes to thosewho waitAsynchronous calls to Datastore Plus
  2. 2. What you should know•toplevels/tasklets•futures•yield (you are going to build lots of generators)•youll use yield instead of return (most of the time)•dont think about threads•..._async•the query API changes•an insanely great use for comparator overrides
  3. 3. How it used to be doneclass Fact(model.Model):    text = model.TextProperty()    rating = model.FloatProperty(default = 400.)    random_index = model.ComputedProperty(        lambda self : random.randint(0,            sys.maxint))(...)for i in range(10):    Fact(text = Fact %d % i).put()
  4. 4. Digression: _ah/stats•A good way to understand the performance of your apps•If you are doing something wrong, itll become obviousEasy to enable on app.yaml:builtins:(...)- appstats: on
  5. 5. How the first examplebehaves locally
  6. 6. … and on the server
  7. 7. The asynchronous wayfutures = []for i in range(10):    futures.append(Fact(text = Fact %d %         i).put_async())[ f.get_result() for f in futures ] Gives the opportunity to aggregate puts into one large datastoreoperation (and we dont have to worry about it)
  8. 8. _ah/stats
  9. 9. On the server
  10. 10. Better: toplevel/tasklet@context.toplevel(decorating something – usually a request handler – that will call the following tasklet) @tasklets.taskletdef init_facts()  futures = []  for i in range(10):    futures.append(Fact(text = Fact %d %       i).put_async())  yield futures Yield allows the toplevel event loop to manage other generators making asynchronous calls
  11. 11. … even better@context.toplevel(decorating your handler)@tasklets.taskletdef init_facts() Futures = [] for i in range(10): futures.append(Fact(text = Fact %d % i).put_async()) raise tasklets.Return(futures)Because its considered polite to raise things when a generator has nothing else to generate
  12. 12. ab -n 10000 -c 50 (synchronous)Connection Times (ms) min  mean[+/-sd] median   maxConnect: 140 159 69.1 145 976Processing: 338 7408 5452.2 6231    46247Waiting: 338 7407 5452.2 6230    46247Total: 482 7567 5442.4 6377   46401Percentage of the requests served within a certain time (ms)  50%   6377  66%   8540  75%  10131  80%  11068  90%  13419  95%  16077  98%  23883  99%  30173 100%  46401 (longest request)
  13. 13. ab -n 10000 -c 50 (asynchronous)Connection Times (ms) min mean[+/-sd] median   maxConnect: 140 669 1375.6 151 21193Processing: 189 338 300.0 256 15320Waiting: 189 335 243.7 255 4143Total: 332 1007 1407.6 438 21450Percentage of the requests served within a certain time (ms)  50%    438  66%    565  75%    732  80%   1272  90%   3372  95%   3456  98%   3762  99%   9366 100%  21450 (longest request)
  14. 14. Whats the difference?
  15. 15. Wrongfor f in Fact.query(): f.rating = random.normalvariate(400, 20) f.put()
  16. 16. Why?
  17. 17. Right: map_async@tasklets.taskletdef randomize_rating(f):  f.rating = random.normalvariate(400, 20)  raise tasklets.Return(f.put_async())@context.topleveldef randomize_all():  Fact.query().map_async(randomize_rating)
  18. 18. Right: map_async
  19. 19. What else should I know?•context and its event loop•caches•new datatypes•new names for old types•repeated = True•StructuredProperty, LocalStructuredProperty•compress•shorter response times and more efficient instance usage
  20. 20. Where do I find it?•official builds•http://code.google.com/p/appengine-ndb-experiment/downloads/list•"Bleeding" edge•hg clone https://code.google.com/p/appengine-ndb-experiment/•version 0.7 released yesterday
  21. 21. To know more•documentation: http://code.google.com/p/appengine-ndb-experiment/•Google group: http://groups.google.com/group/appengine-ndb-discuss/
  22. 22. ThanksThanks to the fine people who hang out on the appengine-ndb-discussgroup, in special to Guido and Vladimir, whose suggestions pointed me onthe right direction.

×