The document discusses patterns for asynchronous flow control using Twisted deferreds. It introduces common building blocks like succeed, fail, maybeDeferred, DeferredList, and DeferredQueue. It then describes classes that allow replacing synchronous data structures like queues with asynchronous equivalents. Specific classes discussed include tee, RetryingCall, DeferredPooler, ControllableDeferred, DeferredPool, and ResizableDispatchQueue.

1. Deferred Gratification
Patterns of asynchronous flow of control
with Twisted deferreds
Terry Jones
terry@fluidinfo.com
@terrycojones

2. Not an intro talk. Sorry!
• Assume you understand Deferreds
• Assume you understand async

3. Read defer.py
• Lots of people don’t
• Much of it is beautiful
• Deferreds seem cryptic
• They’re are actually very simple

4. Building blocks
Simple functions that help with deferreds
• succeed
• fail
• maybeDeferred
Classes that build on simple deferreds
• DeferredList
• DeferredQueue
• DeferredSemaphore
• DeferredLock

5. Deferreds let you replace
synchronous data structures
with elegant asynchronous
equivalents

6. Some of our classes
http://jon.es/other/europython-2010-deferreds.tar.bz2
• tee
• RetryingCall
• DeferredPooler
}
• ControllableDeferred
• DeferredPool rdq.py
• ResizableDispatchQueue

7. DeferredPooler
client
client server whatever
client

8. DeferredPooler
client fe
tc
h(
x,
y)
client server whatever
client

9. DeferredPooler
client fe
tc
h(
x,
y)
fetch(x,y)
client server whatever
client

10. DeferredPooler
client fe
tc
h(
x,
y)
fetch(x,y) fetch(x,y)
client server whatever
client

11. DeferredPooler
client fe
tc
h(
x,
y)
fetch(x,y) fetch(x,y)
client server whatever
, y)
( x
ch
f et
client

12. DeferredPooler
client fe
tc
h(
x,
y)
fetch(x,y) fetch(x,y)
client server whatever
result
, y)
( x
ch
f et
client

13. DeferredPooler
client fe
tc
re h(
su x,
lt y)
fetch(x,y) fetch(x,y)
client server whatever
result result
, y)
x
h(
tc
fe ult
r es
client

14. Normal Deferred
1 d = someFunc(arg1,arg2)
cb1 cb2 cb3
2 d.addCallback()
cb1 cb2 cb3 cb4 cb5
3 # Hope d fires!

15. Controllable Deferred
1 d = ControllableDeferred(someFunc,arg1,arg2)
cb1 cb2 cb3
2 d.addCallback()
cb1 cb2 cb3 ? cb4 cb5
3 # If call to someFunc fires, call d.
# Allow caller to call d.cancel

16. RetryingCall
Replace something like this
d = getPage(url)
With this
r = RetryingCall(getPage, url)
d = r.start()

17. Resizable Dispatch Queue
Tickery adder RDQ
pending
WWW put put a b c d
underway
Twitter e
doWork f
FluidDB call g
admin pause, resume, stop, size,
widen, narrow, underway, cancel