Your SlideShare is downloading. ×

Asynchronous I/O in Python 3


Published on

Feihong talks about PEP 3156 and basic usage of Tulip, the reference implementation. …

Feihong talks about PEP 3156 and basic usage of Tulip, the reference implementation.


Source code:

Published in: Technology, Spiritual
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. Asychronous I/O in Python 3 Feihong Hsu Chicago Python User Group July 11, 2013
  • 2. What is PEP 3156? • "Asynchronous IO Support Rebooted" • Describes a new event loop API • Aims to be compatible with current popular event loops (Twisted,Tornado, etc.) • Should land in the standard library in 3.4 (early 2014)
  • 3. The big take-away We're going to be able to write asynchronous code without threads or callbacks!
  • 4. Threads vs event loop • I'm not going to talk about this • Twisted being around for more than a decade kinda validates the event loop model • For a detailed discussion see An Intro to Asynchronous Programming and Twisted
  • 5. Twisted vs Tulip Ugh, how to explain this? I could just give you a boring list of differences... Or I could use an analogy...
  • 6. Programming with Twisted is like... Being a mystical ninja warrior!
  • 7. Mystical Ninja Warrior • Can defeat demons and demigods by deploying an army of shadow clones • Mental gymnastics required to coordinate shadow clones requires years of brutal training • Ninja magic comes with a terrible price: the more clones you make, the more insane you become
  • 8. Programming with Tulip is like... Playing PlantsVs Zombies
  • 9. Playing PlantsVs Zombies • Can defeat a zombie horde by deploying and managing a large number of horticultural specimens • Not particularly mentally taxing • Somehow end up playing for hours without eating or sleeping
  • 10. Alrighty With the analogies out of the way, let's look at some examples.
  • 11. Fetching a web page in Twisted master/examples/
  • 12. Control flow diagram I was going to draw one but then I got sleepy.
  • 13. Problems with callback- based asynch code • Can be hard to understand control flow • Error messages are not friendly • Debugging is harder
  • 14. Fetching a web page in Tulip master/examples/
  • 15. Fetching a web page in Tulip w/ callbacks master/examples/3-tulip-download-
  • 16. Fetching a web page in Twisted w/ inline callbacks master/examples/4-twisted-download-inline-
  • 17. The Tulip API • Basics are documented in PEP 3156 • Not available in Python Package Index • Includes sockets, file I/O, etc. • Includes concurrency data structures like locks, queues, semaphores • Includes SSL and HTTP (with support for websockets!)
  • 18. Coroutines vs tasks When I first started, the most confusing thing about Tulip. Coroutines are executed by "yield from". Tasks are not executed by "yield from".
  • 19. Coroutine • Basically a function that contains at least one "yield from" statement. • Not the same thing as a generator function, which is a function that contains a yield expression. • Tulip will barf if you try to make it execute a generator.
  • 20. Coroutine @tulip.coroutine def download(url): response = yield from tulip.http.request('GET', url) return yield from Calling "download()" returns a generator object, but otherwise does nothing!You need to do "yield from download()" to run the body of a coroutine.
  • 21. Task • An object that manages a coroutine • Roughly equivalent to the Deferred object in Twisted
  • 22. Task @tulip.task def download(url): response = yield from tulip.http.request('GET', url) return yield from Calling "download()" actually does run the body of the function.The "yield from" part is done implicitly for you.
  • 23. Task In Tulip, there are two ways of creating tasks: tulip.task is a decorator that produces a task-wrapping function tulip.Task is a constructor that accepts a coroutine
  • 24. Why bother to use tasks? • To interoperate with callback-based frameworks like Twisted • To cancel an already-running coroutine • To start a new coroutine from within another coroutine
  • 25. Wait, so what's a Future? PEP 3156 makes frequent mention of Futures. But talking about Futures is a little confusing, since there are two Future classes: tulip.Future and concurrent.futures.Future.
  • 26. Future is the superclass of Task • Future don't necessarily manage a coroutine • In practice you never create Future objects, only Task objects • Futures are acceptable to yield from expressions
  • 27. Methods on Future • cancel(), cancelled() • running(), done() • result(), set_result() • add_done_callback(), remove_done_callback() • ...
  • 28. Web development with Tulip • All the classes you need are in the tulip.http module • Make subclass of tulip.http.ServerHttpProtocol • Override the handle_request() method
  • 29. Hello World! master/examples/
  • 30. Some observations The HTTP API is fairly simple, but a bit low level for everyday web programming. Expect a thousand microframeworks to bloom in the near future. Speaking of microframeworks...
  • 31. Introducing viol • Tiny web framework based on Tulip • After initial page load, messages between client and server are exchanged via websockets • Makes code demos a bit more visual
  • 32. Tulip API demos Now for a bunch of demos...
  • 33. Tulip API demos
  • 34. Serial version=3&autoplay=1&&start=1546
  • 35. Simultaneous version=3&autoplay=1&&start=1666
  • 36. Wait for both coroutines to finish version=3&autoplay=1&&start=1713
  • 37. Loop through coroutine results in order of finishing version=3&autoplay=1&&start=1869
  • 38. Execute synchronous code in another thread version=3&autoplay=1&&start=1930
  • 39. Talk to event loop from another thread version=3&autoplay=1&&start=2060
  • 40. A regex debugging tool version=3&autoplay=1&&start=2158
  • 41. Chat application version=3&autoplay=1&&start=2236
  • 42. Tulip Links • PEP 3156 • Guido's PyCon 2013 keynote • Tulip project page on GoogleCode
  • 43. Other links • An Introduction to Asynchronous Programming and Twisted • Code for this talk on Github • Viol project page on Github
  • 44. Questions?