The future of async i/o in Python

4,900 views

Published on

Task given at the Amsterdam Python Meetup Group, about PEP-3156 and Tulip.

Published in: Technology

The future of async i/o in Python

  1. 1. Saúl Ibarra Corretgé The future of async i/o in Python Amsterdam Python Meetup Group, 2nd October 2013 Thursday, October 3, 2013
  2. 2. __repr__(self) @saghul I work on VoIP and Real Time Communications (RTC) stuff Python is my (main) hammer Thursday, October 3, 2013
  3. 3. while True libuv, pyuv tulip, rose tornado-pyuv twisted-pyuv uvent evergreen Thursday, October 3, 2013
  4. 4. help(async_io) Blocking Is Bad (TM) Threads → overhead i/o multiplexors: “Iʼll let you know when you can write” Windows: “Iʼll let you know when Iʼm done writing” Thursday, October 3, 2013
  5. 5. import platform Thursday, October 3, 2013
  6. 6. raise NotImplementedError “Included batteries donʼt fit” asyncore, asyncchat Many frameworks Twisted Tornado Eventlet / Gevent ... Thursday, October 3, 2013
  7. 7. Thursday, October 3, 2013
  8. 8. class MyLoop Almost no reusable code between frameworks Multiple equivalent event loop implementations Multiple protocol implementations HTTP, anyone? Thursday, October 3, 2013
  9. 9. import tulip Thursday, October 3, 2013
  10. 10. import tulip PEP-3156: Async I/O support rebooted A set of components / primitives for async i/o Python >= 3.3 First version to ship with Python 3.4 (Boring) name for stdlib: asyncio Thursday, October 3, 2013
  11. 11. dir(tulip) Pluggable event loop Easy to build adapters Transport / protocol abstractions Inspired by Twisted High-level scheduler based on PEP-380 (yield from), Futures, Tasks Coroutine friendly Queue, Lock, Event, ... Thursday, October 3, 2013
  12. 12. print(“hello world”) from tulip import events def test(): print("Hello Tulip!") loop.stop() loop = events.new_event_loop() loop.call_soon(test) loop.run_forever() Thursday, October 3, 2013
  13. 13. from tulip import tasks from tulip import events, tasks @tasks.coroutine def sleeper(name, n): while True: yield from tasks.sleep(n, loop=loop) print('Hello from', name) loop = events.new_event_loop() t1 = tasks.async(sleeper('Task1', 1), loop=loop) t2 = tasks.async(sleeper('Task2', 2), loop=loop) loop.run_forever() Thursday, October 3, 2013
  14. 14. echo echo echo import os from tulip import events, protocols class EchoProtocol(protocols.Protocol): def connection_made(self, transport): print('Client connected') self.transport = transport def data_received(self, data): print('Received data:',data) self.transport.write(data) def connection_lost(self, exc): print('Connection closed', exc) def Factory(): return EchoProtocol() loop = events.new_event_loop() f = loop.start_serving(Factory, '127.0.0.1', int(os.getenv('PORT') or 1234)) sockets = loop.run_until_complete(f) print('Server started') loop.run_forever() Thursday, October 3, 2013
  15. 15. print(sys.modules) People already started implementing stuff with Tulip Redis client, websockets, Gunicorn worker, ... https://code.google.com/p/tulip/wiki/ ThirdParty Make your next library Tulip compatible! (bonus points if it has a nice flower name) Thursday, October 3, 2013
  16. 16. import __future__ All current frameworks would integrate with Tulip, directly or using an adapter Reusable protocols Integration with Twistedʼs Deferred ... Thursday, October 3, 2013
  17. 17. help() Checkout the code http://code.google.com/p/tulip/ Join the mailing list https://groups.google.com/forum/? fromgroups#!forum/python-tulip Thursday, October 3, 2013
  18. 18. input(‘Questions?’) bettercallsaghul.com Thursday, October 3, 2013
  19. 19. We are hiring! VoIP, Instant Messaging, Presence Python, C PyQt, PyObjC Windows, Linux, OSX ... http://ag-projects.com Thursday, October 3, 2013

×