This document discusses Python asynchronous concurrency frameworks and their future. It compares Twisted, Tornado, Gevent, and the new asyncio module. Twisted and Tornado use callbacks while Gevent uses greenlets. asyncio aims to provide an event loop like Twisted. It also introduces coroutines, tasks, and futures. The document argues that asyncio could serve as a common event loop that existing frameworks adapt to for better interoperability in the future.

1. Python 开源异步并发
框架的未来
About Those
Python Asynchronous Concurrency Frameworks

2. def about(self):
http://about.me/fantix

3. Python 3.4
胶水语言，内含电池

4. 并行 vs. 并发
__name__ ==
„__main__‟

7. 多个窗口
并行处理

8. 照相1
照相2
照相3
取号2
取号1
办理1 办理2 办理3 办理4

9. 照相1
照相2
照相3
取号2
取号1
办理1 办理2 办理3 办理4
17
23
16
5 7 3 6
12
15

10. 最多
同时容纳

11. 并行

12. 并发
服务器
?

13. 多进程／多线程
• “多进程” not in self.topic
• Python GIL
• PyPy STM

14. {
select(2),
epoll(7),
kqueue(2),
}

15. 事件驱动
while True:
张三 = select(“10万客户端”)
try:
data = 张三.recv()
张三.send(data)
except EAGAIN:
pass

16. sock = socket.socket()
sock.setblocking(0)
sock.bind((“”, 80))
sock.listen(128)
def on_conn(fd, events):
conn, address = sock.accept()
conn.send(b‟Hello‟)
io_loop = ioloop.IOLoop.instance()
io_loop.add_handler(sock.fileno(), on_conn, io_loop.READ)
io_loop.start()

17. from twisted.internet import protocol, reactor
class Echo(protocol.Protocol):
def dataReceived(self, data):
self.transport.write(data)
class EchoFactory(protocol.Factory):
def buildProtocol(self, addr):
return Echo()
reactor.listenTCP(1234, EchoFactory())
reactor.run()
另外...

18. lol, Deferred

19. generator
@defer.inlineCallbacks
def main(endpoint, username="alice", password="secret"):
endpoint = endpoints.clientFromString(reactor, strport)
factory = protocol.Factory()
factory.protocol = imap4.IMAP4Client
try:
endpoint.connect(factory)client = yield
yield client.login(username, password)
yield client.select('INBOX')
info = yield client.fetchEnvelope(imap4.MessageSet(1))
print 'First message subject:', info[1]['ENVELOPE'][1]
except:
print "IMAP4 client interaction failed"
failure.Failure().printTraceback()
task.react(main, sys.argv[1:])

20. greenlet
@defer.inlineCallbacks
def main(endpoint, username="alice", password=“secret”):
endpoint = endpoints.clientFromString(reactor, strport)
factory = protocol.Factory()
factory.protocol = imap4.IMAP4Client
try:
client = yield endpoint.connect(factory)
yield client.login(username, password)
yield client.select('INBOX')
info = yield client.fetchEnvelope(imap4.MessageSet(1))
print 'First message subject:', info[1]['ENVELOPE'][1]
except:
print "IMAP4 client interaction failed"
failure.Failure().printTraceback()
task.react(main, sys.argv[1:])
def main(endpoint, username="alice", password=“secret”):
endpoint = endpoints.clientFromString(reactor, strport)
factory = protocol.Factory()
factory.protocol = imap4.IMAP4Client
try:
client = endpoint.connect(factory)
client.login(username, password)
client.select('INBOX')
info = client.fetchEnvelope(imap4.MessageSet(1))
print 'First message subject:', info[1]['ENVELOPE'][1]
except:
print "IMAP4 client interaction failed"
failure.Failure().printTraceback()
task.react(main, sys.argv[1:])

21. Eventlet / Gevent
from gevent import monkey
monkey.patch_all()
def handle(socket, address):
url = socket.recv()
data = urllib.urlopen(url).read()
socket.send(data)
server = StreamServer(('127.0.0.1', 1234), handle)
server.serve_forever()

22. https://glyph.twistedmatrix.com/2014/02/unyielding.html

23. __feature__
callback generator greenlet Python 3
Twisted ✅ ✅ 57%
Tornado ✅ ✅ ✅
Gevent ✅ ✅ 80%

24. activity in day * 30
issues mails commits
Twisted 122 40 19 (258)
Tornado 31 36 17
Gevent 12 5 26
2014.03.18

25. 互操作性－今天
Twisted
Tornado
Eventlet
/
Gevent
Stackless

26. 互操作性－今天
redis postgresql mysql
Twisted txredisapi txpostgres txmysql
Tornado tornado-redis momoko amysql
Gevent <monkey> psycogreen greenify

27. from __future__

28. tulip / asyncio
by Guido van Rossum

29. Python 3.4
胶水语言，内含电池

30. 像Twisted

31. callback
import asyncio
def print_and_repeat(loop):
print('Hello World')
loop.call_later(2, print_and_repeat, loop)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
print_and_repeat(loop)
loop.run_forever()

32. protocol && transport
class MyServerUdpEchoProtocol:
def connection_made(self, transport):
print('start', transport)
self.transport = transport
def datagram_received(self, data, addr):
print('Data received:', data, addr)
self.transport.sendto(data, addr)
def error_received(self, exc):
print('Error received:', exc)
def connection_lost(self, exc):
print('stop', exc)

33. Future
In [1]: from asyncio.futures import Future
In [2]: import asyncio
In [3]: loop = asyncio.get_event_loop()
In [4]: f = Future()
In [5]: loop.call_later(2, f.set_result, 'OK')
Out[5]: TimerHandle(74709.18277206, <bound method
Future.set_result of Future<PENDING>>, ('OK',))
In [6]: loop.run_until_complete(f)
Out[6]: 'OK'

34. Task && Coroutine
import asyncio
from asyncio.tasks import Task
@asyncio.coroutine
def greet_every_two_seconds():
while True:
print('Hello World')
yield from asyncio.sleep(2)
if __name__ == '__main__':
loop = asyncio.get_event_loop()
t = Task(greet_every_two_seconds())
loop.run_until_complete(t)

35. PEP 3156
asyncio
Event Loop
RI
apps libs
Third-party
Event Loop
adapter

36. 原厂发动机
asyncio
Event Loop
RI
Twisted gevent
adapter adapter

37. “组装”发动机
asyncio
Twisted
Event Loop
Twisted
gevent
adapter

38. 互操作性－明天
Twisted
Tornado
Eventlet
/
Gevent
Stackless
asyncio
txredisapi
txmysql
momokoamysql psycogreen

39. 互操作性－明天
框架 适配 asyncio
Twisted 讨论中
Tornado 实验中
Gevent https://github.com/decentfox/gevent3

40. 谢谢！
http://about.me/fantix
http://github.com/fantix
http://weibo.com/fantix
http://twitter.com/fantix
http://www.linkedin.com/in/fantix