Python 开源异步并发
框架的未来
About Those
Python Asynchronous Concurrency Frameworks
def about(self):
http://about.me/fantix
Python 3.4
胶水语言,内含电池
并行 vs. 并发
__name__ ==
„__main__‟
多个窗口
并行处理
照相1
照相2
照相3
取号2
取号1
办理1 办理2 办理3 办理4
照相1
照相2
照相3
取号2
取号1
办理1 办理2 办理3 办理4
17
23
16
5 7 3 6
12
15
最多
同时容纳
并行
并发
服务器
?
多进程/多线程
• “多进程” not in self.topic
• Python GIL
• PyPy STM
{
select(2),
epoll(7),
kqueue(2),
}
事件驱动
while True:
张三 = select(“10万客户端”)
try:
data = 张三.recv()
张三.send(data)
except EAGAIN:
pass
sock = socket.socket()
sock.setblocking(0)
sock.bind((“”, 80))
sock.listen(128)
def on_conn(fd, events):
conn, address = s...
from twisted.internet import protocol, reactor
class Echo(protocol.Protocol):
def dataReceived(self, data):
self.transport...
lol, Deferred
generator
@defer.inlineCallbacks
def main(endpoint, username="alice", password="secret"):
endpoint = endpoints.clientFromS...
greenlet
@defer.inlineCallbacks
def main(endpoint, username="alice", password=“secret”):
endpoint = endpoints.clientFromSt...
Eventlet / Gevent
from gevent import monkey
monkey.patch_all()
def handle(socket, address):
url = socket.recv()
data = url...
https://glyph.twistedmatrix.com/2014/02/unyielding.html
__feature__
callback generator greenlet Python 3
Twisted ✅ ✅ 57%
Tornado ✅ ✅ ✅
Gevent ✅ ✅ 80%
activity in day * 30
issues mails commits
Twisted 122 40 19 (258)
Tornado 31 36 17
Gevent 12 5 26
2014.03.18
互操作性-今天
Twisted
Tornado
Eventlet
/
Gevent
Stackless
互操作性-今天
redis postgresql mysql
Twisted txredisapi txpostgres txmysql
Tornado tornado-redis momoko amysql
Gevent <monkey> p...
from __future__
tulip / asyncio
by Guido van Rossum
Python 3.4
胶水语言,内含电池
像Twisted
callback
import asyncio
def print_and_repeat(loop):
print('Hello World')
loop.call_later(2, print_and_repeat, loop)
if __n...
protocol && transport
class MyServerUdpEchoProtocol:
def connection_made(self, transport):
print('start', transport)
self....
Future
In [1]: from asyncio.futures import Future
In [2]: import asyncio
In [3]: loop = asyncio.get_event_loop()
In [4]: f...
Task && Coroutine
import asyncio
from asyncio.tasks import Task
@asyncio.coroutine
def greet_every_two_seconds():
while Tr...
PEP 3156
asyncio
Event Loop
RI
apps libs
Third-party
Event Loop
adapter
原厂发动机
asyncio
Event Loop
RI
Twisted gevent
adapter adapter
“组装”发动机
asyncio
Twisted
Event Loop
Twisted
gevent
adapter
互操作性-明天
Twisted
Tornado
Eventlet
/
Gevent
Stackless
asyncio
txredisapi
txmysql
momokoamysql psycogreen
互操作性-明天
框架 适配 asyncio
Twisted 讨论中
Tornado 实验中
Gevent https://github.com/decentfox/gevent3
谢谢!
http://about.me/fantix
http://github.com/fantix
http://weibo.com/fantix
http://twitter.com/fantix
http://www.linkedin....
About Those Python Async Concurrent Frameworks - Fantix @ OSTC 2014
About Those Python Async Concurrent Frameworks - Fantix @ OSTC 2014
Upcoming SlideShare
Loading in …5
×

About Those Python Async Concurrent Frameworks - Fantix @ OSTC 2014

721
-1

Published on

Python 开源异步并发框架的未来
fantix @ OSTC 2014

Published in: Internet, Technology, Education
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
721
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
12
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

About Those Python Async Concurrent Frameworks - Fantix @ OSTC 2014

  1. 1. Python 开源异步并发 框架的未来 About Those Python Asynchronous Concurrency Frameworks
  2. 2. def about(self): http://about.me/fantix
  3. 3. Python 3.4 胶水语言,内含电池
  4. 4. 并行 vs. 并发 __name__ == „__main__‟
  5. 5. 多个窗口 并行处理
  6. 6. 照相1 照相2 照相3 取号2 取号1 办理1 办理2 办理3 办理4
  7. 7. 照相1 照相2 照相3 取号2 取号1 办理1 办理2 办理3 办理4 17 23 16 5 7 3 6 12 15
  8. 8. 最多 同时容纳
  9. 9. 并行
  10. 10. 并发 服务器 ?
  11. 11. 多进程/多线程 • “多进程” not in self.topic • Python GIL • PyPy STM
  12. 12. { select(2), epoll(7), kqueue(2), }
  13. 13. 事件驱动 while True: 张三 = select(“10万客户端”) try: data = 张三.recv() 张三.send(data) except EAGAIN: pass
  14. 14. sock = socket.socket() sock.setblocking(0) sock.bind((“”, 80)) sock.listen(128) def on_conn(fd, events): conn, address = sock.accept() conn.send(b‟Hello‟) io_loop = ioloop.IOLoop.instance() io_loop.add_handler(sock.fileno(), on_conn, io_loop.READ) io_loop.start()
  15. 15. from twisted.internet import protocol, reactor class Echo(protocol.Protocol): def dataReceived(self, data): self.transport.write(data) class EchoFactory(protocol.Factory): def buildProtocol(self, addr): return Echo() reactor.listenTCP(1234, EchoFactory()) reactor.run() 另外...
  16. 16. lol, Deferred
  17. 17. generator @defer.inlineCallbacks def main(endpoint, username="alice", password="secret"): endpoint = endpoints.clientFromString(reactor, strport) factory = protocol.Factory() factory.protocol = imap4.IMAP4Client try: endpoint.connect(factory)client = yield yield client.login(username, password) yield client.select('INBOX') info = yield client.fetchEnvelope(imap4.MessageSet(1)) print 'First message subject:', info[1]['ENVELOPE'][1] except: print "IMAP4 client interaction failed" failure.Failure().printTraceback() task.react(main, sys.argv[1:])
  18. 18. greenlet @defer.inlineCallbacks def main(endpoint, username="alice", password=“secret”): endpoint = endpoints.clientFromString(reactor, strport) factory = protocol.Factory() factory.protocol = imap4.IMAP4Client try: client = yield endpoint.connect(factory) yield client.login(username, password) yield client.select('INBOX') info = yield client.fetchEnvelope(imap4.MessageSet(1)) print 'First message subject:', info[1]['ENVELOPE'][1] except: print "IMAP4 client interaction failed" failure.Failure().printTraceback() task.react(main, sys.argv[1:]) def main(endpoint, username="alice", password=“secret”): endpoint = endpoints.clientFromString(reactor, strport) factory = protocol.Factory() factory.protocol = imap4.IMAP4Client try: client = endpoint.connect(factory) client.login(username, password) client.select('INBOX') info = client.fetchEnvelope(imap4.MessageSet(1)) print 'First message subject:', info[1]['ENVELOPE'][1] except: print "IMAP4 client interaction failed" failure.Failure().printTraceback() task.react(main, sys.argv[1:])
  19. 19. Eventlet / Gevent from gevent import monkey monkey.patch_all() def handle(socket, address): url = socket.recv() data = urllib.urlopen(url).read() socket.send(data) server = StreamServer(('127.0.0.1', 1234), handle) server.serve_forever()
  20. 20. https://glyph.twistedmatrix.com/2014/02/unyielding.html
  21. 21. __feature__ callback generator greenlet Python 3 Twisted ✅ ✅ 57% Tornado ✅ ✅ ✅ Gevent ✅ ✅ 80%
  22. 22. activity in day * 30 issues mails commits Twisted 122 40 19 (258) Tornado 31 36 17 Gevent 12 5 26 2014.03.18
  23. 23. 互操作性-今天 Twisted Tornado Eventlet / Gevent Stackless
  24. 24. 互操作性-今天 redis postgresql mysql Twisted txredisapi txpostgres txmysql Tornado tornado-redis momoko amysql Gevent <monkey> psycogreen greenify
  25. 25. from __future__
  26. 26. tulip / asyncio by Guido van Rossum
  27. 27. Python 3.4 胶水语言,内含电池
  28. 28. 像Twisted
  29. 29. callback import asyncio def print_and_repeat(loop): print('Hello World') loop.call_later(2, print_and_repeat, loop) if __name__ == '__main__': loop = asyncio.get_event_loop() print_and_repeat(loop) loop.run_forever()
  30. 30. protocol && transport class MyServerUdpEchoProtocol: def connection_made(self, transport): print('start', transport) self.transport = transport def datagram_received(self, data, addr): print('Data received:', data, addr) self.transport.sendto(data, addr) def error_received(self, exc): print('Error received:', exc) def connection_lost(self, exc): print('stop', exc)
  31. 31. Future In [1]: from asyncio.futures import Future In [2]: import asyncio In [3]: loop = asyncio.get_event_loop() In [4]: f = Future() In [5]: loop.call_later(2, f.set_result, 'OK') Out[5]: TimerHandle(74709.18277206, <bound method Future.set_result of Future<PENDING>>, ('OK',)) In [6]: loop.run_until_complete(f) Out[6]: 'OK'
  32. 32. Task && Coroutine import asyncio from asyncio.tasks import Task @asyncio.coroutine def greet_every_two_seconds(): while True: print('Hello World') yield from asyncio.sleep(2) if __name__ == '__main__': loop = asyncio.get_event_loop() t = Task(greet_every_two_seconds()) loop.run_until_complete(t)
  33. 33. PEP 3156 asyncio Event Loop RI apps libs Third-party Event Loop adapter
  34. 34. 原厂发动机 asyncio Event Loop RI Twisted gevent adapter adapter
  35. 35. “组装”发动机 asyncio Twisted Event Loop Twisted gevent adapter
  36. 36. 互操作性-明天 Twisted Tornado Eventlet / Gevent Stackless asyncio txredisapi txmysql momokoamysql psycogreen
  37. 37. 互操作性-明天 框架 适配 asyncio Twisted 讨论中 Tornado 实验中 Gevent https://github.com/decentfox/gevent3
  38. 38. 谢谢! http://about.me/fantix http://github.com/fantix http://weibo.com/fantix http://twitter.com/fantix http://www.linkedin.com/in/fantix

×