Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PyconKR 2018 Deep dive into Coroutine

1,464 views

Published on

It explains how coroutines work internally.

Published in: Software
  • Be the first to comment

PyconKR 2018 Deep dive into Coroutine

  1. 1. Deep Dive into Coroutine 김대희
  2. 2. Environment - OS : macOS High Sierra (10.13.6) - Version : CPython 3.6.5
  3. 3. # coroutine.py import asyncio async def coroutine1(): print(“coro1 first entry point”) await asyncio.sleep(1) print(“coro1 second entry point”) async def coroutine2(): print(“coro2 first entry point”) await asyncio.sleep(2) print(“coro2 second entry point”) loop = asyncio.get_event_loop() loop.create_task(coroutine1()) loop.create_task(coroutine2()) loop.run_forever()
  4. 4. # coroutine.py import asyncio async def coroutine1(): print(“coro1 first entry point”) await asyncio.sleep(1) print(“coro1 second entry point”) async def coroutine2(): print(“coro2 first entry point”) await asyncio.sleep(2) print(“coro2 second entry point”) loop = asyncio.get_event_loop() loop.create_task(coroutine1()) loop.create_task(coroutine2()) loop.run_forever() $ python coroutine.py coro1 first entry point coro2 first entry point coro1 second entry point coro2 second entry point→ → → → → → → →
  5. 5. # coroutine.py import asyncio async def coroutine1(): print(“coro1 first entry point”) await asyncio.sleep(1) print(“coro1 second entry point”) async def coroutine2(): print(“coro2 first entry point”) await asyncio.sleep(2) print(“coro2 second entry point”) loop = asyncio.get_event_loop() loop.create_task(coroutine1()) loop.create_task(coroutine2()) loop.run_forever() $ python coroutine.py coro1 first entry point coro2 first entry point coro1 second entry point coro2 second entry point
  6. 6. Multiple Entry Points
  7. 7. # coroutine.py import asyncio async def coroutine1(): print(“coro1 first entry point”) await asyncio.sleep(1) print(“coro1 second entry point”) async def coroutine2(): print(“coro2 first entry point”) await asyncio.sleep(2) print(“coro2 second entry point”) loop = asyncio.get_event_loop() loop.create_task(coroutine1()) loop.create_task(coroutine2()) loop.run_forever() → → → → → → → → → → → → → → Resuming(Entry points) - First lines of functions are entry points. - Lines after `await` can be resumed(entry points) if the `await` suspends. → → Suspending - `await` means it may suspend but not always.
  8. 8. In [1]: In [2]: In [3]: ...: ...: ...: ...: ...: ...: In [4]: In [5]: import inspect frame = None def func(): global frame x = 10 y = 20 print(x + y) frame = inspect.currentframe() func() 30 clear Frame object - Contains information for executing a function.
  9. 9. In [6]: f_localsframe. frame.f_locals frame.f_lasti frame.f_back frame.f_code frame.f_locals
  10. 10. f_locals - Local variables are stored In [6]: frame.f_locals Out[6]: {‘x’: 10, ‘y’: 20} In [7]:
  11. 11. In [6]: frame.f_locals Out[6]: {‘x’: 10, ‘y’: 20} In [7]: f_backframe. f_locals - Local variables are stored frame.f_locals frame.f_lasti frame.f_back frame.f_codeframe.f_back
  12. 12. f_locals - Local variables are stored f_back - Previous stack frame, this frame’s caller In [6]: frame.f_locals Out[6]: {‘x’: 10, ‘y’: 20} In [7]: frame.f_back Out[7]: <frame ...> ThreadState frame Frame f_back Frame f_back Frame f_back → Reference → Call
  13. 13. f_locals - Local variables are stored f_back - Previous stack frame, this frame’s caller In [6]: frame.f_locals Out[6]: {‘x’: 10, ‘y’: 20} In [7]: frame.f_back Out[7]: <frame ...> In [8]: frame.f_locals frame.f_lasti frame.f_back frame.f_code frame.f_lasti frame.f_lasti
  14. 14. f_locals - Local variables are stored f_back - Previous stack frame, this frame’s caller In [6]: frame.f_locals Out[6]: {‘x’: 10, ‘y’: 20} In [7]: frame.f_back Out[7]: <frame ...> In [8]: frame.f_lasti Out[8]: 30 In [9]:
  15. 15. In [1]: In [2]: In [1]: In [2]: In [3]: ...: ...: ...: ...: ...: ...: import inspect frame = None def func(): global frame x = 10 y = 20 print(x + y) frame = inspect.currentframe() import dis dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 1 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) . . 28 LOAD_CONST 0 (None) 30 RETURN_VALUE→ →
  16. 16. f_locals - Local variables are stored f_back - Previous stack frame, this frame’s caller f_lasti - Index of last attempted instruction in bytecode. In [6]: frame.f_locals Out[6]: {‘x’: 10, ‘y’: 20} In [7]: frame.f_back Out[7]: <frame ...> In [8]: frame.f_lasti Out[8]: 30 In [9]:
  17. 17. In [6]: frame.f_locals Out[6]: {‘x’: 10, ‘y’: 20} In [7]: frame.f_back Out[7]: <frame ...> In [8]: frame.f_lasti Out[8]: 30 In [9]: frame.f_locals frame.f_lasti frame.f_back frame.f_codeframe.f_code f_locals - Local variables are stored f_back - Previous stack frame, this frame’s caller f_lasti - Index of last attempted instruction in bytecode.frame..f_code
  18. 18. In [6]: frame.f_locals Out[6]: {‘x’: 10, ‘y’: 20} In [7]: frame.f_back Out[7]: <frame ...> In [8]: frame.f_lasti Out[8]: 30 In [9]: frame.f_code code.co_consts code.co_varnames code.co_names code.co_code f_locals - Local variables are stored f_back - Previous stack frame, this frame’s caller f_lasti - Index of last attempted instruction in bytecode.is func.__code__ Out[9]: True In[10]: code = frame.f_code In[11]: code.
  19. 19. In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE # test.py def func(): x = 10 y = 20 print(x + y) Code Line Number Bytecode index Opcode Operand Operand value
  20. 20. In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE # test.py def func(): x = 10 y = 20 print(x + y) print(func.__code__.co_code) >> b'dx01}x00dx02}x01tx00|x00|x01x17 x00x83x01x01x00dx00Sx00’ print(list(func.__code__.co_code)) >> [100, 1, 125, 0, 100, 2, 125, 1, 116, 0, 124, 0, 124, 1, 23, 0, 131, 1, 1, 0, 100, 0, 83, 0] import opcode print(opcode.opname[100]) >> ‘LOAD_CONST’
  21. 21. # test.py def func(): x = 10 y = 20 print(x + y) print(func.__code__.co_consts) >> (None, 10, 20) print(func.__code__.co_varnames) >> ('x', 'y') print(func.__code__.co_names) >> ('print’,) In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  22. 22. # test.py def func(): x = 10 y = 20 print(x + y) Value stack code.co_names 0 print frame.f_local code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y Mutable Immutable In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  23. 23. # test.py def func(): x = 10 y = 20 print(x + y) Value stack 10 frame.f_local Mutable Immutable code.co_names 0 print code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  24. 24. # test.py def func(): x = 10 y = 20 print(x + y) Value stack frame.f_local x 10 Mutable Immutable code.co_names 0 print code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  25. 25. # test.py def func(): x = 10 y = 20 print(x + y) Value stack 20 frame.f_local x 10 Mutable Immutable code.co_names 0 print code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  26. 26. # test.py def func(): x = 10 y = 20 print(x + y) Value stack code.co_names 0 print frame.f_local x 10 y 20 code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y Mutable Immutable In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  27. 27. # test.py def func(): x = 10 y = 20 print(x + y) Value stack print code.co_names 0 print frame.f_local x 10 y 20 code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y Mutable Immutable In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  28. 28. # test.py def func(): x = 10 y = 20 print(x + y) Value stack print 10 20 code.co_names 0 print frame.f_local x 10 y 20 code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y Mutable Immutable In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  29. 29. # test.py def func(): x = 10 y = 20 print(x + y) Value stack print 30 code.co_names 0 print frame.f_local x 10 y 20 code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y Mutable Immutable In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  30. 30. # test.py def func(): x = 10 y = 20 print(x + y) Value stack print 30 code.co_names 0 print frame.f_local x 10 y 20 code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y Mutable Immutable In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  31. 31. # test.py def func(): x = 10 y = 20 print(x + y) Value stack None code.co_names 0 print frame.f_local x 10 y 20 code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y Mutable Immutable In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  32. 32. # test.py def func(): x = 10 y = 20 print(x + y) Value stack code.co_names 0 print frame.f_local x 10 y 20 code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y Mutable Immutable In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  33. 33. # test.py def func(): x = 10 y = 20 print(x + y) Value stack None code.co_names 0 print frame.f_local x 10 y 20 code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y Mutable Immutable In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  34. 34. # test.py def func(): x = 10 y = 20 print(x + y) Value stack code.co_names 0 print frame.f_local x 10 y 20 code.co_consts 0 None 1 10 2 20 code.co_varnames 0 x 1 y Mutable Immutable In [1]: import dis In [2]: from test import func In [3]: dis.dis(func) 2 0 LOAD_CONST 1 (10) 2 STORE_FAST 0 (x) 3 4 LOAD_CONST 2 (20) 6 STORE_FAST 1 (y) 4 8 LOAD_GLOBAL 0 (print) 10 LOAD_FAST 0 (x) 12 LOAD_FAST 1 (y) 14 BINARY_ADD 16 CALL_FUNCION 1 18 POP_TOP 20 LOAD_CONST 0 (None) 22 RETURN_VALUE
  35. 35. f_locals - Local variables are stored f_back - Previous stack frame, this frame’s caller f_lasti - Index of last attempted instruction in bytecode. f_code - co_consts - Constant values - co_names - Global variable names - co_varnames - Local variable names - co_code - Complied bytecode In [6]: frame.f_locals Out[6]: {‘x’: 10, ‘y’: 20} In [7]: frame.f_back Out[7]: <frame ...> In [8]: frame.f_lasti Out[8]: 30 In [9]: frame.f_code is func.__code__ Out[9]: True In[10]: code = frame.f_code In[11]: code. code.co_consts code.co_varnames code.co_names code.co_code
  36. 36. In [1]: import dis In [2]: dis.dis(coroutine1) ... 3 8 LOAD_GLOBAL 1 (asyncio) 10 LOAD_ATTR 2 (sleep) 12 LOAD_CONST 2 (1) 14 CALL_FUNCTION 1 16 GET_AWAITABLE 18 LOAD_CONST 0 (None) 20 YIELD_FROM 22 POP_TOP ... import asyncio async def coroutine1(): print(“coro1 first entry point”) await asyncio.sleep(1) print(“coro1 second entry point”)
  37. 37. import asyncio @asyncio.coroutine def coroutine3(): print(“coro1 first entry point”) yield from asyncio.sleep(1) print(“coro1 second entry point”) In [1]: import dis In [2]: dis.dis(coroutine1) ... 3 8 LOAD_GLOBAL 1 (asyncio) 10 LOAD_ATTR 2 (sleep) 12 LOAD_CONST 2 (1) 14 CALL_FUNCTION 1 16 GET_YIELD_FROM_ITER 18 LOAD_CONST 0 (None) 20 YIELD_FROM 22 POP_TOP ...
  38. 38. YIELD_FROM - Delegating to a subgenerator - Both coroutines use ‘YIELD_FROM’.
  39. 39. def generator(): recv = yield 1 return recv gen = generator() gen.send(None) 1 gen.send(2) 2 In [1]: In [2]: Out[2]: In [3]: StopIteration:
  40. 40. def generator(): recv = yield 1 return recv 2 0 LOAD_CONST 1 (1) 2 YIELD_VALUE 4 STORE_FAST 0 (recv) 3 6 LOAD_FAST 0 (recv) 8 RETURN_VALUE In [1]: gen = generator() In [2]: gen.send(None) Out[2]: 1 lasti = gen.gi_frame.f_lasti lasti 2 code = gen.gi_code code is gen.gi_frame.f_code True op = code[lasti] import opcode opcode.opname[op] ‘YIELD_VALUE’ In [3]: In [4]: Out[4]: In [5]: In [6]: Out[6]: In [7]: In [8]: In [9]: Out[9]:
  41. 41. async def coroutine(): pass In [1]: coro = coroutine() In [2]: coro.cr_frame Out[2]: <frame at ...> In [3]: coro.cr_code Out[3]: <code object coroutine at ...>
  42. 42. In [1]: gen = generator() In [2]: gen.send(None) Out[2]: 1 In [3]: gen.send(2) StopIteration: 2 static PyObject *gen_send_ex(PyGenObject *gen, PyObject *arg, int exc, int closing) { PyThreadState *tstate = PyThreadState_GET(); PyFrameObject *f = gen->gi_frame; PyObject *result; result = arg ? arg : Py_None; Py_INCREF(result); *(f->f_stacktop++) = result; f->f_back = tstate->frame; result = PyEval_EvalFrameEx(f, exc); Py_CLEAR(f->f_back); ...
  43. 43. PyEval_EvalFrameEx - The code object associated with the execution frame is executed, interpreting bytecode and executing calls as needed. PyObject *PyEval_EvalFrameEx(PyFrameObject *f, int throwflag){ PyThreadState *tstate = PyThreadState_GET(); tstate->frame = f; main_loop: for (;;) { f->f_lasti = INSTR_OFFSET(); switch (opcode) { case: LOAD_FAST { ... } case: LOAD_CONST { ... } } tstate->frame = f->f_back; ...
  44. 44. Frame object - Used on executing a function - Contains information for executing a function - Call stack - Value stack - Local variables - Last attempted bytecode instruction Coroutine - Based on generator - Contains a frame object like thread state - The frame memorizes which index of bytecode is executed. - The frame stores local variables.
  45. 45. Frame f_back f_locals f_lasti f_code co_consts co_varnames co_names co_code value stack
  46. 46. Frame f_back f_locals f_lasti f_code co_consts co_varnames co_names co_code value stack
  47. 47. Frame f_back f_locals f_lasti f_code co_consts co_varnames co_names co_code value stack Frame f_bac f_loc f_las f_cod value
  48. 48. Frame f_back f_locals f_lasti ThreadState frame Coroutine gi_frame(cr_frame)
  49. 49. Event Loop for non-preemptive multitasking
  50. 50. Get an event loop from this thread. Schedule ‘coroutine1’ to the event loop as a task. Schedule ‘coroutine2’ to the event loop as a task. Run the event loop executing scheduled tasks. # coroutine.py import asyncio async def coroutine1(): print(“coro1 first entry point”) await asyncio.sleep(1) print(“coro1 second entry point”) async def coroutine2(): print(“coro2 first entry point”) await asyncio.sleep(2) print(“coro2 second entry point”) loop = asyncio.get_event_loop() loop.create_task(coroutine1()) loop.create_task(coroutine2()) loop.run_forever() loop = asyncio.get_event_loop() loop.create_task(coroutine1()) loop.create_task(coroutine2()) loop.run_forever()
  51. 51. # coroutine.py import asyncio async def coroutine1(): print(“coro1 first entry point”) await asyncio.sleep(1) print(“coro1 second entry point”) async def coroutine2(): print(“coro2 first entry point”) await asyncio.sleep(2) print(“coro2 second entry point”) loop = asyncio.get_event_loop() loop.create_task(coroutine1()) loop.create_task(coroutine2()) loop.run_forever() await asyncio.sleep(1) await asyncio.sleep(2) Get an event loop from this thread. Schedule ‘coroutine1’ to the event loop as a task. Schedule ‘coroutine2’ to the event loop as a task. Run the event loop executing scheduled tasks.
  52. 52. async def sleep(delay, result=None, *, loop=None): if delay <= 0: await __sleep0() return result if loop is None: loop = events.get_event_loop() future = loop.create_future() h = loop.call_later(delay, future.set_result, future, result) return await future @types.coroutine def __sleep0(): yield # the code has been modified for your better understanding
  53. 53. class Future: def __await__(self): if not self.done(): yield self # This tells Task to wait for completion. if not self.done(): raise RuntimeError("await wasn't used with future") return self.result() __iter__ = __await__ # make compatible with 'yield from’. # the code has been modified for your better understanding
  54. 54. class Task(Future): def _step(self): try: result = self.coro.send(None) except StopIteration as exc: self.set_result(exc.value) except Exception as exc: self.set_exception(exc) else: if result is None: self._loop.call_soon(self._step) elif isinstance(result, Future): result.add_done_callback(self._step) # the code has been modified for your better understanding
  55. 55. class Task(Future): def __init__(self, coro, *, loop=None): super().__init__(loop=loop) self._coro = coro self._loop.call_soon(self._step) ... # the code has been modified for your better understanding
  56. 56. class Future: def add_done_callback(self, fn): self._callbacks.append(fn) def set_result(self, result): self._result = result self._state = _FINISHED self._schedule_callbacks() def _schedule_callbacks(self): callbacks = self._callbacks[:] if not callbacks: return self._callbacks[:] = [] for callback in callbacks: self._loop.call_soon(callback, self) # the code has been modified for your better understanding
  57. 57. Event Loop Task._step if result is None Scheduled by call_soon Task result = coroutine.send(None)
  58. 58. Event Loop Task._step if result is None Scheduled by call_soon Task result = coroutine.send(None) Scheduled by call_soon
  59. 59. Event Loop Task._step if result is None Scheduled by call_soon Task result = coroutine.send(None) if result is Future Future Future.callbacksAdded by add_done_callback Future.set_result Scheduled by call_soon Scheduled by call_soon
  60. 60. class Handle: def __init__(self, callback, args, loop): self._callback = callback self._args = args self._loop = loop def _run(self): self._callback(*self._args) class TimerHandle(Handle): def __init__(self, when, callback, args, loop): super().__init__(callback, args, loop) self._when = when def __gt__(self, other): return self._when > other._when ... # the code has been modified for your better understanding
  61. 61. class CustomEventLoop(AbstractEventLoop): def create_future(self): return Future(loop=self) def create_task(self, coro): return Task(coro, loop=self) def time(self): return time.monotonic() def get_debug(self): pass def _timer_handle_cancelled(self, handle): pass # the code has been modified for your better understanding
  62. 62. class CustomEventLoop(AbstractEventLoop): def __init__(self): self._scheduled = [] self._ready = deque() def call_soon(self, callback, *args): handle = Handle(callback, args, self) self._ready.append(handle) return handle def call_later(self, delay, callback, *args): timer = self.call_at(self.time() + delay, callback, *args) return timer def call_at(self, when, callback, *args): timer = TimerHandle(when, callback, args, self) heappush(self._scheduled, timer) return timer # the code has been modified for your better understanding
  63. 63. class CustomEventLoop(AbstractEventLoop): def run_forever(self): while True: self._run_once() # the code has been modified for your better understanding
  64. 64. class CustomEventLoop(AbstractEventLoop): def _run_once(self): while self._scheduled and self._scheduled[0]._when <= self.time(): timer = heappop(self._scheduled) self._ready.append(timer) len_ready = len(self._ready) for _ in range(len_ready): handle = self._ready.popleft() handle._run() timeout = 0 if self._scheduled and not self._ready: timeout = max(0, self._scheduled[0]._when - self.time()) time.sleep(timeout) # the code has been modified for your better understanding
  65. 65. timeout = 0 if self._scheduled and not self._ready: timeout = max(0, self._scheduled[0]._when - self.time()) time.sleep(timeout) Spinning if ‘timeout’ is zero Freezing if ‘timeout’ is like infinity
  66. 66. Selector  SelectSelector  PollSelector  EpollSelector  KqueueSelector  DevpollSelector DefaultSelector – An alias to the most efficient implementation available on the current platform
  67. 67. # main thread import selectors import socket ssocket, csocket = socket.socketpair() ssocket.setblocking(False) csocket.setblocking(False) selector = selectors.DefaultSelector() selector.register(ssocket.fileno(), selectors.EVENT_READ) selector.select(timeout=None) # waiting ssocket.recv(1) # b’0’ # other threads csocket.send(b’0’)
  68. 68. class CustomEventLoop(AbstractEventLoop): def __init__(self): self._scheduled = [] self._ready = deque() self._selector = selectors.DefaultSelector() self._ssocket, self._csocket = socket.socketpair() self._ssocket.setblocking(False) self._csocket.setblocking(False) self._selector.register(self._ssocket.fileno(), selectors.EVENT_READ) def call_soon_threadsafe(self, callback, *args): handle = self.call_soon(callback, *args) self._csocket.send(b'0') return handle # the code has been modified for your better understanding
  69. 69. timeout = None if self._ready: timeout = 0 elif self._scheduled: timeout = max(0, self._scheduled[0]._when - self.time()) events = self._selector.select(timeout) if events: self._ssocket.recv(1) timeout = 0 if self._scheduled and not self._ready: timeout = max(0, self.time() - self._scheduled[0]._when) time.sleep(timeout) # the code has been modified for your better understanding
  70. 70. Demo
  71. 71. Thank you

×