• Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
3,694
On Slideshare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
134
Comments
0
Likes
12

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Transcript

  • 1. Python yield Mail & Popo: yangjunwei@ 1
  • 2. Agenda• why thinking in yield?• yield• yield• yield 2
  • 3. why thinking in yield? 3
  • 4. why thinking in yield?db.begin()if check_error1_happen(db): db.rollback() return Response(“error1 happens”)if check_error2_happen(db): db.rollback() return Response(“error2 happens”)db.table.update()....db.commit()return Response(“success”) 4
  • 5. why thinking in yield?• pythonic• bug 5
  • 6. why thinking in yield? @contextmanager def transaction(conn):class Job(object): trans = Job() def __init__(self): conn.begin() self._finished = False try: yield trans def is_finished(self): except: return self._finished conn.rollback() raise def finish(self): if trans.is_finished(): self._finished = True conn.commit() else: conn.rollback() 6
  • 7. why thinking in yield?with transaction(db) as trans: if check_error1_happen(db): return Response(“error1 happens”) if check_error2_happen(db): return Response(“error2 happens”) db.table.update() .... trans.finish()return Response(“success”) 7
  • 8. why thinking in yield?main main A yield function B C yield function 8
  • 9. yield 9
  • 10. yieldUsing a yield expression in a function definition is sufficient tocause that definition to create a generator function instead ofa normal function. 10
  • 11. yieldUsing a yield expression in a function definition is sufficient tocause that definition to create a generator function instead ofa normal function. >>> def f(): ... yield "hello world" ... >>> type(f()) <type generator> 10
  • 12. yieldThat generator then controls the execution of a generatorfunction. 11
  • 13. yieldThat generator then controls the execution of a generatorfunction. • send() • next() ==> send(None) • throw() • close() ==> throw(GeneratorExit) 11
  • 14. yieldyield is a expression, not a statement. New in python 2.5 12
  • 15. yieldyield is a expression, not a statement. New in python 2.5>>> def f():... a = yield "hello"... print a...>>> b = f()>>> b.send(None)hello>>> b.send("world")worldStopIteration 12
  • 16. yieldgenerator is also a iterator. 13
  • 17. yieldgenerator is also a iterator.>>> a = f()>>> a is a.__iter__()True 13
  • 18. yieldgenerator object 14
  • 19. yieldgenerator object>>> def f():... yield "hello world"...>>> a = f()>>> a.next()“hello world”>>> a.next()StopIteration>>> a.next()StopIteration 14
  • 20. yield 15
  • 21. yieldtstate_head next next PyThreadState PyThreadState ... frame PyFrameObject f_back PyFrameObject f_back python ... 16
  • 22. yield code block compile PyCodeObject MAKE_FUNCTIONPyFunctionObject CALL_FUNCTION PyFrameObject 17
  • 23. yield[frameobject.c]typedef struct { ... struct _frame *f_back; PyCodeObject *f_code; PyObject *f_builtins; PyObject *f_globals; PyObject *f_locals; PyObject *f_valuestack; PyObject *f_stacktop; ... int f_lasti; ...} PyFrameObject; 18
  • 24. yield[codeobject.c]typedef struct { ... int co_flags; ... PyObject *co_code; PyObject *co_consts; PyObject *co_names; ...} PyCodeObject; 19
  • 25. yield[a.py] [b.py]def f(): def f(): return “hello world” yield “hello world”f() f() PyCodeObject <==> code block 20
  • 26. yield4 0 LOAD_CONST 0 (<code object f ..>) 3 MAKE_FUNCTION 0 6 STORE_NAME 0 (f)7 9 LOAD_NAME 0 (f) 12 CALL_FUNCTION 0 15 POP_TOP 16 LOAD_CONST 1 (None) 19 RETURN_VALUE dis.dis(main_code) 21
  • 27. yield5 0 LOAD_CONST 1 (hello world) 3 RETURN_VALUE5 0 LOAD_CONST 1 (hello world) 3 YIELD_VALUE 4 POP_TOP 5 LOAD_CONST 0 (None) 8 RETURN_VALUE dis.dis(f_code) 22
  • 28. yield return generator 23
  • 29. yield[ceval.c] CALL_FUNCTIONf = PyFrame_New(tstate, co, globals, locals);...if (co->co_flags & CO_GENERATOR) { ... f->f_back = NULL; ... return PyGen_New(f);}retval = PyEval_EvalFrameEx(f,0); 24
  • 30. yield[frameobject.c]PyFrame_New(PyThreadState *tstate,PyCodeObject *code, PyObject *globals, PyObject *locals){ ... PyFrameObject *back = tstate->frame; PyFrameObject *f; f = PyObject_GC_Resize(PyFrameObject, f, extras); ... f->f_code = code; ... f->f_back = back; f->f_tstate = tstate; return f;} 25
  • 31. yield[ceval.c]PyObject *PyEval_EvalFrameEx(PyFrameObject *f, int throwflag){ ... PyThreadState *tstate = PyThreadState_GET(); tstate->frame = f; ... // exec opcode ... tstate->frame = f->f_back; return retval;} 26
  • 32. yieldtstate_head next PyThreadState ... frame A frame f_back B frame f_back ... 27
  • 33. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... 27
  • 34. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... 27
  • 35. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
  • 36. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
  • 37. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
  • 38. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
  • 39. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
  • 40. yieldtstate_head next PyThreadState ... frame A frame f_back B frame f_back ... yield 28
  • 41. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... yield 28
  • 42. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... yield 28
  • 43. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... yield 28
  • 44. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... yield 28
  • 45. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... yield 28
  • 46. yield[genobject.c]static PyObject * gen_send_ex(PyGenObject *gen, PyObject *arg, intexc) { PyFrameObject *f = gen->gi_frame; f->f_back = tstate->frame; ... if (f->f_lasti == -1) { .... } else { result = arg ? arg : Py_None; Py_INCREF(result); *(f->f_stacktop++) = result; } ... result = PyEval_EvalFrameEx(f, exc);} 29
  • 47. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back B frame(creator) f_back ... send 30
  • 48. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back B frame(creator) f_back ... send 30
  • 49. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back f_back B frame(creator) f_back ... send 30
  • 50. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back f_back B frame(creator) f_back ... send 30
  • 51. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back f_back B frame(creator) f_back ... send 30
  • 52. yieldresume frame 31
  • 53. yield[ceval.c]PyObject *PyEval_EvalFrameEx(PyFrameObject *f, int throwflag){ ... next_instr = first_instr + f->f_lasti + 1; stack_pointer = f->f_stacktop; assert(stack_pointer != NULL); f->f_stacktop = NULL; ... case YIELD_VALUE: retval = POP(); f->f_stacktop = stack_pointer; why = WHY_YIELD; goto fast_yield;} 32
  • 54. yield[genobject.c]static PyObject *gen_throw(PyGenObject *gen,PyObject *args){ ... PyErr_Restore(typ, val, tb); return gen_send_ex(gen, Py_None, 1);} throw 33
  • 55. yielddef close(self): try: self.throw(GeneratorExit) except (GeneratorExit, StopIteration): pass else: raise RuntimeError("generator ignored GeneratorExit") # Other exceptions are not caught close 34
  • 56. yield 35
  • 57. • xrange• with_statement“ ”• python Trampoline• generator coroutine 36
  • 58. xrange 37
  • 59. xrange[rangeobject.c]static PyObject *rangeiter_next(rangeiterobject *r){ if (r->index < r->len) return PyInt_FromLong(r->start + (r->index++) * r->step); return NULL;} xrange 38
  • 60. xrangedef xrange(start, stop, step=1): i = start - step while True: i = i + step if i < stop: yield i else: return yield 39
  • 61. xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10)) list generator 40
  • 62. xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10)) list generator 40
  • 63. xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10)) list generatordef f(): for x in xrange(10): yield x * x 40
  • 64. with_statement“ ” 41
  • 65. “with_statement” with with 42
  • 66. “with_statement” with withwith EXPR as VAR: BLOCK 42
  • 67. “with_statement” with withwith EXPR as VAR: BLOCK 42
  • 68. “with_statement”mgr = (EXPR)exit = type(mgr).__exit__ # Not calling it yetvalue = type(mgr).__enter__(mgr)exc = Truetry: try: VAR = value # Only if "as VAR" is present BLOCK except: exc = False if not exit(mgr, *sys.exc_info()): raisefinally: if exc: exit(mgr, None, None, None) 43
  • 69. “with_statement”class Transaction(object): def __init__(self, db): self._db = db self._trans = Job() def __enter__(self): self._db.begin() return self._trans def __exit__(self, type, value, traceback): if type is not None: self._db.rollback() else: self._db.commit() yield transaction 44
  • 70. “with_statement”@contextmanagerdef transaction(conn): trans = Job() conn.begin() try: yield trans except: conn.rollback() raise if trans.is_finished(): conn.commit() else: conn.rollback() 45
  • 71. “with_statement”@contextmanagerdef transaction(conn): trans = Job() conn.begin() try: yield trans except: conn.rollback() raise if trans.is_finished(): conn.commit() else: conn.rollback() 45
  • 72. python Trampoline 46
  • 73. python Trampolinedef f(n): if n < 2: return n else: return n + f(n - 1)>>> f(999)499500>>> f(1000)RuntimeError: maximum recursion depth exceeded 47
  • 74. python Trampolinesys.getrecursionlimit()Return the current value of the recursion limit, themaximum depth of the Python interpreter stack. Thislimit prevents infinite recursion from causing anoverflow of the C stack and crashing Python. It can beset by setrecursionlimit(). 48
  • 75. python Trampoline 49
  • 76. python Trampoline f(n) f(n) control f(n-1) f(n-1) f(n-2) scheduler f(n-2)f_back ... ... f(1) f(1) 50
  • 77. python Trampoline• let generator "call" other generator by yielding the generator they wish to invoke.• Any non-generator value yielded by a generator is returned to the generator that "called" the one yielding the value. 51
  • 78. python Trampolinedef f(n): if n < 2: yield n else: yield n + (yield f(n-1)) 52
  • 79. python Trampolinedef trampoline(main): stack = [] value = main while True: if type(value) is types.GeneratorType: stack.append(value) value = stack[-1].next() else: stack.pop() if stack: value = stack[-1].send(value) else: return value 53
  • 80. generator coroutine 54
  • 81. generator coroutineCoroutine( )" " " " 55
  • 82. generator coroutineCoroutine( )" " " " Pythons generator functions are almost coroutines. 55
  • 83. generator coroutinegenerator coroutine 56
  • 84. generator coroutinegenerator coroutine• coroutine “call” coroutine• coroutine scheduler 56
  • 85. generator coroutinecoroutine-based Python networking library• Eventlet(http://eventlet.net/)• Gevent(http://www.gevent.org/) 57
  • 86. Resources & References• python references: yield expression• python• PEP 342: Coroutines via Enhanced Generators• PEP 343: the with statement 58
  • 87. Q&A 59
  • 88. Thanks! 60