Your SlideShare is downloading. ×
0
Python yield     Mail & Popo: yangjunwei@     1
Agenda•   why thinking in yield?•   yield•   yield•   yield                             2
why thinking in yield?          3
why thinking in yield?db.begin()if check_error1_happen(db):    db.rollback()    return Response(“error1 happens”)if check_...
why thinking in yield?•     pythonic•   bug                      5
why thinking in yield?                                      @contextmanager                                      def trans...
why thinking in yield?with transaction(db) as trans:   if check_error1_happen(db):       return Response(“error1 happens”)...
why thinking in yield?main                         main           A                              yield function       B   ...
yield        9
yieldUsing a yield expression in a function definition is sufficient tocause that definition to create a generator function i...
yieldUsing a yield expression in a function definition is sufficient tocause that definition to create a generator function i...
yieldThat generator then controls the execution of a generatorfunction.                              11
yieldThat generator then controls the execution of a generatorfunction.  •   send()  •   next() ==> send(None)  •   throw(...
yieldyield is a expression, not a statement. New in python 2.5                               12
yieldyield is a expression, not a statement. New in python 2.5>>> def f():... a = yield "hello"... print a...>>> b = f()>>...
yieldgenerator is also a iterator.                                13
yieldgenerator is also a iterator.>>> a = f()>>> a is a.__iter__()True                                13
yieldgenerator object                       14
yieldgenerator object>>> def f():... yield "hello world"...>>> a = f()>>> a.next()“hello world”>>> a.next()StopIteration>>...
yield        15
yieldtstate_head                    next                   next    PyThreadState          PyThreadState          ...   fra...
yield   code block              compile PyCodeObject              MAKE_FUNCTIONPyFunctionObject              CALL_FUNCTION...
yield[frameobject.c]typedef struct {   ...   struct _frame *f_back;   PyCodeObject *f_code;   PyObject *f_builtins;   PyOb...
yield[codeobject.c]typedef struct {   ...   int co_flags;   ...   PyObject *co_code;   PyObject *co_consts;   PyObject *co_...
yield[a.py]                        [b.py]def f():                      def f():  return “hello world”          yield “hell...
yield4   0 LOAD_CONST            0 (<code object f ..>)    3 MAKE_FUNCTION         0    6 STORE_NAME            0 (f)7    ...
yield5   0 LOAD_CONST              1 (hello world)    3 RETURN_VALUE5   0 LOAD_CONST              1 (hello world)    3 YIE...
yield  return generator        23
yield[ceval.c] CALL_FUNCTIONf = PyFrame_New(tstate, co, globals, locals);...if (co->co_flags & CO_GENERATOR) {    ...    f-...
yield[frameobject.c]PyFrame_New(PyThreadState *tstate,PyCodeObject *code, PyObject *globals, PyObject *locals){   ...   Py...
yield[ceval.c]PyObject *PyEval_EvalFrameEx(PyFrameObject *f,  int throwflag){  ...  PyThreadState *tstate = PyThreadState_G...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame   f...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame   f...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yieldtstate_head                          next    PyThreadState                ...                  frame      A frame    ...
yield[genobject.c]static PyObject * gen_send_ex(PyGenObject *gen, PyObject *arg, intexc) {   PyFrameObject *f = gen->gi_fr...
yieldtstate_head                          next    PyThreadState                ...                  frame   A frame(caller...
yieldtstate_head                          next    PyThreadState                ...                  frame   A frame(caller...
yieldtstate_head                          next    PyThreadState                ...                  frame   A frame(caller...
yieldtstate_head                          next    PyThreadState                ...                  frame   A frame(caller...
yieldtstate_head                          next    PyThreadState                ...                  frame   A frame(caller...
yieldresume frame                 31
yield[ceval.c]PyObject *PyEval_EvalFrameEx(PyFrameObject *f,  int throwflag){  ...  next_instr = first_instr + f->f_lasti + ...
yield[genobject.c]static PyObject *gen_throw(PyGenObject *gen,PyObject *args){   ...   PyErr_Restore(typ, val, tb); return...
yielddef close(self):  try:      self.throw(GeneratorExit)  except (GeneratorExit, StopIteration):      pass  else:      r...
yield    35
•   xrange•     with_statement“          ”•   python Trampoline•     generator    coroutine                        36
xrange         37
xrange[rangeobject.c]static PyObject *rangeiter_next(rangeiterobject *r){ if (r->index < r->len)  return PyInt_FromLong(r-...
xrangedef xrange(start, stop, step=1):  i = start - step  while True:      i = i + step     if i < stop:          yield i ...
xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10))           list           generator                  40
xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10))           list           generator                  40
xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10))           list            generatordef f():    for x in x...
with_statement“   ”          41
“with_statement”      with        with             42
“with_statement”           with        withwith EXPR as VAR:   BLOCK                  42
“with_statement”           with        withwith EXPR as VAR:   BLOCK                  42
“with_statement”mgr = (EXPR)exit = type(mgr).__exit__ # Not calling it yetvalue = type(mgr).__enter__(mgr)exc = Truetry:  ...
“with_statement”class Transaction(object):    def __init__(self, db):        self._db = db        self._trans = Job()  def...
“with_statement”@contextmanagerdef transaction(conn):  trans = Job()  conn.begin()  try:     yield trans  except:     conn...
“with_statement”@contextmanagerdef transaction(conn):  trans = Job()  conn.begin()  try:     yield trans  except:     conn...
python Trampoline        46
python Trampolinedef f(n):    if n < 2:         return n    else:         return n + f(n - 1)>>> f(999)499500>>> f(1000)Ru...
python Trampolinesys.getrecursionlimit()Return the current value of the recursion limit, themaximum depth of the Python in...
python Trampoline        49
python Trampoline         f(n)                               f(n)                                  control     f(n-1)     ...
python Trampoline•    let generator "call" other generator by yielding    the generator they wish to invoke.•    Any non-g...
python Trampolinedef f(n):  if n < 2:      yield n  else:      yield n + (yield f(n-1))                           52
python Trampolinedef trampoline(main):  stack = []  value = main  while True:      if type(value) is types.GeneratorType: ...
generator        coroutine            54
generator        coroutineCoroutine(           )"    "   "   "                         55
generator         coroutineCoroutine(                 )"       "   "     "    Pythons generator functions are almost corou...
generator        coroutinegenerator   coroutine                        56
generator        coroutinegenerator     coroutine•   coroutine “call” coroutine•   coroutine scheduler                    ...
generator        coroutinecoroutine-based Python networking library•   Eventlet(http://eventlet.net/)•   Gevent(http://www...
Resources & References•   python references: yield expression•   python•   PEP 342: Coroutines via Enhanced Generators•   ...
Q&A 59
Thanks!          60
Upcoming SlideShare
Loading in...5
×

Python Yield

3,880

Published on

Python yield

Published in: Technology
0 Comments
13 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,880
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
140
Comments
0
Likes
13
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript of "Python Yield"

    1. 1. Python yield Mail & Popo: yangjunwei@ 1
    2. 2. Agenda• why thinking in yield?• yield• yield• yield 2
    3. 3. why thinking in yield? 3
    4. 4. why thinking in yield?db.begin()if check_error1_happen(db): db.rollback() return Response(“error1 happens”)if check_error2_happen(db): db.rollback() return Response(“error2 happens”)db.table.update()....db.commit()return Response(“success”) 4
    5. 5. why thinking in yield?• pythonic• bug 5
    6. 6. why thinking in yield? @contextmanager def transaction(conn):class Job(object): trans = Job() def __init__(self): conn.begin() self._finished = False try: yield trans def is_finished(self): except: return self._finished conn.rollback() raise def finish(self): if trans.is_finished(): self._finished = True conn.commit() else: conn.rollback() 6
    7. 7. why thinking in yield?with transaction(db) as trans: if check_error1_happen(db): return Response(“error1 happens”) if check_error2_happen(db): return Response(“error2 happens”) db.table.update() .... trans.finish()return Response(“success”) 7
    8. 8. why thinking in yield?main main A yield function B C yield function 8
    9. 9. yield 9
    10. 10. yieldUsing a yield expression in a function definition is sufficient tocause that definition to create a generator function instead ofa normal function. 10
    11. 11. yieldUsing a yield expression in a function definition is sufficient tocause that definition to create a generator function instead ofa normal function. >>> def f(): ... yield "hello world" ... >>> type(f()) <type generator> 10
    12. 12. yieldThat generator then controls the execution of a generatorfunction. 11
    13. 13. yieldThat generator then controls the execution of a generatorfunction. • send() • next() ==> send(None) • throw() • close() ==> throw(GeneratorExit) 11
    14. 14. yieldyield is a expression, not a statement. New in python 2.5 12
    15. 15. yieldyield is a expression, not a statement. New in python 2.5>>> def f():... a = yield "hello"... print a...>>> b = f()>>> b.send(None)hello>>> b.send("world")worldStopIteration 12
    16. 16. yieldgenerator is also a iterator. 13
    17. 17. yieldgenerator is also a iterator.>>> a = f()>>> a is a.__iter__()True 13
    18. 18. yieldgenerator object 14
    19. 19. yieldgenerator object>>> def f():... yield "hello world"...>>> a = f()>>> a.next()“hello world”>>> a.next()StopIteration>>> a.next()StopIteration 14
    20. 20. yield 15
    21. 21. yieldtstate_head next next PyThreadState PyThreadState ... frame PyFrameObject f_back PyFrameObject f_back python ... 16
    22. 22. yield code block compile PyCodeObject MAKE_FUNCTIONPyFunctionObject CALL_FUNCTION PyFrameObject 17
    23. 23. yield[frameobject.c]typedef struct { ... struct _frame *f_back; PyCodeObject *f_code; PyObject *f_builtins; PyObject *f_globals; PyObject *f_locals; PyObject *f_valuestack; PyObject *f_stacktop; ... int f_lasti; ...} PyFrameObject; 18
    24. 24. yield[codeobject.c]typedef struct { ... int co_flags; ... PyObject *co_code; PyObject *co_consts; PyObject *co_names; ...} PyCodeObject; 19
    25. 25. yield[a.py] [b.py]def f(): def f(): return “hello world” yield “hello world”f() f() PyCodeObject <==> code block 20
    26. 26. yield4 0 LOAD_CONST 0 (<code object f ..>) 3 MAKE_FUNCTION 0 6 STORE_NAME 0 (f)7 9 LOAD_NAME 0 (f) 12 CALL_FUNCTION 0 15 POP_TOP 16 LOAD_CONST 1 (None) 19 RETURN_VALUE dis.dis(main_code) 21
    27. 27. yield5 0 LOAD_CONST 1 (hello world) 3 RETURN_VALUE5 0 LOAD_CONST 1 (hello world) 3 YIELD_VALUE 4 POP_TOP 5 LOAD_CONST 0 (None) 8 RETURN_VALUE dis.dis(f_code) 22
    28. 28. yield return generator 23
    29. 29. yield[ceval.c] CALL_FUNCTIONf = PyFrame_New(tstate, co, globals, locals);...if (co->co_flags & CO_GENERATOR) { ... f->f_back = NULL; ... return PyGen_New(f);}retval = PyEval_EvalFrameEx(f,0); 24
    30. 30. yield[frameobject.c]PyFrame_New(PyThreadState *tstate,PyCodeObject *code, PyObject *globals, PyObject *locals){ ... PyFrameObject *back = tstate->frame; PyFrameObject *f; f = PyObject_GC_Resize(PyFrameObject, f, extras); ... f->f_code = code; ... f->f_back = back; f->f_tstate = tstate; return f;} 25
    31. 31. yield[ceval.c]PyObject *PyEval_EvalFrameEx(PyFrameObject *f, int throwflag){ ... PyThreadState *tstate = PyThreadState_GET(); tstate->frame = f; ... // exec opcode ... tstate->frame = f->f_back; return retval;} 26
    32. 32. yieldtstate_head next PyThreadState ... frame A frame f_back B frame f_back ... 27
    33. 33. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... 27
    34. 34. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... 27
    35. 35. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
    36. 36. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
    37. 37. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
    38. 38. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
    39. 39. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
    40. 40. yieldtstate_head next PyThreadState ... frame A frame f_back B frame f_back ... yield 28
    41. 41. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... yield 28
    42. 42. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... yield 28
    43. 43. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... yield 28
    44. 44. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... yield 28
    45. 45. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... yield 28
    46. 46. yield[genobject.c]static PyObject * gen_send_ex(PyGenObject *gen, PyObject *arg, intexc) { PyFrameObject *f = gen->gi_frame; f->f_back = tstate->frame; ... if (f->f_lasti == -1) { .... } else { result = arg ? arg : Py_None; Py_INCREF(result); *(f->f_stacktop++) = result; } ... result = PyEval_EvalFrameEx(f, exc);} 29
    47. 47. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back B frame(creator) f_back ... send 30
    48. 48. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back B frame(creator) f_back ... send 30
    49. 49. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back f_back B frame(creator) f_back ... send 30
    50. 50. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back f_back B frame(creator) f_back ... send 30
    51. 51. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back f_back B frame(creator) f_back ... send 30
    52. 52. yieldresume frame 31
    53. 53. yield[ceval.c]PyObject *PyEval_EvalFrameEx(PyFrameObject *f, int throwflag){ ... next_instr = first_instr + f->f_lasti + 1; stack_pointer = f->f_stacktop; assert(stack_pointer != NULL); f->f_stacktop = NULL; ... case YIELD_VALUE: retval = POP(); f->f_stacktop = stack_pointer; why = WHY_YIELD; goto fast_yield;} 32
    54. 54. yield[genobject.c]static PyObject *gen_throw(PyGenObject *gen,PyObject *args){ ... PyErr_Restore(typ, val, tb); return gen_send_ex(gen, Py_None, 1);} throw 33
    55. 55. yielddef close(self): try: self.throw(GeneratorExit) except (GeneratorExit, StopIteration): pass else: raise RuntimeError("generator ignored GeneratorExit") # Other exceptions are not caught close 34
    56. 56. yield 35
    57. 57. • xrange• with_statement“ ”• python Trampoline• generator coroutine 36
    58. 58. xrange 37
    59. 59. xrange[rangeobject.c]static PyObject *rangeiter_next(rangeiterobject *r){ if (r->index < r->len) return PyInt_FromLong(r->start + (r->index++) * r->step); return NULL;} xrange 38
    60. 60. xrangedef xrange(start, stop, step=1): i = start - step while True: i = i + step if i < stop: yield i else: return yield 39
    61. 61. xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10)) list generator 40
    62. 62. xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10)) list generator 40
    63. 63. xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10)) list generatordef f(): for x in xrange(10): yield x * x 40
    64. 64. with_statement“ ” 41
    65. 65. “with_statement” with with 42
    66. 66. “with_statement” with withwith EXPR as VAR: BLOCK 42
    67. 67. “with_statement” with withwith EXPR as VAR: BLOCK 42
    68. 68. “with_statement”mgr = (EXPR)exit = type(mgr).__exit__ # Not calling it yetvalue = type(mgr).__enter__(mgr)exc = Truetry: try: VAR = value # Only if "as VAR" is present BLOCK except: exc = False if not exit(mgr, *sys.exc_info()): raisefinally: if exc: exit(mgr, None, None, None) 43
    69. 69. “with_statement”class Transaction(object): def __init__(self, db): self._db = db self._trans = Job() def __enter__(self): self._db.begin() return self._trans def __exit__(self, type, value, traceback): if type is not None: self._db.rollback() else: self._db.commit() yield transaction 44
    70. 70. “with_statement”@contextmanagerdef transaction(conn): trans = Job() conn.begin() try: yield trans except: conn.rollback() raise if trans.is_finished(): conn.commit() else: conn.rollback() 45
    71. 71. “with_statement”@contextmanagerdef transaction(conn): trans = Job() conn.begin() try: yield trans except: conn.rollback() raise if trans.is_finished(): conn.commit() else: conn.rollback() 45
    72. 72. python Trampoline 46
    73. 73. python Trampolinedef f(n): if n < 2: return n else: return n + f(n - 1)>>> f(999)499500>>> f(1000)RuntimeError: maximum recursion depth exceeded 47
    74. 74. python Trampolinesys.getrecursionlimit()Return the current value of the recursion limit, themaximum depth of the Python interpreter stack. Thislimit prevents infinite recursion from causing anoverflow of the C stack and crashing Python. It can beset by setrecursionlimit(). 48
    75. 75. python Trampoline 49
    76. 76. python Trampoline f(n) f(n) control f(n-1) f(n-1) f(n-2) scheduler f(n-2)f_back ... ... f(1) f(1) 50
    77. 77. python Trampoline• let generator "call" other generator by yielding the generator they wish to invoke.• Any non-generator value yielded by a generator is returned to the generator that "called" the one yielding the value. 51
    78. 78. python Trampolinedef f(n): if n < 2: yield n else: yield n + (yield f(n-1)) 52
    79. 79. python Trampolinedef trampoline(main): stack = [] value = main while True: if type(value) is types.GeneratorType: stack.append(value) value = stack[-1].next() else: stack.pop() if stack: value = stack[-1].send(value) else: return value 53
    80. 80. generator coroutine 54
    81. 81. generator coroutineCoroutine( )" " " " 55
    82. 82. generator coroutineCoroutine( )" " " " Pythons generator functions are almost coroutines. 55
    83. 83. generator coroutinegenerator coroutine 56
    84. 84. generator coroutinegenerator coroutine• coroutine “call” coroutine• coroutine scheduler 56
    85. 85. generator coroutinecoroutine-based Python networking library• Eventlet(http://eventlet.net/)• Gevent(http://www.gevent.org/) 57
    86. 86. Resources & References• python references: yield expression• python• PEP 342: Coroutines via Enhanced Generators• PEP 343: the with statement 58
    87. 87. Q&A 59
    88. 88. Thanks! 60
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×