Your SlideShare is downloading. ×
Python Yield
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Python Yield

3,737
views

Published on

Python yield

Python yield

Published in: Technology

0 Comments
12 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
3,737
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
135
Comments
0
Likes
12
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Transcript

    • 1. Python yield Mail & Popo: yangjunwei@ 1
    • 2. Agenda• why thinking in yield?• yield• yield• yield 2
    • 3. why thinking in yield? 3
    • 4. why thinking in yield?db.begin()if check_error1_happen(db): db.rollback() return Response(“error1 happens”)if check_error2_happen(db): db.rollback() return Response(“error2 happens”)db.table.update()....db.commit()return Response(“success”) 4
    • 5. why thinking in yield?• pythonic• bug 5
    • 6. why thinking in yield? @contextmanager def transaction(conn):class Job(object): trans = Job() def __init__(self): conn.begin() self._finished = False try: yield trans def is_finished(self): except: return self._finished conn.rollback() raise def finish(self): if trans.is_finished(): self._finished = True conn.commit() else: conn.rollback() 6
    • 7. why thinking in yield?with transaction(db) as trans: if check_error1_happen(db): return Response(“error1 happens”) if check_error2_happen(db): return Response(“error2 happens”) db.table.update() .... trans.finish()return Response(“success”) 7
    • 8. why thinking in yield?main main A yield function B C yield function 8
    • 9. yield 9
    • 10. yieldUsing a yield expression in a function definition is sufficient tocause that definition to create a generator function instead ofa normal function. 10
    • 11. yieldUsing a yield expression in a function definition is sufficient tocause that definition to create a generator function instead ofa normal function. >>> def f(): ... yield "hello world" ... >>> type(f()) <type generator> 10
    • 12. yieldThat generator then controls the execution of a generatorfunction. 11
    • 13. yieldThat generator then controls the execution of a generatorfunction. • send() • next() ==> send(None) • throw() • close() ==> throw(GeneratorExit) 11
    • 14. yieldyield is a expression, not a statement. New in python 2.5 12
    • 15. yieldyield is a expression, not a statement. New in python 2.5>>> def f():... a = yield "hello"... print a...>>> b = f()>>> b.send(None)hello>>> b.send("world")worldStopIteration 12
    • 16. yieldgenerator is also a iterator. 13
    • 17. yieldgenerator is also a iterator.>>> a = f()>>> a is a.__iter__()True 13
    • 18. yieldgenerator object 14
    • 19. yieldgenerator object>>> def f():... yield "hello world"...>>> a = f()>>> a.next()“hello world”>>> a.next()StopIteration>>> a.next()StopIteration 14
    • 20. yield 15
    • 21. yieldtstate_head next next PyThreadState PyThreadState ... frame PyFrameObject f_back PyFrameObject f_back python ... 16
    • 22. yield code block compile PyCodeObject MAKE_FUNCTIONPyFunctionObject CALL_FUNCTION PyFrameObject 17
    • 23. yield[frameobject.c]typedef struct { ... struct _frame *f_back; PyCodeObject *f_code; PyObject *f_builtins; PyObject *f_globals; PyObject *f_locals; PyObject *f_valuestack; PyObject *f_stacktop; ... int f_lasti; ...} PyFrameObject; 18
    • 24. yield[codeobject.c]typedef struct { ... int co_flags; ... PyObject *co_code; PyObject *co_consts; PyObject *co_names; ...} PyCodeObject; 19
    • 25. yield[a.py] [b.py]def f(): def f(): return “hello world” yield “hello world”f() f() PyCodeObject <==> code block 20
    • 26. yield4 0 LOAD_CONST 0 (<code object f ..>) 3 MAKE_FUNCTION 0 6 STORE_NAME 0 (f)7 9 LOAD_NAME 0 (f) 12 CALL_FUNCTION 0 15 POP_TOP 16 LOAD_CONST 1 (None) 19 RETURN_VALUE dis.dis(main_code) 21
    • 27. yield5 0 LOAD_CONST 1 (hello world) 3 RETURN_VALUE5 0 LOAD_CONST 1 (hello world) 3 YIELD_VALUE 4 POP_TOP 5 LOAD_CONST 0 (None) 8 RETURN_VALUE dis.dis(f_code) 22
    • 28. yield return generator 23
    • 29. yield[ceval.c] CALL_FUNCTIONf = PyFrame_New(tstate, co, globals, locals);...if (co->co_flags & CO_GENERATOR) { ... f->f_back = NULL; ... return PyGen_New(f);}retval = PyEval_EvalFrameEx(f,0); 24
    • 30. yield[frameobject.c]PyFrame_New(PyThreadState *tstate,PyCodeObject *code, PyObject *globals, PyObject *locals){ ... PyFrameObject *back = tstate->frame; PyFrameObject *f; f = PyObject_GC_Resize(PyFrameObject, f, extras); ... f->f_code = code; ... f->f_back = back; f->f_tstate = tstate; return f;} 25
    • 31. yield[ceval.c]PyObject *PyEval_EvalFrameEx(PyFrameObject *f, int throwflag){ ... PyThreadState *tstate = PyThreadState_GET(); tstate->frame = f; ... // exec opcode ... tstate->frame = f->f_back; return retval;} 26
    • 32. yieldtstate_head next PyThreadState ... frame A frame f_back B frame f_back ... 27
    • 33. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... 27
    • 34. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... 27
    • 35. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
    • 36. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
    • 37. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
    • 38. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
    • 39. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... 27
    • 40. yieldtstate_head next PyThreadState ... frame A frame f_back B frame f_back ... yield 28
    • 41. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... yield 28
    • 42. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... yield 28
    • 43. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... yield 28
    • 44. yieldtstate_head next PyThreadState ... frame A frame new frame f_back f_back B frame f_back ... yield 28
    • 45. yieldtstate_head next PyThreadState ... frame A frame new frame f_back B frame f_back ... yield 28
    • 46. yield[genobject.c]static PyObject * gen_send_ex(PyGenObject *gen, PyObject *arg, intexc) { PyFrameObject *f = gen->gi_frame; f->f_back = tstate->frame; ... if (f->f_lasti == -1) { .... } else { result = arg ? arg : Py_None; Py_INCREF(result); *(f->f_stacktop++) = result; } ... result = PyEval_EvalFrameEx(f, exc);} 29
    • 47. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back B frame(creator) f_back ... send 30
    • 48. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back B frame(creator) f_back ... send 30
    • 49. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back f_back B frame(creator) f_back ... send 30
    • 50. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back f_back B frame(creator) f_back ... send 30
    • 51. yieldtstate_head next PyThreadState ... frame A frame(caller) new frame f_back f_back B frame(creator) f_back ... send 30
    • 52. yieldresume frame 31
    • 53. yield[ceval.c]PyObject *PyEval_EvalFrameEx(PyFrameObject *f, int throwflag){ ... next_instr = first_instr + f->f_lasti + 1; stack_pointer = f->f_stacktop; assert(stack_pointer != NULL); f->f_stacktop = NULL; ... case YIELD_VALUE: retval = POP(); f->f_stacktop = stack_pointer; why = WHY_YIELD; goto fast_yield;} 32
    • 54. yield[genobject.c]static PyObject *gen_throw(PyGenObject *gen,PyObject *args){ ... PyErr_Restore(typ, val, tb); return gen_send_ex(gen, Py_None, 1);} throw 33
    • 55. yielddef close(self): try: self.throw(GeneratorExit) except (GeneratorExit, StopIteration): pass else: raise RuntimeError("generator ignored GeneratorExit") # Other exceptions are not caught close 34
    • 56. yield 35
    • 57. • xrange• with_statement“ ”• python Trampoline• generator coroutine 36
    • 58. xrange 37
    • 59. xrange[rangeobject.c]static PyObject *rangeiter_next(rangeiterobject *r){ if (r->index < r->len) return PyInt_FromLong(r->start + (r->index++) * r->step); return NULL;} xrange 38
    • 60. xrangedef xrange(start, stop, step=1): i = start - step while True: i = i + step if i < stop: yield i else: return yield 39
    • 61. xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10)) list generator 40
    • 62. xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10)) list generator 40
    • 63. xrangel = [x for x in xrange(10)]g = (x * x for x in xrange(10)) list generatordef f(): for x in xrange(10): yield x * x 40
    • 64. with_statement“ ” 41
    • 65. “with_statement” with with 42
    • 66. “with_statement” with withwith EXPR as VAR: BLOCK 42
    • 67. “with_statement” with withwith EXPR as VAR: BLOCK 42
    • 68. “with_statement”mgr = (EXPR)exit = type(mgr).__exit__ # Not calling it yetvalue = type(mgr).__enter__(mgr)exc = Truetry: try: VAR = value # Only if "as VAR" is present BLOCK except: exc = False if not exit(mgr, *sys.exc_info()): raisefinally: if exc: exit(mgr, None, None, None) 43
    • 69. “with_statement”class Transaction(object): def __init__(self, db): self._db = db self._trans = Job() def __enter__(self): self._db.begin() return self._trans def __exit__(self, type, value, traceback): if type is not None: self._db.rollback() else: self._db.commit() yield transaction 44
    • 70. “with_statement”@contextmanagerdef transaction(conn): trans = Job() conn.begin() try: yield trans except: conn.rollback() raise if trans.is_finished(): conn.commit() else: conn.rollback() 45
    • 71. “with_statement”@contextmanagerdef transaction(conn): trans = Job() conn.begin() try: yield trans except: conn.rollback() raise if trans.is_finished(): conn.commit() else: conn.rollback() 45
    • 72. python Trampoline 46
    • 73. python Trampolinedef f(n): if n < 2: return n else: return n + f(n - 1)>>> f(999)499500>>> f(1000)RuntimeError: maximum recursion depth exceeded 47
    • 74. python Trampolinesys.getrecursionlimit()Return the current value of the recursion limit, themaximum depth of the Python interpreter stack. Thislimit prevents infinite recursion from causing anoverflow of the C stack and crashing Python. It can beset by setrecursionlimit(). 48
    • 75. python Trampoline 49
    • 76. python Trampoline f(n) f(n) control f(n-1) f(n-1) f(n-2) scheduler f(n-2)f_back ... ... f(1) f(1) 50
    • 77. python Trampoline• let generator "call" other generator by yielding the generator they wish to invoke.• Any non-generator value yielded by a generator is returned to the generator that "called" the one yielding the value. 51
    • 78. python Trampolinedef f(n): if n < 2: yield n else: yield n + (yield f(n-1)) 52
    • 79. python Trampolinedef trampoline(main): stack = [] value = main while True: if type(value) is types.GeneratorType: stack.append(value) value = stack[-1].next() else: stack.pop() if stack: value = stack[-1].send(value) else: return value 53
    • 80. generator coroutine 54
    • 81. generator coroutineCoroutine( )" " " " 55
    • 82. generator coroutineCoroutine( )" " " " Pythons generator functions are almost coroutines. 55
    • 83. generator coroutinegenerator coroutine 56
    • 84. generator coroutinegenerator coroutine• coroutine “call” coroutine• coroutine scheduler 56
    • 85. generator coroutinecoroutine-based Python networking library• Eventlet(http://eventlet.net/)• Gevent(http://www.gevent.org/) 57
    • 86. Resources & References• python references: yield expression• python• PEP 342: Coroutines via Enhanced Generators• PEP 343: the with statement 58
    • 87. Q&A 59
    • 88. Thanks! 60