Your SlideShare is downloading. ×
0
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Understanding greenlet

1,823

Published on

Slides from my talk about how greenlet works internally. Talk given at the Amsterdam Python Meetup Group.

Slides from my talk about how greenlet works internally. Talk given at the Amsterdam Python Meetup Group.

Published in: Technology
0 Comments
4 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,823
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
42
Comments
0
Likes
4
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Understanding greenlet Saúl Ibarra Corretgé - @saghul Amsterdam Python Meetup Group
  • 2. Quick check Gevent or Eventlet, anyone? Who knows what greenlet is? Who has used it (standalone)? Who understands how it works?
  • 3. Greenlet Micro-threads with no implicit scheduling Lightweight Only one can run at a time Cooperative
  • 4. Greenlet API greenlet(func, parent=None): creates a greenlet to run ‘func’ greenlet.switch(*args, **kw): switches execution to the target greenlet, the first time func(*args, **kw) will be executed greenlet.throw([typ, [val, [tb]]]): switches execution to the target greenlet and raises the specified exception (GreenletExit by default)
  • 5. Example import greenlet main = greenlet.getcurrent() def foo(n): main.switch(n) def bar(n): foo(n) return 'hello' g1 = greenlet.greenlet(bar) print g1.switch(42) # 42 print g1.switch() # 'hello' print g1.dead # True
  • 6. How does it work? Organized in a tree structure Each greenlet has a ‘parent’, except main When a greenlet dies, control is switched to its parent Execution order isn’t always obvious A glorified GOTO?!
  • 7. How does it work? (II) greenlet2 greenlet1 greenlet3 greenlet5 greenlet4
  • 8. How does it work? (III) ‘Stack switching’ Non portable asm code Copy stack slices on the heap State is saved and restored when switching CPU registers Current Python frame, recursion depth and exception state
  • 9. Enter PyPy New shiny and fast implementation of Python Vast amount of fairy-dust covered unicorns included Includes an implementation of greenlet Implemented on top of “continulet” objects
  • 10. import _continuation Continulets are one-shot continuations Switching code is a standalone C library: stacklet rpython/translator/c/src/stacklet/
  • 11. Continulet API continulet(func, *args, **kw): create a continulet object which will call fun(cont, *args, **kw) continulet.switch(value=None, to=None): start the continulet or activate the previously suspended one. If to is specified a ‘double switch’ is performed continulet.throw(type, value=None, tb=None, to=None): similar to switch, but raise the given exception after the switch is done
  • 12. Stacklet Tiny library implementing one-shot continuations for C Single C file (~400 lines) + per-platform asm Supports x86, x86_64 and ARM Nice and simple API
  • 13. Stacklet API stacklet_newthread(): creates a new thread handle stacklet_new(thread_handle, run_func, run_arg): calls run(arg) in a new stacklet, starts immediately stacklet_switch(target): switches execution to target stacklet
  • 14. #include <assert.h> #include "stacklet.h" Example static stacklet_thread_handle thrd; stacklet_handle empty_callback(stacklet_handle h, void *arg) { assert(arg == (void *)123); return h; } void test_new(void) { stacklet_handle h = stacklet_new(thrd, empty_callback, (void *)123); assert(h == EMPTY_STACKLET_HANDLE); } int main(int argc, char **argv) { thrd = stacklet_newthread(); test_new(); stacklet_deletethread(thrd); return 0; }
  • 15. The Stacklet Sandwich (TM) greenlet continulet stacklet
  • 16. Using greenlet Too low level, it’s usually used through a framework gevent, eventlet, evergreen *, gruvi *, ... In these frameworks, switching happens through a “hub” *: these frameworks use python-fibers now
  • 17. Using greenlet (II) main g1 hub g2 g3 ...
  • 18. Undoing callbacks import greenlet # ... def foo_async(cb): call cb(result, error) eventually pass def foo_sync(): current = greenlet.getcurrent() def cb(result, error): if error is not None: current.throw(Exception(error)) else: current.switch(result) foo_async(cb) return hub.switch()
  • 19. import fibers shameless plug! Micro-threadling library API inspired by Python threads and greenlet Uses stacklet underneath Works on CPython and PyPy On PyPy it uses continulets github.com/saghul/python-fibers Or pip install fibers
  • 20. Questions? @saghul bettercallsaghul.com

×