Understanding greenlet
Saúl Ibarra Corretgé - @saghul
Amsterdam Python Meetup Group
Quick check
Gevent or Eventlet, anyone?
Who knows what greenlet is?
Who has used it (standalone)?
Who understands how it w...
Greenlet
Micro-threads with no implicit scheduling
Lightweight
Only one can run at a time
Cooperative
Greenlet API
greenlet(func, parent=None): creates a greenlet to
run ‘func’
greenlet.switch(*args, **kw): switches executio...
Example
import greenlet

main = greenlet.getcurrent()
def foo(n):
main.switch(n)
def bar(n):
foo(n)
return 'hello'

g1 = g...
How does it work?
Organized in a tree structure
Each greenlet has a ‘parent’, except main
When a greenlet dies, control is...
How does it work? (II)
greenlet2
greenlet1
greenlet3
greenlet5
greenlet4
How does it work? (III)
‘Stack switching’
Non portable asm code
Copy stack slices on the heap

State is saved and restored...
Enter PyPy
New shiny and fast implementation of Python
Vast amount of fairy-dust covered unicorns
included
Includes an imp...
import _continuation
Continulets are one-shot continuations
Switching code is a standalone C library: stacklet
rpython/tra...
Continulet API
continulet(func, *args, **kw): create a continulet
object which will call fun(cont, *args, **kw)
continulet...
Stacklet
Tiny library implementing one-shot continuations
for C
Single C file (~400 lines) + per-platform asm

Supports x86...
Stacklet API
stacklet_newthread(): creates a new thread handle
stacklet_new(thread_handle, run_func, run_arg):
calls run(a...
#include <assert.h>
#include "stacklet.h"

Example

static stacklet_thread_handle thrd;

stacklet_handle empty_callback(st...
The Stacklet Sandwich (TM)
greenlet

continulet

stacklet
Using greenlet
Too low level, it’s usually used through a
framework
gevent, eventlet, evergreen *, gruvi *, ...
In these f...
Using greenlet (II)
main

g1

hub

g2

g3

...
Undoing callbacks
import greenlet

# ...
def foo_async(cb):
call cb(result, error) eventually
pass
def foo_sync():
current...
import fibers

shameless
plug!

Micro-threadling library API inspired by Python
threads and greenlet
Uses stacklet undernea...
Questions?

@saghul
bettercallsaghul.com
Understanding greenlet
Understanding greenlet
Understanding greenlet
Understanding greenlet
Upcoming SlideShare
Loading in...5
×

Understanding greenlet

2,591
-1

Published on

Slides from my talk about how greenlet works internally. Talk given at the Amsterdam Python Meetup Group.

Published in: Technology

Understanding greenlet

  1. 1. Understanding greenlet Saúl Ibarra Corretgé - @saghul Amsterdam Python Meetup Group
  2. 2. Quick check Gevent or Eventlet, anyone? Who knows what greenlet is? Who has used it (standalone)? Who understands how it works?
  3. 3. Greenlet Micro-threads with no implicit scheduling Lightweight Only one can run at a time Cooperative
  4. 4. Greenlet API greenlet(func, parent=None): creates a greenlet to run ‘func’ greenlet.switch(*args, **kw): switches execution to the target greenlet, the first time func(*args, **kw) will be executed greenlet.throw([typ, [val, [tb]]]): switches execution to the target greenlet and raises the specified exception (GreenletExit by default)
  5. 5. Example import greenlet main = greenlet.getcurrent() def foo(n): main.switch(n) def bar(n): foo(n) return 'hello' g1 = greenlet.greenlet(bar) print g1.switch(42) # 42 print g1.switch() # 'hello' print g1.dead # True
  6. 6. How does it work? Organized in a tree structure Each greenlet has a ‘parent’, except main When a greenlet dies, control is switched to its parent Execution order isn’t always obvious A glorified GOTO?!
  7. 7. How does it work? (II) greenlet2 greenlet1 greenlet3 greenlet5 greenlet4
  8. 8. How does it work? (III) ‘Stack switching’ Non portable asm code Copy stack slices on the heap State is saved and restored when switching CPU registers Current Python frame, recursion depth and exception state
  9. 9. Enter PyPy New shiny and fast implementation of Python Vast amount of fairy-dust covered unicorns included Includes an implementation of greenlet Implemented on top of “continulet” objects
  10. 10. import _continuation Continulets are one-shot continuations Switching code is a standalone C library: stacklet rpython/translator/c/src/stacklet/
  11. 11. Continulet API continulet(func, *args, **kw): create a continulet object which will call fun(cont, *args, **kw) continulet.switch(value=None, to=None): start the continulet or activate the previously suspended one. If to is specified a ‘double switch’ is performed continulet.throw(type, value=None, tb=None, to=None): similar to switch, but raise the given exception after the switch is done
  12. 12. Stacklet Tiny library implementing one-shot continuations for C Single C file (~400 lines) + per-platform asm Supports x86, x86_64 and ARM Nice and simple API
  13. 13. Stacklet API stacklet_newthread(): creates a new thread handle stacklet_new(thread_handle, run_func, run_arg): calls run(arg) in a new stacklet, starts immediately stacklet_switch(target): switches execution to target stacklet
  14. 14. #include <assert.h> #include "stacklet.h" Example static stacklet_thread_handle thrd; stacklet_handle empty_callback(stacklet_handle h, void *arg) { assert(arg == (void *)123); return h; } void test_new(void) { stacklet_handle h = stacklet_new(thrd, empty_callback, (void *)123); assert(h == EMPTY_STACKLET_HANDLE); } int main(int argc, char **argv) { thrd = stacklet_newthread(); test_new(); stacklet_deletethread(thrd); return 0; }
  15. 15. The Stacklet Sandwich (TM) greenlet continulet stacklet
  16. 16. Using greenlet Too low level, it’s usually used through a framework gevent, eventlet, evergreen *, gruvi *, ... In these frameworks, switching happens through a “hub” *: these frameworks use python-fibers now
  17. 17. Using greenlet (II) main g1 hub g2 g3 ...
  18. 18. Undoing callbacks import greenlet # ... def foo_async(cb): call cb(result, error) eventually pass def foo_sync(): current = greenlet.getcurrent() def cb(result, error): if error is not None: current.throw(Exception(error)) else: current.switch(result) foo_async(cb) return hub.switch()
  19. 19. import fibers shameless plug! Micro-threadling library API inspired by Python threads and greenlet Uses stacklet underneath Works on CPython and PyPy On PyPy it uses continulets github.com/saghul/python-fibers Or pip install fibers
  20. 20. Questions? @saghul bettercallsaghul.com
  1. Gostou de algum slide específico?

    Recortar slides é uma maneira fácil de colecionar informações para acessar mais tarde.

×