1. Gevent
What’s the point?
Sean McQuillan
sean@bittorrent.com

2. Bottom Line
Gevent is fast (libev)
Pythonic style
Single threaded event loop

3. Gevent is Fast
application code
gevent
libev greenlet
kernel

4. Pythonic Style
def get_db():
return db.query(...)
vs
def _db_cb(result):
consumer.send(result)
def get_db():
x = db.query(...)
x.addCallback(_db_cb)

5. Single Threaded
Event Loop
Single threaded

6. Single Threaded
Event Loop
Single threaded
wait

7. Single Threaded
Event Loop
Single threaded
wait
what?

8. Execution Model
One OS thread
Only ONE thing happens at a time...
Switching constantly

9. More Execution Model
def example_function(sock):
r = sock.recv(4096)
count = len(r)
for c in xrange(count):
tokenizer.send(c)
r += sock.recv(4096)

10. More Execution Model
def example_function(sock):
r = sock.recv(4096)
count = len(r)
for c in xrange(count):
tokenizer.send(c)
r += sock.recv(4096)

11. More Execution Model
def get_db():
return db.query(...)

12. More Execution Model
def get_db():
return db.query(...)

13. Basic Example

14. from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello_world():
import time; time.sleep(1)
return 'Hello World!'
if __name__ == '__main__':
app.run()

15. from flask import Flask
app = Flask(__name__)
import gevent.monkey
gevent.monkey.patch_all()
@app.route('/')
def hello_world():
import time; time.sleep(1)
return 'Hello World!'
if __name__ == '__main__':
app.run()

16. from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello_world():
import time; time.sleep(1)
return 'Hello World!'
if __name__ == '__main__':
app.run()

17. from flask import Flask
app = Flask(__name__)

18. from flask import Flask
app = Flask(__name__)
import gevent.monkey
gevent.monkey.patch_all()
@app.route('/')
def hello_world():
import time; time.sleep(1)
return 'Hello World!'
if __name__ == '__main__':
app.run()

19. from flask import Flask
app = Flask(__name__)
import gevent.monkey
gevent.monkey.patch_all()

20. Interlude:
Monkey Patching

21. Monkey Patching
Replace existing libraries
Not generally done in python

22. Monkey Patching
sys.modules[socket] = gevent.socket
sys.modules[time] = gevent.sleep
...

23. Part II: Event Loops
Low overhead
Single threaded
Handle LOTS of connections

25. poll, select, epoll, kqueue
“Given a list of file descriptors which are
ready for read or write.”

26. Lets dig in
application code
gevent
libev greenlet
kernel

27. Lets dig in
socket 1 write callback event socket 1 read callback
socket 2 write callback loop socket 2 read callback
socket 1, socket 2, ...
kernel
write buffer 1 read buffer 1
write buffer 2 read buffer 2

28. Example
Contrived event loop example
See t wisted event loop for real example
t wisted/interent/selectreactor.py

29. Example
rlist = []
wlist = []
callbacks = {}
pending = defaultdict(str)
def want_to_read(sock, cb):
callbacks[sock.fileno] = cb
global rlist
rlist = list(set(rlist + [sock.fileno]))
def want_to_write(sock, data):
pending[sock.fileno] += data
global wlist
wlist = list(set(wlist + [sock.fileno])))

30. Example
def event_loop():
while True:
reads, writes, _ = select(rlist, wlist, [])
for readfd in reads:
data = _do_actual_read(readfd)
callbacks[readfd](data)
for writefd in writes:
_do_actual_write(pending[writefd])

32. Part III: Sockets
Sockets
What magic lies here

33. patched socket.recv
def recv(self, *args):
sock = self._sock
while True:
try:
return sock.recv(*args)
except error:
ex = sys.exc_info()[1]
if ex.args[0] == EBADF:
return ''
if (ex.args[0] != EWOULDBLOCK or
self.timeout == 0.0 ):
raise
self._wait(self._read_event)

34. patched socket.recv
def recv(self, *args):
sock = self._sock
while True:
try:
return sock.recv(*args)
except error:
ex = sys.exc_info()[1]
if ex.args[0] == EBADF:
return ''
if (ex.args[0] != EWOULDBLOCK or
self.timeout == 0.0 ):
raise
self._wait(self._read_event)

35. patched socket.recv
def recv(self, *args):
sock = self._sock
while True:
try:
return sock.recv(*args)
except error:
ex = sys.exc_info()[1]
if ex.args[0] == EBADF:
return ''
if (ex.args[0] != EWOULDBLOCK or
self.timeout == 0.0 ):
raise
self.want_to_read(sock,
self.mygreenthread.switch)

37. Part IV: Greenlets
“green” threads

38. Part IV: Greenlets
“green” threads
(that means fake threads)

39. Part IV: Greenlets
“green” threads
(that means fake threads)

40. Green Threads Good
Never switch accidentally
Never spend time blocked on IO
High performance switching

41. Green Threads Bad
Never preemptively interrupt
No SMP

42. Green Thread Execution
Python Initial Stack

43. Green Thread Execution
Gevent Initialized
Python Initial Stack

44. Green Thread Execution
Green
Thread
Stack
Gevent Initialized
Python Initial Stack

45. Green Thread Execution
Green
Thread
Green Stack
Thread
Stack
Gevent Initialized
Python Initial Stack

46. Green Thread Execution
Green
Thread
Green Stack
Thread
Stack
Gevent Initialized
Python Initial Stack

47. Green Thread Execution
Green Green
Thread Thread
Green Stack Stack
Thread
Stack
Gevent Initialized
Python Initial Stack

48. Green Thread Execution
Green Green
Thread Thread
Green Stack Stack
Thread
Stack
Gevent Initialized
Python Initial Stack

49. Green Thread Execution
Green Green
Thread Thread
Green Stack Stack
Thread
Stack
Gevent Initialized
Python Initial Stack

50. How does it switch?
(x86 assembly)
void *ebp, *ebx;
unsigned short cw;
register int *stackref, stsizediff;
__asm__ volatile ("" : : : "esi", "edi");
__asm__ volatile ("fstcw %0" : "=m" (cw));
__asm__ volatile ("movl %%ebp, %0" : "=m" (ebp));
__asm__ volatile ("movl %%ebx, %0" : "=m" (ebx));
__asm__ ("movl %%esp, %0" : "=g" (stackref));
{
SLP_SAVE_STATE(stackref, stsizediff);
__asm__ volatile (
"addl %0, %%espn"
"addl %0, %%ebpn"
:
: "r" (stsizediff)
);
SLP_RESTORE_STATE();
}
__asm__ volatile ("movl %0, %%ebx" : : "m" (ebx));
__asm__ volatile ("movl %0, %%ebp" : : "m" (ebp));
__asm__ volatile ("fldcw %0" : : "m" (cw));
__asm__ volatile ("" : : : "esi", "edi");
return 0;

52. When to use Gevent
Replace thread-based ser vers
Replace thread-based clients
Lighter than multiprocessing

53. When not to use gevent
When SMP is required
High CPU load
Latency guarntees
Memory star ved systems

54. Twisted
Mature, async-baked, implementations
of most protocols
DNS Ser ver
SSH Server
AWS
AMQP
Web

55. Tornado
HTTP
Highly performant
Less protocol / library support
I’m not an expert

56. Erlang
Scales more vertically (dozens of cores)
Similar execution model
Less library support

57. Python Threading
I’ve never tried to write a “web-scale”
server with it, would love to hear your
experience

58. Python
Multiproccessing
Great SMP usage

59. Go
Never used, benchmarks look good

60. The End