1.
Concurrency
& Gevent
Scaling Real World Applications

2.
Concurrency
handling a number of things at the same time
Examples: (Incoming) WebServers, Database Servers
(Outgoing) SSH Mux

3.
SSH Mux
●
Execute a command on a remote
SSH server
●
Handle concurrent SSH Clients
●
Command execution time varies
from seconds to days
●
Command execution happens on
remote servers, SSH mux is I/O
bound

4.
SSH Client
1. Init session
2. Authenticate
3. Get a channel
4. Issue command
5. Read output

5.
Need Concurrency?
●
Process blocks on read()
●
No new connections can be inititated
●
Need ability to handle multiple clients at the same
time

6.
Multiprocessing
●
One process is the master
●
Master can spawn workers
●
Each worker handles one request at a time
●
Pre-forked pool of workers

7.
Concurrent SSH Clients

8.
SSH Mux Memory Usage
ssh mux memory usage
600
500
400
Memory (MB)
300 Processes
200
100
0
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
# Concurrent Reqs

9.
SSH Mux Performace
ssh mux performance sheet
160
140
120
100
Time(s)
80 Processes
60
40
20
0
20 40 80 150 200 400 800 1200 1500 1800 2100
# Concurrent Reqs

10.
Multiprocessing yay
●
Easy to get started
●
OS guaranteed process isolation & fairness
●
Covers up for misbehaving workers
●
Add more concurrency by adding more workers
●
Convenient when numbers are smaller numbers

11.
Multiprocessing nay
●
Concurrency limited by number of processes
●
Memory heavy
●
Implicit scheduling
●
Synchronization is not trivial

12.
More Concurrency?
●
Command execution is happening on remote servers, we
are mostly blocked on I/O
●
Handle multiple I/O in a single process?

13.
Gevent
gevent is a coroutine-based Python networking library that
uses greenlet to provide a high-level synchronous API on
top of the libevent event loop.

14.
Greenlets
●
Lightweight 'threads' - not OS threads
●
Explicit scheduling - Cooperative
●
Minimal stack
●
Application decides execution flow
●
Easy to synchronize/ Avoid locks
●
All run inside one process

15.
Libevent
●
Use fastest mechanism to poll (portable)
●
Fast Event loop
●
In Gevent, event loop runs in a greenlet (event hub)
●
Instead of blocking, greenlets switch to event hub
●
It's all done behind the scene

16.
Monkey Patching
Monkey patching
Modifies behaviour of blocking calls such as select, sleep to
non-blocking
Patches the python standard socket library

17.
Gevent
●
Greenlet 1 is running
●
Greenlet 2 and 3 are ready

18.
Gevent
●
Greenlet 1 has to wait for read
●
Greenlet 1 switches to Event hub

19.
Gevent
●
Event hub switches to Greenlet 3

20.
Gevent
●
Greenlet 2 runs

21.
Gevent
●
Greenlet 2 wants to sleep
●
Greenlet 2 switches to Event hub

22.
Gevent
●
Greenlet 1 data has come, moved to ready state
●
Eventhub switches to Greenlet 3

23.
Gevent
●
Greenlet 3 runs

24.
Gevent
●
When Greenlet 1 resumes, its from next instruction
●
It's as if it were a blocking call

25.
Green SSH Client
1. Init session
2. Authenticate
3. Get a channel
4. Issue command
5. Read output

26.
A closer look

27.
Going Concurrent
Use pre-forked processes to use all cores

28.
Memory usage
ssh mux memory usage
45
40
35
30
Memory(MB)
25
Gevent+Processes
20
15
10
5
0
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
# Concurrent Reqs

29.
SSH Mux Performace
ssh mux performace chart
70
60
50
40
Time(s)
Gevent+Processes
30
20
10
0
20 40 80 150 200 400 800 1200 1500 1800 2100
# Concurrent Reqs

30.
SSH Mux memory usage
ssh mux memory usage
600
500
400
Memory(MB)
Processes
300
Gevent
200
100
0
0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100
# Concurrent Reqs

31.
SSH Mux Performance
ssh mux performance sheet
160
140
120
100
Processes
Time(s)
80 Gevent+Processes
60
40
20
0
20 40 80 150 200 400 800 1200 1500 1800 2100
# Concurrent Reqs

32.
Gevent yay!
●
Untwist – write linear non blocking code
●
Explicit scheduling, dictate the execution flow
●
Timeouts
●
Events, AsyncResults for Synchronization
●
gevent.wsgi
●
Pre-spawned pool of greenlets

33.
Gevent beware of
●
No multicore support
●
Not great for CPU bound applications
●
Third party libs must be green (non blocking)
●
Misbehaving workers can be lethal
●
No fairness when it comes to scheduling

34.
Take Away
●
Gevent lets you write asynchronous code in a
synchronous manner
●
No multicore support, still need multiprocessing
●
Not so great for CPU bound applications
●
Split your application into CPU bound and IO bound
parts
●
Be willing to contribute patches
●
Code available at
git@github.com:aaloksood/pyexamples.git

35.
Thank you
That's all folks!

36.
Countdown Timer
●
Count down from 200000000
●
Split work among workers

37.
Threads
Multithreading wonder
25
20
15
1 Core
Time(s)
4 cores
10
5
0
1 2 3 4 5 6 7 8 9 10
# Workers

38.
One core
Execution time One Core
14.5
14
13.5
13 Processes
1 Core
Time (s)
12.5
Gevent_1
12 Gevent_4
11.5
11
10.5
1 2 3 4 5 6 7 8 9 10
# Workers

39.
Four cores
Execution time 4 cores
25
20
Process
15
Threads
Time(s)
Gevent_1
10
Gevent_4
5
0
1 2 3 4 5 6 7 8 9 10
# Workers