Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Concurrency & GeventScaling Real World Applications
Concurrencyhandling a number of things at the same timeExamples: (Incoming) WebServers, Database Servers         (Outgoing...
SSH Mux●    Execute a command on a remote    SSH server●    Handle concurrent SSH Clients●    Command execution time varie...
SSH Client1. Init session2. Authenticate3. Get a channel4. Issue command5. Read output
Need Concurrency?●    Process blocks on read()●    No new connections can be inititated●    Need ability to handle multipl...
Multiprocessing●    One process is the master●    Master can spawn workers●    Each worker handles one request at a time● ...
Concurrent SSH Clients
SSH Mux Memory Usage                                              ssh mux memory usage               600               500...
SSH Mux Performace                                     ssh mux performance sheet          160          140          120   ...
Multiprocessing yay●    Easy to get started●    OS guaranteed process isolation & fairness●    Covers up for misbehaving w...
Multiprocessing nay●    Concurrency limited by number of processes●    Memory heavy●    Implicit scheduling●    Synchroniz...
More Concurrency?●    Command execution is happening on remote servers, we    are mostly blocked on I/O●    Handle multipl...
Geventgevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on ...
Greenlets●    Lightweight threads - not OS threads●    Explicit scheduling - Cooperative●    Minimal stack●    Application...
Libevent●    Use fastest mechanism to poll (portable)●    Fast Event loop●    In Gevent, event loop runs in a greenlet (ev...
Monkey PatchingMonkey patchingModifies behaviour of blocking calls such as select, sleep tonon-blockingPatches the python ...
Gevent●   Greenlet 1 is running●   Greenlet 2 and 3 are ready
Gevent●    Greenlet 1 has to wait for read●    Greenlet 1 switches to Event hub
Gevent●    Event hub switches to Greenlet 3
Gevent●   Greenlet 2 runs
Gevent●    Greenlet 2 wants to sleep●    Greenlet 2 switches to Event hub
Gevent●    Greenlet 1 data has come, moved to ready state●    Eventhub switches to Greenlet 3
Gevent●   Greenlet 3 runs
Gevent●   When Greenlet 1 resumes, its from next instruction●   Its as if it were a blocking call
Green SSH Client1. Init session2. Authenticate3. Get a channel4. Issue command5. Read output
A closer look
Going Concurrent Use pre-forked processes to use all cores
Memory usage                                               ssh mux memory usage             45             40             ...
SSH Mux Performace                                           ssh mux performace chart           70           60           ...
SSH Mux memory usage                                              ssh mux memory usage                600                5...
SSH Mux Performance                                        ssh mux performance sheet          160          140          12...
Gevent yay!●    Untwist – write linear non blocking code●    Explicit scheduling, dictate the execution flow●    Timeouts●...
Gevent beware of●    No multicore support●    Not great for CPU bound applications●    Third party libs must be green (non...
Take Away●    Gevent lets you write asynchronous code in a    synchronous manner●    No multicore support, still need mult...
Thank you            Thats all folks!
Countdown Timer●    Count down from 200000000●    Split work among workers
Threads                                 Multithreading wonder            25            20            15                   ...
One core                                    Execution time One Core             14.5              14             13.5     ...
Four cores                                  Execution time 4 cores             25             20                          ...
Upcoming SlideShare
Loading in …5
×

Scaling real world applications using gevent

3,248 views

Published on

Talk on gevent at Pycon by Sood Aalok the magnificent

Published in: Technology
  • Be the first to comment

Scaling real world applications using gevent

  1. 1. Concurrency & GeventScaling Real World Applications
  2. 2. Concurrencyhandling a number of things at the same timeExamples: (Incoming) WebServers, Database Servers (Outgoing) SSH Mux
  3. 3. SSH Mux● Execute a command on a remote SSH server● Handle concurrent SSH Clients● Command execution time varies from seconds to days● Command execution happens on remote servers, SSH mux is I/O bound
  4. 4. SSH Client1. Init session2. Authenticate3. Get a channel4. Issue command5. Read output
  5. 5. Need Concurrency?● Process blocks on read()● No new connections can be inititated● Need ability to handle multiple clients at the same time
  6. 6. Multiprocessing● One process is the master● Master can spawn workers● Each worker handles one request at a time● Pre-forked pool of workers
  7. 7. Concurrent SSH Clients
  8. 8. SSH Mux Memory Usage ssh mux memory usage 600 500 400 Memory (MB) 300 Processes 200 100 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 # Concurrent Reqs
  9. 9. SSH Mux Performace ssh mux performance sheet 160 140 120 100Time(s) 80 Processes 60 40 20 0 20 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs
  10. 10. Multiprocessing yay● Easy to get started● OS guaranteed process isolation & fairness● Covers up for misbehaving workers● Add more concurrency by adding more workers● Convenient when numbers are smaller numbers
  11. 11. Multiprocessing nay● Concurrency limited by number of processes● Memory heavy● Implicit scheduling● Synchronization is not trivial
  12. 12. More Concurrency?● Command execution is happening on remote servers, we are mostly blocked on I/O● Handle multiple I/O in a single process?
  13. 13. Geventgevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libevent event loop.
  14. 14. Greenlets● Lightweight threads - not OS threads● Explicit scheduling - Cooperative● Minimal stack● Application decides execution flow● Easy to synchronize/ Avoid locks● All run inside one process
  15. 15. Libevent● Use fastest mechanism to poll (portable)● Fast Event loop● In Gevent, event loop runs in a greenlet (event hub)● Instead of blocking, greenlets switch to event hub● Its all done behind the scene
  16. 16. Monkey PatchingMonkey patchingModifies behaviour of blocking calls such as select, sleep tonon-blockingPatches the python standard socket library
  17. 17. Gevent● Greenlet 1 is running● Greenlet 2 and 3 are ready
  18. 18. Gevent● Greenlet 1 has to wait for read● Greenlet 1 switches to Event hub
  19. 19. Gevent● Event hub switches to Greenlet 3
  20. 20. Gevent● Greenlet 2 runs
  21. 21. Gevent● Greenlet 2 wants to sleep● Greenlet 2 switches to Event hub
  22. 22. Gevent● Greenlet 1 data has come, moved to ready state● Eventhub switches to Greenlet 3
  23. 23. Gevent● Greenlet 3 runs
  24. 24. Gevent● When Greenlet 1 resumes, its from next instruction● Its as if it were a blocking call
  25. 25. Green SSH Client1. Init session2. Authenticate3. Get a channel4. Issue command5. Read output
  26. 26. A closer look
  27. 27. Going Concurrent Use pre-forked processes to use all cores
  28. 28. Memory usage ssh mux memory usage 45 40 35 30Memory(MB) 25 Gevent+Processes 20 15 10 5 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 # Concurrent Reqs
  29. 29. SSH Mux Performace ssh mux performace chart 70 60 50 40 Time(s) Gevent+Processes 30 20 10 0 20 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs
  30. 30. SSH Mux memory usage ssh mux memory usage 600 500 400 Memory(MB) Processes 300 Gevent 200 100 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 # Concurrent Reqs
  31. 31. SSH Mux Performance ssh mux performance sheet 160 140 120 100 ProcessesTime(s) 80 Gevent+Processes 60 40 20 0 20 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs
  32. 32. Gevent yay!● Untwist – write linear non blocking code● Explicit scheduling, dictate the execution flow● Timeouts● Events, AsyncResults for Synchronization● gevent.wsgi● Pre-spawned pool of greenlets
  33. 33. Gevent beware of● No multicore support● Not great for CPU bound applications● Third party libs must be green (non blocking)● Misbehaving workers can be lethal● No fairness when it comes to scheduling
  34. 34. Take Away● Gevent lets you write asynchronous code in a synchronous manner● No multicore support, still need multiprocessing● Not so great for CPU bound applications● Split your application into CPU bound and IO bound parts● Be willing to contribute patches● Code available at git@github.com:aaloksood/pyexamples.git
  35. 35. Thank you Thats all folks!
  36. 36. Countdown Timer● Count down from 200000000● Split work among workers
  37. 37. Threads Multithreading wonder 25 20 15 1 Core Time(s) 4 cores 10 5 0 1 2 3 4 5 6 7 8 9 10 # Workers
  38. 38. One core Execution time One Core 14.5 14 13.5 13 Processes 1 Core Time (s) 12.5 Gevent_1 12 Gevent_4 11.5 11 10.5 1 2 3 4 5 6 7 8 9 10 # Workers
  39. 39. Four cores Execution time 4 cores 25 20 Process 15 Threads Time(s) Gevent_1 10 Gevent_4 5 0 1 2 3 4 5 6 7 8 9 10 # Workers

×