Scaling real world applications using gevent

3,567 views

Published on

Talk at Pycon India 2012 by Aalok the Magnificent

Published in: Technology
1 Comment
13 Likes
Statistics
Notes
  • You can share the source code?
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
3,567
On SlideShare
0
From Embeds
0
Number of Embeds
318
Actions
Shares
0
Downloads
83
Comments
1
Likes
13
Embeds 0
No embeds

No notes for slide

Scaling real world applications using gevent

  1. 1. Concurrency & GeventScaling Real World Applications
  2. 2. Concurrencyhandling a number of things at the same timeExamples: (Incoming) WebServers, Database Servers (Outgoing) SSH Mux
  3. 3. SSH Mux● Execute a command on a remote SSH server● Handle concurrent SSH Clients● Command execution time varies from seconds to days● Command execution happens on remote servers, SSH mux is I/O bound
  4. 4. SSH Client1. Init session2. Authenticate3. Get a channel4. Issue command5. Read output
  5. 5. Need Concurrency?● Process blocks on read()● No new connections can be inititated● Need ability to handle multiple clients at the same time
  6. 6. Multiprocessing● One process is the master● Master can spawn workers● Each worker handles one request at a time● Pre-forked pool of workers
  7. 7. Concurrent SSH Clients
  8. 8. SSH Mux Memory Usage ssh mux memory usage 600 500 400 Memory (MB) 300 Processes 200 100 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 # Concurrent Reqs
  9. 9. SSH Mux Performace ssh mux performance sheet 160 140 120 100Time(s) 80 Processes 60 40 20 0 20 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs
  10. 10. Multiprocessing yay● Easy to get started● OS guaranteed process isolation & fairness● Covers up for misbehaving workers● Add more concurrency by adding more workers● Convenient when numbers are smaller numbers
  11. 11. Multiprocessing nay● Concurrency limited by number of processes● Memory heavy● Implicit scheduling● Synchronization is not trivial
  12. 12. More Concurrency?● Command execution is happening on remote servers, we are mostly blocked on I/O● Handle multiple I/O in a single process?
  13. 13. Geventgevent is a coroutine-based Python networking library that uses greenlet to provide a high-level synchronous API on top of the libevent event loop.
  14. 14. Greenlets● Lightweight threads - not OS threads● Explicit scheduling - Cooperative● Minimal stack● Application decides execution flow● Easy to synchronize/ Avoid locks● All run inside one process
  15. 15. Libevent● Use fastest mechanism to poll (portable)● Fast Event loop● In Gevent, event loop runs in a greenlet (event hub)● Instead of blocking, greenlets switch to event hub● Its all done behind the scene
  16. 16. Monkey PatchingMonkey patchingModifies behaviour of blocking calls such as select, sleep tonon-blockingPatches the python standard socket library
  17. 17. Gevent● Greenlet 1 is running● Greenlet 2 and 3 are ready
  18. 18. Gevent● Greenlet 1 has to wait for read● Greenlet 1 switches to Event hub
  19. 19. Gevent● Event hub switches to Greenlet 3
  20. 20. Gevent● Greenlet 2 runs
  21. 21. Gevent● Greenlet 2 wants to sleep● Greenlet 2 switches to Event hub
  22. 22. Gevent● Greenlet 1 data has come, moved to ready state● Eventhub switches to Greenlet 3
  23. 23. Gevent● Greenlet 3 runs
  24. 24. Gevent● When Greenlet 1 resumes, its from next instruction● Its as if it were a blocking call
  25. 25. Green SSH Client1. Init session2. Authenticate3. Get a channel4. Issue command5. Read output
  26. 26. A closer look
  27. 27. Going Concurrent Use pre-forked processes to use all cores
  28. 28. Memory usage ssh mux memory usage 45 40 35 30Memory(MB) 25 Gevent+Processes 20 15 10 5 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 # Concurrent Reqs
  29. 29. SSH Mux Performace ssh mux performace chart 70 60 50 40 Time(s) Gevent+Processes 30 20 10 0 20 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs
  30. 30. SSH Mux memory usage ssh mux memory usage 600 500 400 Memory(MB) Processes 300 Gevent 200 100 0 0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 # Concurrent Reqs
  31. 31. SSH Mux Performance ssh mux performance sheet 160 140 120 100 ProcessesTime(s) 80 Gevent+Processes 60 40 20 0 20 40 80 150 200 400 800 1200 1500 1800 2100 # Concurrent Reqs
  32. 32. Gevent yay!● Untwist – write linear non blocking code● Explicit scheduling, dictate the execution flow● Timeouts● Events, AsyncResults for Synchronization● gevent.wsgi● Pre-spawned pool of greenlets
  33. 33. Gevent beware of● No multicore support● Not great for CPU bound applications● Third party libs must be green (non blocking)● Misbehaving workers can be lethal● No fairness when it comes to scheduling
  34. 34. Take Away● Gevent lets you write asynchronous code in a synchronous manner● No multicore support, still need multiprocessing● Not so great for CPU bound applications● Split your application into CPU bound and IO bound parts● Be willing to contribute patches● Code available at git@github.com:aaloksood/pyexamples.git
  35. 35. Thank you Thats all folks!
  36. 36. Countdown Timer● Count down from 200000000● Split work among workers
  37. 37. Threads Multithreading wonder 25 20 15 1 Core Time(s) 4 cores 10 5 0 1 2 3 4 5 6 7 8 9 10 # Workers
  38. 38. One core Execution time One Core 14.5 14 13.5 13 Processes 1 Core Time (s) 12.5 Gevent_1 12 Gevent_4 11.5 11 10.5 1 2 3 4 5 6 7 8 9 10 # Workers
  39. 39. Four cores Execution time 4 cores 25 20 Process 15 Threads Time(s) Gevent_1 10 Gevent_4 5 0 1 2 3 4 5 6 7 8 9 10 # Workers

×