Concurrency in Python

3,102 views

Published on

0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
3,102
On SlideShare
0
From Embeds
0
Number of Embeds
38
Actions
Shares
0
Downloads
66
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide

Concurrency in Python

  1. 1. Intro Threads GIL Processes Async Concurrency in Python Konrad Delong March 23 2010, Amsterdam Konrad Delong Concurrency in Python
  2. 2. Intro Threads GIL Processes Async me IT student (Erasmus) Python Thesis Konrad Delong Concurrency in Python
  3. 3. Intro Threads GIL Processes Async concurrency In computer science, concurrency is a property of systems in which several computations are executing simultaneously, and potentially interacting with each other. The computations may be executing on multiple cores in the same chip, preemptively time-shared threads on the same processor, or executed on physically separated processors. (en.wikipedia.org) Konrad Delong Concurrency in Python
  4. 4. Intro Threads GIL Processes Async threads thread threading Konrad Delong Concurrency in Python
  5. 5. Intro Threads GIL Processes Async thread import thread thread.start_new_thread(function, args[, kwargs]) thread.get_ident() thread.allocate_lock() Konrad Delong Concurrency in Python
  6. 6. Intro Threads GIL Processes Async threading from threading import (Thread, Condition, Lock, RLock, Semaphore) class MyThread(Thread): def run(self) pass t = MyThread() # or t = Thread(target=function, args=[], kwargs={}) # and then t.start() t.join() Konrad Delong Concurrency in Python
  7. 7. Intro Threads GIL Processes Async GIL Konrad Delong Concurrency in Python
  8. 8. Intro Threads GIL Processes Async How it works? Interpreter in only one thread Released every check (== 100 ticks) and on blocking operations Konrad Delong Concurrency in Python
  9. 9. Intro Threads GIL Processes Async check if (--_Py_Ticker < 0) { _Py_Ticker = _Py_CheckInterval; // default 100 if (interpreter_lock) { /* Give another thread a chance */ PyThread_release_lock(interpreter_lock); /* Other threads may run now */ PyThread_acquire_lock(interpreter_lock, 1); } } Konrad Delong Concurrency in Python
  10. 10. Intro Threads GIL Processes Async Why GIL is bad? No gains on multicore Latency in responding to events Konrad Delong Concurrency in Python
  11. 11. Intro Threads GIL Processes Async Why GIL is not that bad? Performance on single core Safe environment for C extentions Other solutions than threads exist Konrad Delong Concurrency in Python
  12. 12. Intro Threads GIL Processes Async Remove GIL? Greg Stein, python-safethread PyPy, Unladen Swallow current policy on dropping GIL in cPython Konrad Delong Concurrency in Python
  13. 13. Intro Threads GIL Processes Async Other solutions? Processes instead of threads Jython/IronPython Async Konrad Delong Concurrency in Python
  14. 14. Intro Threads GIL Processes Async multiprocessing module from multiprocessing import (Process, Condition, Lock, RLock, Semaphore) class MyProcess(Process): def run(self) pass t = MyProcess() # or t = Process(target=function, args=[], kwargs={}) # and then t.start() t.join() Konrad Delong Concurrency in Python
  15. 15. Intro Threads GIL Processes Async just like ... from threading import (Thread, Condition, Lock, RLock, Semaphore) class MyThread(Thread): def run(self) pass t = MyThread() # or t = Thread(target=function, args=[], kwargs={}) # and then t.start() t.join() Konrad Delong Concurrency in Python
  16. 16. Intro Threads GIL Processes Async not entirely the same no implicit shared memory (global vars, imported modules, other) IPC (serializable data) separate machines IPC Konrad Delong Concurrency in Python
  17. 17. Intro Threads GIL Processes Async Queues def foo(queue): print queue.get() q = Queue() t = Process(target=foo, args=[q]) t.start() q.put([1, "a"]) # will print [1, "a"] Konrad Delong Concurrency in Python
  18. 18. Intro Threads GIL Processes Async Shared memory from multiprocessing import Value, Array num = Value(’d’, 0.0) arr = Array(’i’, range(10)) Konrad Delong Concurrency in Python
  19. 19. Intro Threads GIL Processes Async Async callbacks no blocking still single CPU, but better scaling (memory management) explicit(Twisted, Tornado) vs hidden Konrad Delong Concurrency in Python
  20. 20. Intro Threads GIL Processes Async Reactor Konrad Delong Concurrency in Python
  21. 21. Intro Threads GIL Processes Async Hidden async Kamaelia select, epoll, kqueue Eventlet, Gevent, PyEvent Konrad Delong Concurrency in Python

×