Concurrency in Python

Intro Threads GIL Processes Async Concurrency in Python Konrad Delong March 23 2010, Amsterdam Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async me IT student (Erasmus) Python Thesis Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async concurrency In computer science, concurrency is a property of systems in which several computations are executing simultaneously, and potentially interacting with each other. The computations may be executing on multiple cores in the same chip, preemptively time-shared threads on the same processor, or executed on physically separated processors. (en.wikipedia.org) Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async threads thread threading Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async thread import thread thread.start_new_thread(function, args[, kwargs]) thread.get_ident() thread.allocate_lock() Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async threading from threading import (Thread, Condition, Lock, RLock, Semaphore) class MyThread(Thread): def run(self) pass t = MyThread() # or t = Thread(target=function, args=[], kwargs={}) # and then t.start() t.join() Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async GIL Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async How it works? Interpreter in only one thread Released every check (== 100 ticks) and on blocking operations Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async check if (--_Py_Ticker < 0) { _Py_Ticker = _Py_CheckInterval; // default 100 if (interpreter_lock) { /* Give another thread a chance */ PyThread_release_lock(interpreter_lock); /* Other threads may run now */ PyThread_acquire_lock(interpreter_lock, 1); } } Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async Why GIL is bad? No gains on multicore Latency in responding to events Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async Why GIL is not that bad? Performance on single core Safe environment for C extentions Other solutions than threads exist Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async Remove GIL? Greg Stein, python-safethread PyPy, Unladen Swallow current policy on dropping GIL in cPython Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async Other solutions? Processes instead of threads Jython/IronPython Async Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async multiprocessing module from multiprocessing import (Process, Condition, Lock, RLock, Semaphore) class MyProcess(Process): def run(self) pass t = MyProcess() # or t = Process(target=function, args=[], kwargs={}) # and then t.start() t.join() Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async just like ... from threading import (Thread, Condition, Lock, RLock, Semaphore) class MyThread(Thread): def run(self) pass t = MyThread() # or t = Thread(target=function, args=[], kwargs={}) # and then t.start() t.join() Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async not entirely the same no implicit shared memory (global vars, imported modules, other) IPC (serializable data) separate machines IPC Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async Queues def foo(queue): print queue.get() q = Queue() t = Process(target=foo, args=[q]) t.start() q.put([1, "a"]) # will print [1, "a"] Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async Shared memory from multiprocessing import Value, Array num = Value(’d’, 0.0) arr = Array(’i’, range(10)) Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async Async callbacks no blocking still single CPU, but better scaling (memory management) explicit(Twisted, Tornado) vs hidden Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async Reactor Konrad Delong Concurrency in Python

Intro Threads GIL Processes Async Hidden async Kamaelia select, epoll, kqueue Eventlet, Gevent, PyEvent Konrad Delong Concurrency in Python

Concurrency in Python

by konryd on Mar 24, 2010

Accessibility

Tags

Upload Details

Usage Rights

1 Embed 31

Statistics

Concurrency in Python — Presentation Transcript