Concurrency in Python
Upcoming SlideShare
Loading in...5
×
 

Concurrency in Python

on

  • 3,763 views

 

Statistics

Views

Total Views
3,763
Views on SlideShare
3,729
Embed Views
34

Actions

Likes
8
Downloads
59
Comments
0

3 Embeds 34

http://www.slideshare.net 31
https://www.linkedin.com 2
http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Concurrency in Python Concurrency in Python Presentation Transcript

  • Intro Threads GIL Processes Async Concurrency in Python Konrad Delong March 23 2010, Amsterdam Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async me IT student (Erasmus) Python Thesis Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async concurrency In computer science, concurrency is a property of systems in which several computations are executing simultaneously, and potentially interacting with each other. The computations may be executing on multiple cores in the same chip, preemptively time-shared threads on the same processor, or executed on physically separated processors. (en.wikipedia.org) Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async threads thread threading Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async thread import thread thread.start_new_thread(function, args[, kwargs]) thread.get_ident() thread.allocate_lock() Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async threading from threading import (Thread, Condition, Lock, RLock, Semaphore) class MyThread(Thread): def run(self) pass t = MyThread() # or t = Thread(target=function, args=[], kwargs={}) # and then t.start() t.join() Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async GIL Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async How it works? Interpreter in only one thread Released every check (== 100 ticks) and on blocking operations Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async check if (--_Py_Ticker < 0) { _Py_Ticker = _Py_CheckInterval; // default 100 if (interpreter_lock) { /* Give another thread a chance */ PyThread_release_lock(interpreter_lock); /* Other threads may run now */ PyThread_acquire_lock(interpreter_lock, 1); } } Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async Why GIL is bad? No gains on multicore Latency in responding to events Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async Why GIL is not that bad? Performance on single core Safe environment for C extentions Other solutions than threads exist Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async Remove GIL? Greg Stein, python-safethread PyPy, Unladen Swallow current policy on dropping GIL in cPython Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async Other solutions? Processes instead of threads Jython/IronPython Async Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async multiprocessing module from multiprocessing import (Process, Condition, Lock, RLock, Semaphore) class MyProcess(Process): def run(self) pass t = MyProcess() # or t = Process(target=function, args=[], kwargs={}) # and then t.start() t.join() Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async just like ... from threading import (Thread, Condition, Lock, RLock, Semaphore) class MyThread(Thread): def run(self) pass t = MyThread() # or t = Thread(target=function, args=[], kwargs={}) # and then t.start() t.join() Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async not entirely the same no implicit shared memory (global vars, imported modules, other) IPC (serializable data) separate machines IPC Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async Queues def foo(queue): print queue.get() q = Queue() t = Process(target=foo, args=[q]) t.start() q.put([1, "a"]) # will print [1, "a"] Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async Shared memory from multiprocessing import Value, Array num = Value(’d’, 0.0) arr = Array(’i’, range(10)) Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async Async callbacks no blocking still single CPU, but better scaling (memory management) explicit(Twisted, Tornado) vs hidden Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async Reactor Konrad Delong Concurrency in Python
  • Intro Threads GIL Processes Async Hidden async Kamaelia select, epoll, kqueue Eventlet, Gevent, PyEvent Konrad Delong Concurrency in Python