Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Concurrency in Python

2,118 views

Published on

Published in: Technology, Education
  • Be the first to comment

Concurrency in Python

  1. 1. Concurrency with Multiprocessing in PythonPhillyPug & Philly.rb RedSnake MeetingFebruary 8th, 2011Gavin M. RoymyYearbook.com
  2. 2. green threads in threading
  3. 3. from threading import Threadimport timeclass MyThread(Thread): def run(self): print "%s running" % self.name time.sleep(5) print "%s done" % self.namefor x in xrange(0,10): thread = MyThread() thread.start() thread.join(0)
  4. 4. gmr-0x04:pika gmr$ python threads.pyMyThread-1 runningMyThread-2 runningMyThread-3 runningMyThread-4 runningMyThread-5 runningMyThread-6 runningMyThread-7 runningMyThread-8 runningMyThread-9 runningMyThread-10 runningMyThread-1 doneMyThread-2 doneMyThread-3 doneMyThread-4 doneMyThread-5 doneMyThread-6 doneMyThread-7 doneMyThread-8 doneMyThread-9 doneMyThread-10 done
  5. 5. threading• Locks• Reentrant Locks Reentrant knows who own the locks and the recursion level• Conditions Additional classes like Queue.Queue• Semaphores• Events• Timers
  6. 6. This GIL From David Beeazley’s GIL Visualization http://www.dabeaz.com/GIL/gilvis/fourthread.html
  7. 7. enter multiprocessing
  8. 8. multiprocessing module• All the things threading has • Connections• Exchanging Objects • Managers TCP Server coordinating shared objects • Queues and Pipes • SyncManager Process Sync• Shared State • Logging • Pipes and Queues• Pools
  9. 9. from multiprocessing import Processimport timeclass MyThread(Process): def run(self): print "%s running" % self.name time.sleep(5) print "%s done" % self.namefor x in xrange(0,10): thread = MyThread() thread.start() thread.join(0)
  10. 10. gmr-0x04:pika gmr$ python processes.pyThread-1 runningThread-2 runningThread-3 runningThread-4 runningThread-5 running Thread-6 running Thread-7 runningThread-8 runningThread-9 runningThread-10 runningThread-1 doneThread-2 doneThread-3 doneThread-4 doneThread-5 doneThread-6 doneThread-7 doneThread-8 doneThread-9 doneThread-10 done
  11. 11. multiprocessing.reduction Photo By Susan NYC: http://www.flickr.com/photos/en321/33868864/
  12. 12. # Process #1from multiprocessing.reduction import reduce_handleimport socket# Create a socketsock = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)# Do stuff here, client or server wise# Create the pickled socket handlehandle = reduce_handle(sock.fileno)# Process #2from multiprocessing.reduction import rebuild_handleimport socket# In other processfd = rebuild_handle(handle)sock = socket.fromfd(fd, socket.AF_INET, socket.SOCK_STREAM)# Now I can read and write from the socket too

×