python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges python-csp: bringing OCCAM to Python Sarah Mount Europython 2010 Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges 1 Ancient history 2 python-csp: features and idioms Process creation Parallel, sequential and repeated processes Communication via channel read / writes More channels: ALTing and choice More channels: poisoning Producer / consumer or worker / farmer models 3 Using built-in processes 4 Future challenges Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges This talk. . . is all about the python-csp project. There is a similar project called PyCSP, these will merge over the next few months. Current code base is here: http://code.google.com/p/python-csp Please do contribute bug ﬁxes / issues / code / docs / rants / . . . Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges INMOS B0042 Board c David May The Transputer was the original “multicore” machine and was built to run the OCCAM language – a realisation of CSP. http://www.cs.bris.ac.uk/~dave/transputer.html Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges Tony Hoare Invented CSP, OCCAM, Quicksort and other baddass stuﬀ. You may remember him from such talks as his Europython 2009 Keynote. Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges OCCAM OCCAM lives on in it’s current implementation as OCCAM-π http://www.cs.kent.ac.uk/projects/ofa/kroc/ It can run on Lego bricks, Arduino boards and other things. Like Python it uses indentation to demark scope. Unlike Python OCCAM is MOSTLY IN UPPERCASE. Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges CSP and message passing This next bit will be revision for many of you! Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges Most of us think of programs like this http://xkcd.com/205/ Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges This is the old “standard” model! Multicore will soon take over the world (if it hasn’t already). Shared memory concurrency / parallelism is hard: Race hazards Deadlock Livelock Tracking which lock goes with which data and what order locks should be used. Operating Systems are old news! Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges Message passing Deﬁnition In message passing concurrency (or parallelism) “everything” is a process and no data is shared between processes. Instead, data is “passed” between channels which connect processes together. Think: OS processes and UNIX pipes. The actor model (similar to Kamaelia) uses asynchronous channels. Processes write data to a channel and continue execution. CSP (and π-calculus) use synchronous channels, or rendezvous, where a process writing to a channel has to wait until the value has been read by another process before proceeding. Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges Process diagram Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges Why synchronous channels? Leslie Lamport (1978). Time, clocks, and the ordering of events in a distributed system”. Communications of the ACM 21 (7) Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Hello World! from c s p . c s p p r o c e s s i m p o r t ∗ @process def h e l l o () : p r i n t ’ H e l l o CSP w o r l d ! ’ hello () . start () Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Example of process creation: web server @process def s e r v e r ( host , port ) : sock = socket . socket ( . . . ) # S e t up s o c k e t s w h i l e True : conn , a d d r = s o c k . a c c e p t ( ) r e q u e s t = conn . r e c v ( 4 0 9 6 ) . s t r i p ( ) i f r e q u e s t . s t a r t s w i t h ( ’GET ’ ) : ok ( r e q u e s t , conn ) . s t a r t ( ) else : n o t f o u n d ( r e q u e s t , conn ) . s t a r t ( ) Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Example of process creation: web server @process d e f n o t f o u n d ( r e q u e s t , conn ) : page = ’<h1>F i l e n o t found </h1> ’ conn . s e n d ( r e s p o n s e ( 4 0 4 , ’ Not Found ’ , page ) ) conn . shutdown ( s o c k e t . SHUT RDWR) conn . c l o s e ( ) return Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Combining processes: in parallel # With a Par o b j e c t : Par ( p1 , p2 , p3 , . . . pn ) . s t a r t ( ) # With s y n t a c t i c s u g a r : p1 // ( p2 , p3 , . . . pn ) # RHS must be a s e q u e n c e t y p e . Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Combining processes: repeating a process sequentially @process def h e l l o () : p r i n t ’ H e l l o CSP w o r l d ! ’ if name == ’ m a i n ’ : hello () ∗ 3 2 ∗ hello () Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Channels @process d e f c l i e n t ( chan ) : p r i n t chan . r e a d ( ) @process d e f s e r v e r ( chan ) : chan . w r i t e ( ’ H e l l o CSP w o r l d ! ’ ) if name == ’ m a i n ’ : ch = C h a n n e l ( ) Par ( c l i e n t ( ch ) , s e r v e r ( ch ) ) . s t a r t ( ) Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models What you need to know about channels Channels are currently UNIX pipes This will likely change Channel rendezvous is “slow” compared with JCSP: 200µ s vs 16µ s This will be ﬁxed by having channels written in C Because channels are, at the OS level, open “ﬁles”, there can only be a limited number of them (around 300 on my machine) This makes me sad :-( Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Message passing an responsiveness This isn’t about speed. Sometimes, you have several channels and reading from them all round-robin may reduce responsiveness if one channel read blocks for a long time. This is BAD: @process def foo ( channels ) : f o r chan i n c h a n n e l s : p r i n t chan . r e a d ( ) Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models ALTing and choice Deﬁnition ALTing, or non-deterministic choice, selects a channel to read from from those channels which are ready to read from. You can do this with (only) two channels: @process d e f f o o ( c1 , c2 ) : p r i n t c1 | c2 p r i n t c1 | c2 Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models ALTing proper (many channels) @process d e f f o o ( c1 , c2 , c3 , c4 , c5 ) : a l t = A l t ( c1 , c2 , c3 , c4 , c5 ) # Choose random a v a i l a b l e o f f e r print alt . se le ct () # Avoid l a s t s e l e c t e d channel print alt . f a i r s e l e c t () # Choose c h a n n e l w i t h l o w e s t i n d e x print alt . p r i s e l e c t () Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Syntactic sugar for ALTing @process d e f f o o ( c1 , c2 , c3 , c4 , c5 ) : gen = A l t ( c1 , c2 , c3 , c4 , c5 ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) p r i n t gen . n e x t ( ) Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models ALTing: when you really, really must return right now @process d e f f o o ( c1 , c2 , c3 ) : # Skip () w i l l / always / complete # a read immediately gen = A l t ( c1 , c2 , c3 , S k i p ( ) ) p r i n t gen . n e x t ( ) ... Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models Channel poisoning Deﬁnition Idiomatically in this style of programming most processes contain inﬁnite loops. Poisoning a channel asks all processes connected to that channel to shut down automatically. Each process which has a poisoned channel will automatically poison any other channels it is connected to. This way, a whole graph of connected processes can be terminated, avoiding race conditions. @process def foo ( channel ) : ... channel . poison () Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models A bigger example: Mandelbrot generator Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models @process d e f main ( IMSIZE ) : channels = [ ] # One p r o d u c e r + c h a n n e l f o r e a c h image column . f o r x i n r a n g e ( IMSIZE [ 0 ] ) : c h a n n e l s . append ( C h a n n e l ( ) ) p r o c e s s e s . append ( m a n d e l b r o t ( x , . . . channels [ x ]) ) p r o c e s s e s . i n s e r t ( 0 , consume ( IMSIZE , channels ) ) mandel = Par ( ∗ p r o c e s s e s ) mandel . s t a r t ( ) Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models @process def mandelbrot ( xcoord , cout ) : # Do some maths h e r e cout . w r i t e ( xcoord , column data ) Sarah Mount python-csp: bringing OCCAM to Python

Process creation Ancient history Parallel, sequential and repeated processes python-csp: features and idioms Communication via channel read / writes Using built-in processes More channels: ALTing and choice Future challenges More channels: poisoning Producer / consumer or worker / farmer models @process d e f consumer ( c i n s ) : # I n i t i a l i s e PyGame . . . gen = l e n ( c i n s ) ∗ A l t ( ∗ c i n s ) f o r i in range ( len ( c i n s ) ) : x c o o r d , column = gen . n e x t ( ) # B l i t t h a t column t o s c r e e n f o r e v e n t i n pygame . e v e n t . g e t ( ) : i f e v e n t . t y p e == pygame . QUIT : for channel in cins : channel . poison () pygame . q u i t ( ) Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges Oscilloscope Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges @process def O s c i ll o s c o p e ( inchan ) : # I n i t i a l i s e PyGame ydata = [ ] w h i l e n o t QUIT : y d a t a . append ( i n c h a n . r e a d ( ) ) y d a t a . pop ( 0 ) # B l i t data to s c r e e n . . . # Deal with e v e n t s Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges from c s p . b u i l t i n s i m p o r t S i n def t r a c e s i n () : chans = Channel ( ) , Channel ( ) Par ( G e n e r a t e F l o a t s ( c h a n s [ 0 ] ) , Sin ( chans [ 0 ] , chans [ 1 ] ) , O s c i l l o s c o p e ( chans [ 1 ] ) ) . s t a r t () return Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges Process diagram Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges def trace mux () : chans = [ Channel ( ) f o r i i n range (6) ] p a r = Par ( G e n e r a t e F l o a t s ( c h a n s [ 0 ] ) , Delta2 ( chans [ 0 ] , chans [ 1 ] , chans [ 2 ] ) , Cos ( c h a n s [ 1 ] , c h a n s [ 3 ] ) , Sin ( chans [ 2 ] , chans [ 4 ] ) , Mux2 ( c h a n s [ 3 ] , c h a n s [ 4 ] , chans [ 5 ] ) , O s c i l l o s c o p e ( chans [ 5 ] ) ) par . s t a r t () Sarah Mount python-csp: bringing OCCAM to Python

Ancient history python-csp: features and idioms Using built-in processes Future challenges Wiring Sarah Mount python-csp: bringing OCCAM to Python

http://python-csp.posterous.com	142
http://snim2.blogspot.com	12
http://translate.googleusercontent.com	2
http://us-w1.rockmelt.com	1
http://snim2.blogspot.nl	1
http://lanyrd.com	1
http://www.linkedin.com	1
http://snim2.blogspot.ro	1

python-csp: bringing OCCAM to Python

by Sarah Mount

Accessibility

Categories

Upload Details

Usage Rights

8 Embeds 161

Statistics