1. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
python-csp: bringing OCCAM to Python
Sarah Mount
Europython 2010
Sarah Mount python-csp: bringing OCCAM to Python

2. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
1 Ancient history
2 python-csp: features and idioms
Process creation
Parallel, sequential and repeated processes
Communication via channel read / writes
More channels: ALTing and choice
More channels: poisoning
Producer / consumer or worker / farmer models
3 Using built-in processes
4 Future challenges
Sarah Mount python-csp: bringing OCCAM to Python

3. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
This talk. . .
is all about the python-csp project. There is a similar project
called PyCSP, these will merge over the next few months. Current
code base is here: http://code.google.com/p/python-csp
Please do contribute bug fixes / issues / code / docs / rants / . . .
Sarah Mount python-csp: bringing OCCAM to Python

4. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
INMOS B0042 Board c David May
The Transputer was the original “multicore” machine and was built
to run the OCCAM language – a realisation of CSP.
http://www.cs.bris.ac.uk/~dave/transputer.html
Sarah Mount python-csp: bringing OCCAM to Python

5. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
Tony Hoare
Invented CSP, OCCAM, Quicksort and other baddass stuff. You
may remember him from such talks as his Europython 2009
Keynote.
Sarah Mount python-csp: bringing OCCAM to Python

6. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
OCCAM
OCCAM lives on in it’s current implementation as OCCAM-π
http://www.cs.kent.ac.uk/projects/ofa/kroc/
It can run on Lego bricks, Arduino boards and other things. Like
Python it uses indentation to demark scope. Unlike Python
OCCAM is MOSTLY IN UPPERCASE.
Sarah Mount python-csp: bringing OCCAM to Python

7. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
CSP and message passing
This next bit will be revision for many of you!
Sarah Mount python-csp: bringing OCCAM to Python

8. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
Most of us think of programs like this
http://xkcd.com/205/
Sarah Mount python-csp: bringing OCCAM to Python

9. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
This is the old “standard” model!
Multicore will soon take over the world (if it hasn’t already).
Shared memory concurrency / parallelism is hard:
Race hazards
Deadlock
Livelock
Tracking which lock goes with which data and what order
locks should be used.
Operating Systems are old news!
Sarah Mount python-csp: bringing OCCAM to Python

10. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
Message passing
Definition
In message passing concurrency (or parallelism) “everything” is a
process and no data is shared between processes. Instead, data is
“passed” between channels which connect processes together.
Think: OS processes and UNIX pipes.
The actor model (similar to Kamaelia) uses asynchronous
channels. Processes write data to a channel and continue
execution.
CSP (and π-calculus) use synchronous channels, or
rendezvous, where a process writing to a channel has to wait
until the value has been read by another process before
proceeding.
Sarah Mount python-csp: bringing OCCAM to Python

11. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
Process diagram
Sarah Mount python-csp: bringing OCCAM to Python

12. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
Why synchronous channels?
Leslie Lamport (1978). Time, clocks, and the ordering of events in
a distributed system”. Communications of the ACM 21 (7)
Sarah Mount python-csp: bringing OCCAM to Python

13. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
Hello World!
from c s p . c s p p r o c e s s i m p o r t ∗
@process
def h e l l o () :
p r i n t ’ H e l l o CSP w o r l d ! ’
hello () . start ()
Sarah Mount python-csp: bringing OCCAM to Python

14. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
Example of process creation: web server
@process
def s e r v e r ( host , port ) :
sock = socket . socket ( . . . )
# S e t up s o c k e t s
w h i l e True :
conn , a d d r = s o c k . a c c e p t ( )
r e q u e s t = conn . r e c v ( 4 0 9 6 ) . s t r i p ( )
i f r e q u e s t . s t a r t s w i t h ( ’GET ’ ) :
ok ( r e q u e s t , conn ) . s t a r t ( )
else :
n o t f o u n d ( r e q u e s t , conn ) . s t a r t ( )
Sarah Mount python-csp: bringing OCCAM to Python

15. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
Example of process creation: web server
@process
d e f n o t f o u n d ( r e q u e s t , conn ) :
page = ’<h1>F i l e n o t found </h1> ’
conn . s e n d ( r e s p o n s e ( 4 0 4 ,
’ Not Found ’ ,
page ) )
conn . shutdown ( s o c k e t . SHUT RDWR)
conn . c l o s e ( )
return
Sarah Mount python-csp: bringing OCCAM to Python

16. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
Combining processes: in parallel
# With a Par o b j e c t :
Par ( p1 , p2 , p3 , . . . pn ) . s t a r t ( )
# With s y n t a c t i c s u g a r :
p1 // ( p2 , p3 , . . . pn )
# RHS must be a s e q u e n c e t y p e .
Sarah Mount python-csp: bringing OCCAM to Python

17. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
Combining processes: repeating a process sequentially
@process
def h e l l o () :
p r i n t ’ H e l l o CSP w o r l d ! ’
if name == ’ m a i n ’ :
hello () ∗ 3
2 ∗ hello ()
Sarah Mount python-csp: bringing OCCAM to Python

18. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
Channels
@process
d e f c l i e n t ( chan ) :
p r i n t chan . r e a d ( )
@process
d e f s e r v e r ( chan ) :
chan . w r i t e ( ’ H e l l o CSP w o r l d ! ’ )
if name == ’ m a i n ’ :
ch = C h a n n e l ( )
Par ( c l i e n t ( ch ) , s e r v e r ( ch ) ) . s t a r t ( )
Sarah Mount python-csp: bringing OCCAM to Python

19. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
What you need to know about channels
Channels are currently UNIX pipes
This will likely change
Channel rendezvous is “slow” compared with JCSP:
200µ s vs 16µ s
This will be fixed by having channels written in C
Because channels are, at the OS level, open “files”, there can
only be a limited number of them (around 300 on my
machine)
This makes me sad :-(
Sarah Mount python-csp: bringing OCCAM to Python

20. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
Message passing an responsiveness
This isn’t about speed.
Sometimes, you have several channels and reading from them
all round-robin may reduce responsiveness if one channel read
blocks for a long time.
This is BAD:
@process
def foo ( channels ) :
f o r chan i n c h a n n e l s :
p r i n t chan . r e a d ( )
Sarah Mount python-csp: bringing OCCAM to Python

21. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
ALTing and choice
Definition
ALTing, or non-deterministic choice, selects a channel to read from
from those channels which are ready to read from.
You can do this with (only) two channels:
@process
d e f f o o ( c1 , c2 ) :
p r i n t c1 | c2
p r i n t c1 | c2
Sarah Mount python-csp: bringing OCCAM to Python

22. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
ALTing proper (many channels)
@process
d e f f o o ( c1 , c2 , c3 , c4 , c5 ) :
a l t = A l t ( c1 , c2 , c3 , c4 , c5 )
# Choose random a v a i l a b l e o f f e r
print alt . se le ct ()
# Avoid l a s t s e l e c t e d channel
print alt . f a i r s e l e c t ()
# Choose c h a n n e l w i t h l o w e s t i n d e x
print alt . p r i s e l e c t ()
Sarah Mount python-csp: bringing OCCAM to Python

23. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
Syntactic sugar for ALTing
@process
d e f f o o ( c1 , c2 , c3 , c4 , c5 ) :
gen = A l t ( c1 , c2 , c3 , c4 , c5 )
p r i n t gen . n e x t ( )
p r i n t gen . n e x t ( )
p r i n t gen . n e x t ( )
p r i n t gen . n e x t ( )
p r i n t gen . n e x t ( )
Sarah Mount python-csp: bringing OCCAM to Python

24. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
ALTing: when you really, really must return right now
@process
d e f f o o ( c1 , c2 , c3 ) :
# Skip () w i l l / always / complete
# a read immediately
gen = A l t ( c1 , c2 , c3 , S k i p ( ) )
p r i n t gen . n e x t ( )
...
Sarah Mount python-csp: bringing OCCAM to Python

25. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
Channel poisoning
Definition
Idiomatically in this style of programming most processes contain
infinite loops. Poisoning a channel asks all processes connected to
that channel to shut down automatically. Each process which has
a poisoned channel will automatically poison any other channels it
is connected to. This way, a whole graph of connected processes
can be terminated, avoiding race conditions.
@process
def foo ( channel ) :
...
channel . poison ()
Sarah Mount python-csp: bringing OCCAM to Python

26. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
A bigger example: Mandelbrot generator
Sarah Mount python-csp: bringing OCCAM to Python

27. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
@process
d e f main ( IMSIZE ) :
channels = [ ]
# One p r o d u c e r + c h a n n e l f o r e a c h
image column .
f o r x i n r a n g e ( IMSIZE [ 0 ] ) :
c h a n n e l s . append ( C h a n n e l ( ) )
p r o c e s s e s . append ( m a n d e l b r o t ( x ,
. . . channels [ x ]) )
p r o c e s s e s . i n s e r t ( 0 , consume ( IMSIZE ,
channels ) )
mandel = Par ( ∗ p r o c e s s e s )
mandel . s t a r t ( )
Sarah Mount python-csp: bringing OCCAM to Python

28. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
@process
def mandelbrot ( xcoord , cout ) :
# Do some maths h e r e
cout . w r i t e ( xcoord , column data )
Sarah Mount python-csp: bringing OCCAM to Python

29. Process creation
Ancient history Parallel, sequential and repeated processes
python-csp: features and idioms Communication via channel read / writes
Using built-in processes More channels: ALTing and choice
Future challenges More channels: poisoning
Producer / consumer or worker / farmer models
@process
d e f consumer ( c i n s ) :
# I n i t i a l i s e PyGame . . .
gen = l e n ( c i n s ) ∗ A l t ( ∗ c i n s )
f o r i in range ( len ( c i n s ) ) :
x c o o r d , column = gen . n e x t ( )
# B l i t t h a t column t o s c r e e n
f o r e v e n t i n pygame . e v e n t . g e t ( ) :
i f e v e n t . t y p e == pygame . QUIT :
for channel in cins :
channel . poison ()
pygame . q u i t ( )
Sarah Mount python-csp: bringing OCCAM to Python

30. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
Oscilloscope
Sarah Mount python-csp: bringing OCCAM to Python

31. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
@process
def O s c i ll o s c o p e ( inchan ) :
# I n i t i a l i s e PyGame
ydata = [ ]
w h i l e n o t QUIT :
y d a t a . append ( i n c h a n . r e a d ( ) )
y d a t a . pop ( 0 )
# B l i t data to s c r e e n . . .
# Deal with e v e n t s
Sarah Mount python-csp: bringing OCCAM to Python

32. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
from c s p . b u i l t i n s i m p o r t S i n
def t r a c e s i n () :
chans = Channel ( ) , Channel ( )
Par ( G e n e r a t e F l o a t s ( c h a n s [ 0 ] ) ,
Sin ( chans [ 0 ] , chans [ 1 ] ) ,
O s c i l l o s c o p e ( chans [ 1 ] ) ) . s t a r t ()
return
Sarah Mount python-csp: bringing OCCAM to Python

33. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
Process diagram
Sarah Mount python-csp: bringing OCCAM to Python

34. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
def trace mux () :
chans = [ Channel ( ) f o r i i n range (6) ]
p a r = Par ( G e n e r a t e F l o a t s ( c h a n s [ 0 ] ) ,
Delta2 ( chans [ 0 ] , chans [ 1 ] ,
chans [ 2 ] ) ,
Cos ( c h a n s [ 1 ] , c h a n s [ 3 ] ) ,
Sin ( chans [ 2 ] , chans [ 4 ] ) ,
Mux2 ( c h a n s [ 3 ] , c h a n s [ 4 ] ,
chans [ 5 ] ) ,
O s c i l l o s c o p e ( chans [ 5 ] ) )
par . s t a r t ()
Sarah Mount python-csp: bringing OCCAM to Python

35. Ancient history
python-csp: features and idioms
Using built-in processes
Future challenges
Wiring
Sarah Mount python-csp: bringing OCCAM to Python