what every web and app developer should know about multithreading

why

why On the desktop, one core is rarely enough On the web, sometimes we need parallel execution Performance requires caching Persistence of connectivity requires responsiveness Disk and network I/O is indispensible and v ery slow

On the desktop, one core is rarely enough

On the web, sometimes we need parallel execution

Performance requires caching

Persistence of connectivity requires responsiveness

Disk and network I/O is indispensible and v ery slow

threads A way to execute two things at once

A way to execute two things at once

threads A way to execute two things almost at once Lightweight Independent execution Almost like a separate process

A way to execute two things almost at once

Lightweight

Independent execution

Almost like a separate process

thread process versus

thread A process is isolated in memory When it dies, its memory is released When it dies, its threads die too Somewhat difficult to talk to other processes versus

A process is isolated in memory

When it dies, its memory is released

When it dies, its threads die too

Somewhat difficult to talk to other processes

All threads in a process share memory Can be started and stopped as needed On some platforms, cheaper to launch than a process Can be native (kernel-based) or user-mode process versus

All threads in a process share memory

Can be started and stopped as needed

On some platforms, cheaper to launch than a process

Can be native (kernel-based) or user-mode

threads

threads Less predictable execution Must control for re-entrancy of code Must be aware of shared data More difficult than it seems

Less predictable execution

Must control for re-entrancy of code

Must be aware of shared data

More difficult than it seems

synchronization We must retain predictability in our programs Two threads fighting for the same variable Thread A my_local_x = x set x = my_local_x + 1 Thread B my_local_x = x set x = my_local_x + 2

We must retain predictability in our programs

Two threads fighting for the same variable

synchronization We must retain predictability in our programs Two threads fighting for the same variable Thread A my_local_x = x set x = my_local_x + 1 Thread B my_local_x = x set x = my_local_x + 2 If we started out with x = 2, we end up with x = 5

We must retain predictability in our programs

Two threads fighting for the same variable

If we started out with x = 2, we end up with x = 5

synchronization We must retain predictability in our programs Two threads fighting for the same variable Thread A my_local_x = x set x = my_local_x + 1 Thread B my_local_x = x set x = my_local_x + 2 If we started out with x = 2, we end up with x = 5

We must retain predictability in our programs

Two threads fighting for the same variable

If we started out with x = 2, we end up with x = 5

synchronization We must retain predictability in our programs Two threads fighting for the same variable Thread A my_local_x = x set x = my_local_x + 1 Thread B my_local_x = x set x = my_local_x + 2 If we started out with x = 2, we end up with x = 4

We must retain predictability in our programs

Two threads fighting for the same variable

If we started out with x = 2, we end up with x = 4

synchronization First rule of synchronization: avoid needing it Thread-local storage Function scope variables No side effects Functional languages list.sort()

First rule of synchronization: avoid needing it

Thread-local storage

Function scope variables

No side effects

Functional languages

synchronization First rule of synchronization: avoid needing it Thread-local storage Function scope variables No side effects Functional languages list.sort() newlist = list.sort()

First rule of synchronization: avoid needing it

Thread-local storage

Function scope variables

No side effects

Functional languages

synchronization Second rule of synchronization: join threads Use a worker thread Join – wait for it to finish, then read its results

Second rule of synchronization: join threads

Use a worker thread

Join – wait for it to finish, then read its results

synchronization Thread A Thread B Main Thread Start A Start B Join A and B Read data

synchronization Third rule of synchronization: go critical Declare a critical section You are alone within your application… … until you end the critical section Application-wide setting Hard to use when you have a bunch of threads

Third rule of synchronization: go critical

Declare a critical section

You are alone within your application…

… until you end the critical section

Application-wide setting

Hard to use when you have a bunch of threads

synchronization Third rule of synchronization: mutual exclusion

Third rule of synchronization: mutual exclusion

synchronization Third rule of synchronization: mut ual ex clusion

Third rule of synchronization: mut ual ex clusion

synchronization Third rule of synchronization: mutex Allows you to “lock” and “unlock” resources Like an object for a mini-critical section Thread A lock M my_local_x = x set x = my_local_x + 1 unlock M Thread B lock M my_local_x = x set x = my_local_x + 1 unlock M

Third rule of synchronization: mutex

Allows you to “lock” and “unlock” resources

Like an object for a mini-critical section

Thread A

lock M

my_local_x = x

set x = my_local_x + 1

unlock M

Thread B

lock M

my_local_x = x

set x = my_local_x + 1

unlock M

synchronization Third rule of synchronization: use a semaphore Like a mutex, but lets more than one thread through Mutex with a counter Checks availability, then “acquires” one count When done, “releases” one count, unblocking others

Third rule of synchronization: use a semaphore

Like a mutex, but lets more than one thread through

Mutex with a counter

Checks availability, then “acquires” one count

When done, “releases” one count, unblocking others

synchronization Synchronization is about blocking Allows you to control access to code and data Protects areas of code that shouldn’t be left to chance

Synchronization is about blocking

Allows you to control access to code and data

Protects areas of code that shouldn’t be left to chance

examples Worker threads Thread pools Producer / Consumer Cache Will use .NET for examples

Worker threads

Thread pools

Producer / Consumer

Cache

Will use .NET for examples

example

example

example

example

bad threading

bad threading It is simple when simple, and fiendish when complex Watch out for race conditions: lock often to prevent Watch for deadlocks: don’t lock too much Watch for incomplete locks: lock carefully

It is simple when simple, and fiendish when complex

Watch out for race conditions: lock often to prevent

Watch for deadlocks: don’t lock too much

Watch for incomplete locks: lock carefully

bad threading Lock for the smallest amount of time If possible, lock for consistency… … then copy the data and use it locally Instead of blocking on locks, wait with a timeout Use lots of debug logging if you’re in trouble

Lock for the smallest amount of time

If possible, lock for consistency…

… then copy the data and use it locally

Instead of blocking on locks, wait with a timeout

Use lots of debug logging if you’re in trouble

photo credits CC BY-NC-ND 2.0 – Clemens Schwaighofer – http://flickr.com/photos/gullevek/257135337/ CC BY-NC-SA 2.0 – Robert Parviainen – http://flickr.com/photos/rtv/2574427997/ CC BY-NC-ND 2.0 – Sudhir Srinivasa – http://flickr.com/photos/sudhirs/111760673/ CC BY-NC-SA 2.0 – Rick Harrison – http://flickr.com/photos/sovietuk/2657691123/ CC BY-NC-ND 2.0 – Joe Chiapputo – http://flickr.com/photos/cocoabeachjoe/1924133031/ CC BY-ND 2.0 – Craig Allen – http://flickr.com/photos/anabadili/2759448841/