Testing Multithreaded Java Applications for Synchronization Problems, ISTA 2011

www.istabg.orgwww.vmware.com www.istabg.orgwww.vmware.com
Testing Multithreaded Java
Applications for
Synchronization Problems
Vassil Popovski

www.istabg.orgwww.vmware.com
Why multithreading?
A modern CPU
…
core 1
cache
core N
cache
shared memory

Why multithreading?
…Thread 1 Thread N
Java heap
A modern Java
Application
synchronization
thread 1 thread N
Java heap

• Non-determinism
• New types of defects exist such as deadlocks,
livelocks and race conditions
New concept -> new problems

Thread 2Thread 1
Thread interleaving
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2

T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2 T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2

Number of different thread interleavings

Q: What is the number of all interleavings for 3
threads with 2 blocks each?
QUIZ#1

• safety: nothing bad happens
• liveness: something good eventually happens
What to test for?

Image source: http://www.doc.ic.ac.uk/~jnm/book/ppt/ch7.ppt
Example 1: Single Lane Bridge
• safety:
– no car crash
• liveness:
– every car eventually get an opportunity to cross the bridge

Example 2: BoundedBuffer
Producer 1
Producer 2
Producer 3
Producer N
Consumer 1
Consumer 2
Consumer M
…
…
• safety:
– If empty, must not allow Get(); If full, must not allow Put()
• liveness (quiz#2):
– Put() on empty and Get() on full will eventually be allowed

How to test for synchronization issues?
Why
• NASA’s Remote Agent
• Therac-25

How to test for synchronization issues?
• Load/Stress testing (blackbox, whitebox)
• Specific interleavings testing (whitebox)
• All interleavings testing (whitebox)
• Instrumentation (blackbox)

• A lot of threads and operations to exercise
different interleavings
Load/Stress testing

Demo!

Deadlock example
Thread 2Thread 1
synchronized ( A ) {
synchronized ( B ) {
synchronized ( B ) {
synchronized ( A ) {
Deadlock !!!

• Tools
– ExecutorService -
http://download.oracle.com/javase/6/docs/api/java/util/concurrent/E
– TestNG -
http://testng.org/doc/documentation-main.html#annotations
– JUnit -
http://www.junit.org/apidocs/junit/extensions/ActiveTestSuite.html
• JUnitPerf -
http://testng.org/doc/documentation-main.html#annotations
– Custom Threading
– GroboUtils -
http://groboutils.sourceforge.net/testing-junit/using_mtt.html
Load/Stress testing

• Deterministic and repeatable tests
Specific interleavings testing

• Based on internal clock
– Tick increases when all threads are blocked
– waitForTick(tick) – blocks the thread until tick reaches
certain value
– assertTick(tick) – asserts the current tick
MultithreadedTC

MultithreadedTC
Thread 2Thread 1
put 42
get 42
put 17
(blocks)
BoundedBuffer with size = 1
get 17

MultithreadedTC
Thread 2Thread 1
put 42
get 42
BoundedBuffer with size = 1
get 17
Tick 0
put 17
(blocks)
Tick 1
waitForTick(1)
assertForTick(1)

• Events based
– IMUnit.fireEvent(“event1”)
– @Schedule(“event1 -> event2”)
– @Schedule(“[event1] -> event2”)
IMUnit

• Two threads only – Main and Secondary
• By default: For each line of Main thread (T1) ->
block and execute fully Secondary thread (T2)
ThreadWeaver
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2
T1: Block A
T1: Block B
T2: Block 1
T2: Block 2

• Two threads only – Main and Secondary
• By default: For each line of Main thread (T1) ->
block and execute fully Secondary thread (T2)
• Powerful Breakpoints
ThreadWeaver

• Tools
– MultithreadedTC -
http://code.google.com/p/multithreadedtc/
• Enhanced version for Junit 4 -
http://code.google.com/p/multithreadedtc-junit4/
– IMUnit - http://mir.cs.illinois.edu/imunit/
– ThreadWeaver -
http://code.google.com/p/thread-weaver/
– Awaitility - http://code.google.com/p/awaitility/
Specific interleavings testing

• Exercise all possible interleavings
All interleavings testing

JavaPathFinder
Traditional testing
OK
Code
Testing
error
Image source: http://javapathfinder.sourceforge.net/events/JPF-workshop-050108/tutorial.ppt

JavaPathFinder
Model Checking with JavaPathFinder
Image source: http://javapathfinder.sourceforge.net/events/JPF-workshop-050108/tutorial.ppt
OK
Code
properties
Model Checking
error trace
Line 5: …
Line 12: …
…
Line 41:…
Line 47:…

Program testing can be used to show the presence
of bugs, but never to show their absence!
--Edsger Dijkstra
JavaPathFinder

• Tools
– JavaPathFinder (JPF) -
http://babelfish.arc.nasa.gov/trac/jpf
• JavaRaceFinder -
http://www.cise.ufl.edu/research/JavaRacefinder/Java_Rac
eFinder/JRF_Home.html
All interleavings testing

• Instrument the code to catch problems easier
Instrumentation

AspectJ
Original code
.class or .jar
AspectJ Compiler
Instrumented
.class or .jar
Aspect Definition
.aj

• Tools
– AspectJ - www.eclipse.org/aspectj/
• RacerAJ - http://www.bodden.de/tools/raceraj/
– CalFuzzer - http://srl.cs.berkeley.edu/~ksen/calfuzzer/
– (commercial) Flashlight / Jsure -
http://www.surelogic.com/concurrency-tools.html
Instrumentation

• CHESS (native DLLs & managed executables) -
http://research.microsoft.com/en-
us/projects/chess/
• MoonWalker (.NET)-
http://code.google.com/p/moonwalker/
Other tools

Recommended books

Thank you!

Q & A

• org.ista2011.multithreaded.common
– Several implementations of AccountManager, Counter, BounderBuffer and
MultipleReadersSingleWriter
• org.ista2011.multithreaded.executorservice
– Load/stress tests using ExecutorService
• org.ista2011.multithreaded.testng
– Load/stress tests using TestNG
• org.ista2011.multithreaded.mtc
– MultithreadedTC tests
• org.ista2011.multithreaded.imunit
– IMUnit tests
• org.ista2011.multithreaded.threadweaver
– ThreadWeaver tests
Source code packages (1)

• org.ista2011.multithreaded.javapathfinder
– JavaPathFinder examples
• ca.mcgill.sable.racer
– RacerAJ source code
• org.ista2011.multithreaded.aspectj
– Custom aspect that increase the chance of hitting multithreading problem
during testing
• org.ista2011.multithreaded.blockbox.*
– Rest based AccountManager client/test and server (in .server package)
Source code packages (2)

Testing Multithreaded Java Applications for Synchronization Problems, ISTA 2011

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Testing Multithreaded Java Applications for Synchronization Problems, ISTA 2011

Similar to Testing Multithreaded Java Applications for Synchronization Problems, ISTA 2011 (20)

Recently uploaded

Recently uploaded (20)

Testing Multithreaded Java Applications for Synchronization Problems, ISTA 2011

Editor's Notes