Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Testing Multithreaded Java           Applications for     Synchronization Problems                 Vassil Popovski        ...
Why multithreading?                      core 1                   core N                                     …            ...
Why multithreading?                      Thread 1                      thread           …           Thread N              ...
Chuck Norris can write multi-threaded Java applications with asingle threadImage source: http://4.bp.blogspot.com/-RhqCAgE...
New concept -> new problemsMultithreading leads to:   • Non-determinism   • New types of defects exist such as deadlocks, ...
Threads interleaving   Thread 1      Thread 2   T1: Block A                 T2: Block 1                 T2: Block 2   T1: ...
Threads interleaving   Thread 1      Thread 2                 T2: Block 1   T1: Block A                 T2: Block 2   T1: ...
T1: Block A       T1: Block A       T1: Block AT1: Block B           T2: Block 1       T2: Block 1    T2: Block 1   T1: Bl...
Number of different thread interleavings
QUIZQ: What is the number of all interleavings for 3threads with 3 blocks each?
QUIZ#2Q: What is the number of all interleavings for 100threads with 5 blocks each?A: 654256643325235644323224249530341682...
What to test for?• safety: nothing bad happens• liveness: something good eventually happens
Example 1: Single Lane Bridge Image source: http://www.doc.ic.ac.uk/~jnm/book/ppt/ch7.ppt• safety:   – no car crash• liven...
Example 2: BoundedBuffer or            Producer/Consumer Producer 1                                                       ...
How to test for synchronization issues?• Load/Stress testing (blackbox, whitebox)• Specific interleavings testing (whitebo...
Load/Stress testing• A lot of threads and operations to exercise  different interleavings
Demo!
Deadlock example      Thread 1                 Thread 2synchronized ( A ) {                         synchronized ( B ) {  ...
Load/Stress testing• Tools  – ExecutorService -    http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Execu  – ...
Specific interleavings testing• Deterministic and repeatable tests
MultithreadedTC• Based on internal clock  – Tick increases when all threads are blocked  – waitForTick(tick) – blocks the ...
MultithreadedTCBoundedBuffer with size = 1                Thread 1      Thread 2                  put 42                  ...
MultithreadedTCBoundedBuffer with size = 1                 Thread 1          Thread 2                                     ...
Demo!
IMUnit• Events based  – IMUnit.fireEvent(“event1”)  – @Schedule(“event1 -> event2”)  – @Schedule(“[event1] -> event2”)
Demo!
ThreadWeaver• Two threads only – Main and Secondary• By default: For each line of Main thread (T1) ->  block and execute f...
ThreadWeaver• Two threads only – Main and Secondary• By default: For each line of Main thread (T1) ->  block and execute f...
Demo!
Specific interleavings testing• Tools  – MultithreadedTC -    http://code.google.com/p/multithreadedtc/     • Enhanced ver...
All interleavings testing• Exercise all possible interleavings
JavaPathFinderTraditional testing                                                                             OK        Co...
JavaPathFinderModel Checking with JavaPathFinder      Code                                                                ...
JavaPathFinderProgram testing can be used to show the presenceof bugs, but never to show their absence!--Edsger Dijkstra
Demo!
All interleavings testing• Tools  – JavaPathFinder (JPF) -    http://babelfish.arc.nasa.gov/trac/jpf     • JavaRaceFinder ...
Instrumentation• Instrument the code to catch problems easier
AspectJ                 AspectJ CompilerOriginal code                        Instrumented ..class or .jar                 ...
Demo!
Instrumentation• Tools  – AspectJ - www.eclipse.org/aspectj/     • RacerAJ - http://www.bodden.de/tools/raceraj/  – CalFuz...
Other tools• CHESS (native DLLs & managed executables) -  http://research.microsoft.com/en-  us/projects/chess/• MoonWalke...
Recommended books
Thank you!
Q&A
Source code packages (1)•   org.java2days2012.multithreaded.common    – Several implementations of AccountManager, Counter...
Source code packages (2)•   org.java2days2012.multithreaded.javapathfinder    – JavaPathFinder examples•   ca.mcgill.sable...
Upcoming SlideShare
Loading in …5
×

Testing multithreaded java applications for synchronization problems

4,315 views

Published on

Published in: Technology
  • Hello there! Get Your Professional Job-Winning Resume Here! http://bit.ly/topresum
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Testing multithreaded java applications for synchronization problems

  1. 1. Testing Multithreaded Java Applications for Synchronization Problems Vassil Popovski vpopovski@vmware.comwww.vmware.com
  2. 2. Why multithreading? core 1 core N … cache cache shared memoryA modern CPU
  3. 3. Why multithreading? Thread 1 thread … Thread N thread synchronization Java heapA modern JavaApplication
  4. 4. Chuck Norris can write multi-threaded Java applications with asingle threadImage source: http://4.bp.blogspot.com/-RhqCAgEyhxs/T7CqtsC0RUI/AAAAAAAAJuw/LnkKl4xBcFA/s1600/ChuckNorris.png
  5. 5. New concept -> new problemsMultithreading leads to: • Non-determinism • New types of defects exist such as deadlocks, livelocks, thread contentions and race conditions
  6. 6. Threads interleaving Thread 1 Thread 2 T1: Block A T2: Block 1 T2: Block 2 T1: Block B
  7. 7. Threads interleaving Thread 1 Thread 2 T2: Block 1 T1: Block A T2: Block 2 T1: Block B
  8. 8. T1: Block A T1: Block A T1: Block AT1: Block B T2: Block 1 T2: Block 1 T2: Block 1 T1: Block B T2: Block 2 T2: Block 2 T2: Block 2 T1: Block B T2: Block 1 T2: Block 1 T2: Block 1 T2: Block 2 T1: Block A T1: Block AT1: Block A T2: Block 2 T1: Block BT1: Block B T1: Block B T2: Block 2
  9. 9. Number of different thread interleavings
  10. 10. QUIZQ: What is the number of all interleavings for 3threads with 3 blocks each?
  11. 11. QUIZ#2Q: What is the number of all interleavings for 100threads with 5 blocks each?A: 654256643325235644323224249530341682689798558504969895155843077522989327608052110684706510794021884753761109216744894417571007359034028210092970532977942118093855090516153565271554457834250091640965333504693915472524842863521707683472687990541535320428681644139277788504865168653330716586970948876921758735376660793466868423614692506980110804828095009115111304919968474372452864438216258436661651149415150907836204374294159422162921916801467371409941078994597491437959423096651940295147131246633777831729233451313619402291921208047034124934515072336907888011881118387091320870766273144025802650979455825709271956695215984277002596448976416503969326271455104144323490546257676541259661664809496397188023136083103000095695249931387824127357795205746244920999444870521575759502862827990114933813566279403884780776803843065828547325552080878639688125368371794927868755931762756337694132525463290778580908338231154468891023062316998265299264207075292536841056562959069339993346497907030777266806096721782581487653665673906956072984789189763083273393296210772311012193942566942014870151342007300083530851043568775842476301945472401025049445878861570784309445821897842612716084833308727230170909590697692873061156171863521770305055016633807107180973183928460790612302673942036701636090627261379044559984247170363121387545156976073503232589545384520717070712826357193754094590158262884843475537865527369260860099762923600463253760690783122018226782819927566148555719195371609778119103882792291769877622640972620027034250733185446711210157633637972796558884333998554201908926433241030984937004054932061950437723982312362664431760768777068958638038910373505305099737990775215550182787315973626327667398087718410375537429058708822137993101329645165183995604817978428125435286138446617286303656453491061778333750467534398482252042350689085025651685787786452108781897103080178867052576325533140254720000000000000000000000000000000000000000000000000
  12. 12. What to test for?• safety: nothing bad happens• liveness: something good eventually happens
  13. 13. Example 1: Single Lane Bridge Image source: http://www.doc.ic.ac.uk/~jnm/book/ppt/ch7.ppt• safety: – no car crash• liveness: – every car eventually get an opportunity to cross the bridge
  14. 14. Example 2: BoundedBuffer or Producer/Consumer Producer 1 Consumer 1 Producer 2 Consumer 2 Producer 3 … … Consumer M Producer N• safety: – head and tail of the queue must not over run each other• liveness: – when the queue contains an item, the consumer process must be able to access the queue and when the queue contains space for another item, the producer process must be able to access the queue
  15. 15. How to test for synchronization issues?• Load/Stress testing (blackbox, whitebox)• Specific interleavings testing (whitebox)• All interleavings testing (whitebox)• Instrumentation (blackbox, whitebox)
  16. 16. Load/Stress testing• A lot of threads and operations to exercise different interleavings
  17. 17. Demo!
  18. 18. Deadlock example Thread 1 Thread 2synchronized ( A ) { synchronized ( B ) { synchronized ( B ) { synchronized ( A ) { Deadlock !!!
  19. 19. Load/Stress testing• Tools – ExecutorService - http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Execu – TestNG - http://testng.org/doc/documentation-main.html#annotations – JUnit - http://www.junit.org/apidocs/junit/extensions/ActiveTestSuite.html • JUnitPerf - http://testng.org/doc/documentation-main.html#annotations – GroboUtils - http://groboutils.sourceforge.net/testing-junit/using_mtt.html – Custom Threading
  20. 20. Specific interleavings testing• Deterministic and repeatable tests
  21. 21. MultithreadedTC• Based on internal clock – Tick increases when all threads are blocked – waitForTick(tick) – blocks the thread until tick reaches certain value – assertTick(tick) – asserts the current tick• Methods that start with “thread” are executed in separate threads
  22. 22. MultithreadedTCBoundedBuffer with size = 1 Thread 1 Thread 2 put 42 put 17 (blocks) get 42 get 17
  23. 23. MultithreadedTCBoundedBuffer with size = 1 Thread 1 Thread 2 Tick 0 put 42 waitForTick(1) put 17 Tick 1 (blocks) get 42 assertForTick(1) get 17
  24. 24. Demo!
  25. 25. IMUnit• Events based – IMUnit.fireEvent(“event1”) – @Schedule(“event1 -> event2”) – @Schedule(“[event1] -> event2”)
  26. 26. Demo!
  27. 27. ThreadWeaver• Two threads only – Main and Secondary• By default: For each line of Main thread (T1) -> block and execute fully Secondary thread (T2) T2: Block 1 T1: Block A T1: Block A T2: Block 2 T2: Block 1 T1: Block B T1: Block A T2: Block 2 T2: Block 1 T1: Block B T1: Block B T2: Block 2
  28. 28. ThreadWeaver• Two threads only – Main and Secondary• By default: For each line of Main thread (T1) -> block and execute fully Secondary thread (T2)• Powerful Breakpoints
  29. 29. Demo!
  30. 30. Specific interleavings testing• Tools – MultithreadedTC - http://code.google.com/p/multithreadedtc/ • Enhanced version for Junit 4 - http://code.google.com/p/multithreadedtc-junit4/ – IMUnit - http://mir.cs.illinois.edu/imunit/ – ThreadWeaver - http://code.google.com/p/thread-weaver/ – Awaitility - http://code.google.com/p/awaitility/
  31. 31. All interleavings testing• Exercise all possible interleavings
  32. 32. JavaPathFinderTraditional testing OK Code Testing error Image source: http://javapathfinder.sourceforge.net/events/JPF-workshop-050108/tutorial.ppt
  33. 33. JavaPathFinderModel Checking with JavaPathFinder Code OK Model Checking error trace properties Line 5: … Line 12: … … Line 41:… Line 47:… Image source: http://javapathfinder.sourceforge.net/events/JPF-workshop-050108/tutorial.ppt
  34. 34. JavaPathFinderProgram testing can be used to show the presenceof bugs, but never to show their absence!--Edsger Dijkstra
  35. 35. Demo!
  36. 36. All interleavings testing• Tools – JavaPathFinder (JPF) - http://babelfish.arc.nasa.gov/trac/jpf • JavaRaceFinder - http://www.cise.ufl.edu/research/JavaRacefinder/Java_Rac eFinder/JRF_Home.html
  37. 37. Instrumentation• Instrument the code to catch problems easier
  38. 38. AspectJ AspectJ CompilerOriginal code Instrumented ..class or .jar class or .jar Aspect Definition .aj
  39. 39. Demo!
  40. 40. Instrumentation• Tools – AspectJ - www.eclipse.org/aspectj/ • RacerAJ - http://www.bodden.de/tools/raceraj/ – CalFuzzer - http://srl.cs.berkeley.edu/~ksen/calfuzzer/ – (commercial) Flashlight / Jsure - http://www.surelogic.com/concurrency-tools.html
  41. 41. Other tools• CHESS (native DLLs & managed executables) - http://research.microsoft.com/en- us/projects/chess/• MoonWalker (.NET)- http://code.google.com/p/moonwalker/
  42. 42. Recommended books
  43. 43. Thank you!
  44. 44. Q&A
  45. 45. Source code packages (1)• org.java2days2012.multithreaded.common – Several implementations of AccountManager, Counter, BounderBuffer and MultipleReadersSingleWriter• org.java2days2012.multithreaded.executorservice – Load/stress tests using ExecutorService• org.java2days2012.multithreaded.testng – Load/stress tests using TestNG• org.java2days2012.multithreaded.mtc – MultithreadedTC tests• org.java2days2012.multithreaded.imunit – IMUnit tests• org.java2days2012.multithreaded.threadweaver – ThreadWeaver tests
  46. 46. Source code packages (2)• org.java2days2012.multithreaded.javapathfinder – JavaPathFinder examples• ca.mcgill.sable.racer – RacerAJ source code• org.java2days2012.multithreaded.aspectj – Custom aspect that increase the chance of hitting multithreading problem during testing• org.java2days2012.multithreaded.blockbox.* – Rest based AccountManager client/test and server (in .server package)

×