Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Unit testing patterns for concurrent code 
Dror Helper 
drorh@oz-code.com | @dhelper | http://blog.drorhelper.com 
Example...
About.ME 
• Senior consultant @CodeValue 
• Developing software (professionally) since 2002 
• Mocking code since 2008 
• ...
We live in a concurrent world! 
The free lunch is over! 
• Multi-core CPUs are the new standard 
• New(er) language constr...
Meanwhile in the unit testing “world” 
[Test] 
public void AddTest() 
{ 
var cut = new Calculator(); 
var result = cut.Add...
The dark art of concurrent code 
Several actions at the same time 
Hard to follow code path 
Non deterministic execution
Good unit tests must be: 
• Trustworthy 
– Consistent results 
– Only fail due to bug or requirement change 
• Maintainabl...
Concurrency test “smells” 
× Inconsistent results 
× Untraceable fail 
× Long running tests 
× Test freeze
How can we test this method 
public void Start() 
{ 
_worker = new Thread(() => { 
while (_isAlive) { 
Thread.Sleep(1000);...
Testing start take #1 
[TestMethod] 
public void ArrivingMessagePublishedTest() 
{ 
var fakeMessageProvider = A.Fake<IMess...
Test smell - “Sleep” in test 
× Time based - fail/pass inconsistently 
× Test runs for too long 
× Hard to investigate fai...
“In concurrent programming if 
something can happen, then sooner 
or later it will, probably at the most 
inconvenient mom...
Testing start take #2 
[TestMethod] 
public async Task ArrivingMessagePublishedTest() 
{ 
var fakeMessageProvider = A.Fake...
Solution: avoid concurrent code!
Pattern #1 – humble object pattern 
We extract all the logic from the hard-to-test component into a component 
that is tes...
public void Start() 
{ 
_worker = new Thread(() => { 
while (_isAlive) { 
Thread.Sleep(1000); 
var msg = _messageProvider....
public void Start() 
{ 
_worker = new Thread(() => { 
while (_isAlive) { 
Thread.Sleep(1000); 
_messageHandler.HandleNextM...
And finally – the test 
[TestMethod] 
public void ArrivingMessagePublishedTest() 
{ 
var fakeMessageProvider = A.Fake<IMes...
Concurrency as part of program flow 
public class MessageManager 
{ 
private IMesseageQueue _messeageQueue; 
public void C...
Test before – Test After 
Start 
Code under test 
Async 
Logic2 
Assert results 
Logic1 
Start Logic1 Fake Assert Results ...
Testing Flow part 1 
[TestMethod] 
public void AddNewMessageProcessedMessageInQueue() 
{ 
var messeageQueue = new AsyncMes...
Testing Flow part 2 
[TestMethod] 
public void QueueRaisedNewMessageEventClientProcessEvent() 
{ 
var messeageQueue = new ...
Avoid concurrency Patterns 
The best possible solution 
No concurrency == no problems 
× Do not test some of the code 
×...
How can we test this class? 
public class ClassWithTimer 
{ 
private Timer _timer; 
public ClassWithTimer(Timer timer) 
{ ...
Not a good idea 
[TestMethod] 
public void ThisIsABadTest() 
{ 
var timer = new Timer(1); 
var cut = new ClassWithTimer(ti...
Set timeout/interval to 1 
Also seen with a very small number (or zero) 
Usually done when need to wait for next tick/time...
Fake & Sync 
Test 
Code under Test 
Fake 
Logic 
Assert Results
Using Typemock Isolator to fake Timer 
[TestMethod, Isolated] 
public void ThisIsAGoodTest() 
{ 
var fakeTimer = Isolate.F...
Unfortunately not everything can be faked 
• Mocking tool limitation (example: inheritance based) 
• Programming language ...
Wrapping Threadpool 
public class ThreadPoolWrapper 
{ 
public static void QueueUserWorkItem(WaitCallback callback) 
{ 
Th...
Wrapping Threadpool 
public class ClassWithWrappedThreadpool 
{ 
public void RunInThread() 
{ 
ThreadPool.QueueUserWorkIte...
Testing with new ThreadpoolWrapper 
[TestClass] 
public class WorkingWithThreadpool 
{ 
[TestMethod, Isolated] 
public voi...
How can we test that an 
asynchronous operation 
never happens?
Test Code Under Test 
Is Test 
Async 
Deterministic Assert Results 
Run in sync
Another day – another class to test 
public void Start() 
{ 
_cancellationTokenSource = new CancellationTokenSource(); 
Ta...
Switching code to “test mode” 
• Dependency injection 
• Preprocessor directives 
• Pass delegate 
• Other
Run in single thread patterns 
• Fake & Sync 
• Async code  sync test
Synchronized run patterns 
When you have to run a concurrent test in a predictable way
The Signaled pattern 
Start 
Run 
Code under test 
Signal 
Fake object 
Call 
Wait Assert result
Using the Signaled pattern 
public void DiffcultCalcAsync(int a, int b) 
{ 
Task.Run(() => 
{ 
Result = a + b; 
_otherClas...
Using the Signaled pattern 
[TestMethod] 
public void TestUsingSignal() 
{ 
var waitHandle = new ManualResetEventSlim(fals...
Busy Assert 
Start 
Run 
Code under 
test 
Assert or 
Timeout
Busy assertion 
[TestMethod] 
public void DifficultCalculationTest() 
{ 
var cut = new ClassWithAsyncOperation(); 
cut.Run...
Synchronized test patterns 
× Harder to investigate failures 
× Cannot test that a call was not made 
Test runs for too l...
Unit testing for concurrency issues 
Because concurrency (and async) needs to be tested as well
Test for async 
Start Method under test 
Fake object 
wait 
Assert results
Test for Deadlock 
Start 
Execute 
Code under 
test 
Thread 
WWaaitit Assert result 
Call Thread
[TestMethod, Timeout(5000)] 
public void CheckForDeadlock() 
{ 
var fakeDependency1 = A.Fake<IDependency>(); 
var fakeDepe...
Avoid 
Concurrency 
Humble 
object 
Test before – 
test after 
Run in single 
thread 
Fake & Sync 
Async in 
production - ...
Conclusion 
It is possible to test concurrent code 
Avoid concurrency 
Run in single thread 
Synchronize test
Dror Helper 
C: 972.50.7668543 
e: drorh@codevalue.net 
B: blog.drorhelper.com 
w: www.codevalue.net
Upcoming SlideShare
Loading in …5
×

Unit testing patterns for concurrent code

2,086 views

Published on

Getting started with unit testing is not hard, the only problem is that most programs are more than a simple calculator with two parameters and a returns value that is easy to verify.

Writing unit tests for multi-threaded code is harder still.

Over the years I discovered useful patterns that helped me to test multi-threaded and asynchronous code and enabled the creation of deterministic, simple and robust unit tests.

Come learn how to test code that uses concurrency and parallelism – so that the excuses of not writing unit tests for such code would become as obsolete as a single core processors.

Published in: Technology
  • Be the first to comment

Unit testing patterns for concurrent code

  1. 1. Unit testing patterns for concurrent code Dror Helper drorh@oz-code.com | @dhelper | http://blog.drorhelper.com Examples: https://github.com/dhelper/ConcurrentUnitTesting
  2. 2. About.ME • Senior consultant @CodeValue • Developing software (professionally) since 2002 • Mocking code since 2008 • Test Driven Developer • Blogger: http://blog.drorhelper.com
  3. 3. We live in a concurrent world! The free lunch is over! • Multi-core CPUs are the new standard • New(er) language constructs • New(ish) languages
  4. 4. Meanwhile in the unit testing “world” [Test] public void AddTest() { var cut = new Calculator(); var result = cut.Add(2, 3); Assert.AreEqual(5, result); }
  5. 5. The dark art of concurrent code Several actions at the same time Hard to follow code path Non deterministic execution
  6. 6. Good unit tests must be: • Trustworthy – Consistent results – Only fail due to bug or requirement change • Maintainable – Robust – Easy to refactor – Simple to update • Readable
  7. 7. Concurrency test “smells” × Inconsistent results × Untraceable fail × Long running tests × Test freeze
  8. 8. How can we test this method public void Start() { _worker = new Thread(() => { while (_isAlive) { Thread.Sleep(1000); var msg = _messageProvider.GetNextMessage(); //Do stuff LastMessage = msg; } }); _worker.Start(); }
  9. 9. Testing start take #1 [TestMethod] public void ArrivingMessagePublishedTest() { var fakeMessageProvider = A.Fake<IMessageProvider>(); A.CallTo(() => fakeMessageProvider.GetNextMessage()).Returns("Hello!"); var server = new Server(fakeMessageProvider); server.Start(); Thread.Sleep(2000); Assert.AreEqual("Hello!", server.LastMessage); }
  10. 10. Test smell - “Sleep” in test × Time based - fail/pass inconsistently × Test runs for too long × Hard to investigate failures
  11. 11. “In concurrent programming if something can happen, then sooner or later it will, probably at the most inconvenient moment” Paul Butcher – Seven concurrency models in seven weeks
  12. 12. Testing start take #2 [TestMethod] public async Task ArrivingMessagePublishedTest() { var fakeMessageProvider = A.Fake<IMessageProvider>(); A.CallTo(() => fakeMessageProvider.GetNextMessage()).Returns("Hello!"); var server = new Server(fakeMessageProvider); server.Start(); await Task.Delay(2000); Assert.AreEqual("Hello!", server.LastMessage); }
  13. 13. Solution: avoid concurrent code!
  14. 14. Pattern #1 – humble object pattern We extract all the logic from the hard-to-test component into a component that is testable via synchronous tests. http://xunitpatterns.com/Humble%20Object.html Perform Action Code under Test Start Humble object Async Perform action Assert Result Production Code
  15. 15. public void Start() { _worker = new Thread(() => { while (_isAlive) { Thread.Sleep(1000); var msg = _messageProvider.GetNextMessage(); //Do stuff LastMessage = msg; } }); _worker.Start(); }
  16. 16. public void Start() { _worker = new Thread(() => { while (_isAlive) { Thread.Sleep(1000); _messageHandler.HandleNextMessage(); } }); _worker.Start(); }
  17. 17. And finally – the test [TestMethod] public void ArrivingMessagePublishedTest() { var fakeMessageProvider = A.Fake<IMessageProvider>(); A.CallTo(() => fakeMessageProvider.GetNextMessage()).Returns("Hello!"); var messageHandler = new MessageHandler(fakeMessageProvider); messageHandler.HandleNextMessage(); Assert.AreEqual("Hello!", messageHandler.LastMessage); }
  18. 18. Concurrency as part of program flow public class MessageManager { private IMesseageQueue _messeageQueue; public void CreateMessage(string message) { // Here Be Code! _messeageQueue.Enqueue(message); } } public class MessageClient { private IMesseageQueue _messeageQueue; public string LastMessage { get; set; } private void OnMessage(object sender, EventArgs e) { // Here Be Code! LastMessage = e.Message; } }
  19. 19. Test before – Test After Start Code under test Async Logic2 Assert results Logic1 Start Logic1 Fake Assert Results Start Fake Logic2 Assert Results
  20. 20. Testing Flow part 1 [TestMethod] public void AddNewMessageProcessedMessageInQueue() { var messeageQueue = new AsyncMesseageQueue(); var manager = new MessageManager(messeageQueue); manager.CreateNewMessage("a new message"); Assert.AreEqual(1, messeageQueue.Count); }
  21. 21. Testing Flow part 2 [TestMethod] public void QueueRaisedNewMessageEventClientProcessEvent() { var messeageQueue = new AsyncMesseageQueue(); var client = new MessageClient(messeageQueue); client.OnMessage(null, new MessageEventArgs("A new message")); Assert.AreEqual("A new message", client.LastMessage); }
  22. 22. Avoid concurrency Patterns The best possible solution No concurrency == no problems × Do not test some of the code × Not applicable in every scenario
  23. 23. How can we test this class? public class ClassWithTimer { private Timer _timer; public ClassWithTimer(Timer timer) { _timer = timer; _timer.Elapsed += OnTimerElapsed; _timer.Start(); } private void OnTimerElapsed(object sender, ElapsedEventArgs e) { SomethingImportantHappened = true; } public bool SomethingImportantHappened { get; private set; } }
  24. 24. Not a good idea [TestMethod] public void ThisIsABadTest() { var timer = new Timer(1); var cut = new ClassWithTimer(timer); Thread.Sleep(100); Assert.IsTrue(cut.SomethingImportantHappened); }
  25. 25. Set timeout/interval to 1 Also seen with a very small number (or zero) Usually done when need to wait for next tick/timeout × Time based == fragile/inconsistent test × Hard to investigate failures × Usually comes with Thread.Sleep
  26. 26. Fake & Sync Test Code under Test Fake Logic Assert Results
  27. 27. Using Typemock Isolator to fake Timer [TestMethod, Isolated] public void ThisIsAGoodTest() { var fakeTimer = Isolate.Fake.Instance<Timer>(); var cut = new ClassWithTimer(fakeTimer); var fakeEventArgs = Isolate.Fake.Instance<ElapsedEventArgs>(); Isolate.Invoke.Event( () => fakeTimer.Elapsed += null, this, fakeEventArgs); Assert.IsTrue(cut.SomethingImportantHappened); }
  28. 28. Unfortunately not everything can be faked • Mocking tool limitation (example: inheritance based) • Programming language attributes • Special cases (example: MSCorlib) Solution – wrap the unfakeable × Problem – it requires code change
  29. 29. Wrapping Threadpool public class ThreadPoolWrapper { public static void QueueUserWorkItem(WaitCallback callback) { ThreadPool.QueueUserWorkItem(callback); } }
  30. 30. Wrapping Threadpool public class ClassWithWrappedThreadpool { public void RunInThread() { ThreadPool.QueueUserWorkItem(_ => { SomethingImportantHappened = true; }); } public bool SomethingImportantHappened { get; private set; } } ThreadPoolWrapper.QueueUserWorkItem(_ =>
  31. 31. Testing with new ThreadpoolWrapper [TestClass] public class WorkingWithThreadpool { [TestMethod, Isolated] public void UsingWrapperTest() { Isolate.WhenCalled(() => ThreadPoolWrapper.QueueUserWorkItem(null)) .DoInstead(ctx => ((WaitCallback)ctx.Parameters[0]).Invoke(null)); var cut = new ClassWithWrappedThreadpool(); cut.RunInThread(); Assert.IsTrue(cut.SomethingImportantHappened); } }
  32. 32. How can we test that an asynchronous operation never happens?
  33. 33. Test Code Under Test Is Test Async Deterministic Assert Results Run in sync
  34. 34. Another day – another class to test public void Start() { _cancellationTokenSource = new CancellationTokenSource(); Task.Run(() => { var message = _messageBus.GetNextMessage(); if(message == null) return; // Do work if (OnNewMessage != null) { OnNewMessage(this, EventArgs.Empty); } }, _cancellationTokenSource.Token); }
  35. 35. Switching code to “test mode” • Dependency injection • Preprocessor directives • Pass delegate • Other
  36. 36. Run in single thread patterns • Fake & Sync • Async code  sync test
  37. 37. Synchronized run patterns When you have to run a concurrent test in a predictable way
  38. 38. The Signaled pattern Start Run Code under test Signal Fake object Call Wait Assert result
  39. 39. Using the Signaled pattern public void DiffcultCalcAsync(int a, int b) { Task.Run(() => { Result = a + b; _otherClass.DoSomething(Result); }); }
  40. 40. Using the Signaled pattern [TestMethod] public void TestUsingSignal() { var waitHandle = new ManualResetEventSlim(false); var fakeOtherClass = A.Fake<IOtherClass>(); A.CallTo(() => fakeOtherClass.DoSomething(A<int>._)).Invokes(waitHandle.Set); var cut = new ClassWithAsyncOperation(fakeOtherClass); cut.DiffcultCalcAsync(2, 3); var wasCalled = waitHandle.Wait(10000); Assert.IsTrue(wasCalled, "OtherClass.DoSomething was never called"); Assert.AreEqual(5, cut.Result); }
  41. 41. Busy Assert Start Run Code under test Assert or Timeout
  42. 42. Busy assertion [TestMethod] public void DifficultCalculationTest() { var cut = new ClassWithAsyncOperation(); cut.RunAsync(2, 3); AssertHelper.BusyAssert(() => cut.Result == 5, 50, 100, "fail!"); }
  43. 43. Synchronized test patterns × Harder to investigate failures × Cannot test that a call was not made Test runs for too long but only when it fails  Use if other patterns are not applicable
  44. 44. Unit testing for concurrency issues Because concurrency (and async) needs to be tested as well
  45. 45. Test for async Start Method under test Fake object wait Assert results
  46. 46. Test for Deadlock Start Execute Code under test Thread WWaaitit Assert result Call Thread
  47. 47. [TestMethod, Timeout(5000)] public void CheckForDeadlock() { var fakeDependency1 = A.Fake<IDependency>(); var fakeDependency2 = A.Fake<IDependency>(); var waitHandle = new ManualResetEvent(false); var cut = new ClassWithDeadlock(fakeDependency1, fakeDependency2); A.CallTo(() => fakeDependency1.Call()).Invokes(() => waitHandle.WaitOne()); A.CallTo(() => fakeDependency2.Call()).Invokes(() => waitHandle.WaitOne()); var t1 = RunInOtherThread(() => cut.Call1Then2()); var t2 = RunInOtherThread(() => cut.Call2Then1()); waitHandle.Set(); t1.Join(); t2.Join(); } T1 T2 L1 L2
  48. 48. Avoid Concurrency Humble object Test before – test after Run in single thread Fake & Sync Async in production - sync in test Synchronize test The Signaled pattern Busy assertion Concurrency tests Test for Deadlock Test for non blocking Concurrent unit testing patterns
  49. 49. Conclusion It is possible to test concurrent code Avoid concurrency Run in single thread Synchronize test
  50. 50. Dror Helper C: 972.50.7668543 e: drorh@codevalue.net B: blog.drorhelper.com w: www.codevalue.net

×