Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

T# @ Agile Tour Montreal 2009 En

736 views

Published on

This presentation was shown at Agile Tour Montreal 2009.

  • Be the first to comment

  • Be the first to like this

T# @ Agile Tour Montreal 2009 En

  1. 1. Ludovic Dubois ldubois@prettyOBJECTS.com www.prettyOBJECTS.com ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  2. 2. Pretty Objects – Visual T#  Survey  A good test is…  Conclusion ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  3. 3. • Mission: help OO developers (1993) • Three service offers: – Training(1993) o In partnership with CRIM o Trained over 10,000 participants – Consultation – Coaching (1994) o Object oriented – automated unit tests o Speciality: Microsoft.NET – Development of development tools(1995) o POCMock: VS addin for whitebox testing (2003) o T#: .NET language to simplify writing unit tests (2009) ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  4. 4. • 2005: Idea to simplify writing tests, in particularly when using mocks • 2005: T# documentation • 2006: Beginning of development • 2007: C# compiler done (over 8,000 tests) Beginning of Visual Studio integration Complete rewrite of T# documentation • 2008: T# compiler done (over 12,000 tests), Visual Studio integration still going! • 05/2009: T# v1.0 (almost 18,000 tests) • 09/2009: T# v1.5 (almost 20,000 tests) • 10/2009: T# v1.6 (over 22.000 tests) ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  5. 5. Pretty Objects – Visual T# Survey  A good test is…  Conclusion ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  6. 6. • Who knows about unit testing theory? • Who develops unit tests? • Who uses NUnit? MSTests? Other? • Who has written over: – 100 tests? – 1,000 tests? – 10,000 tests? ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  7. 7. Pretty Objects – Visual T# Survey A good test is…  Conclusion ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  8. 8.  No test = the worst of tests!  Unless it is…  … the test which verifies the contrary of what we want!  Test = a will to test – Is the good will to test sufficient?  Test written after the code – « Does this test actually tests something? »  Test written before the code – Think about the method’s objective prior to creating it ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  9. 9.  Never fix a bug…  Write a test which proves the existence of the bug  Correct the code to make the test pass  Verify that the bug has disappeared from the original context  Never change a test…  Unless the business objective or need has changed  In case of a new bug: duplicate the test and change the copy ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  10. 10.  Generalize whenever possible – Define general contexts for your tests – Separate tests in many classes (because the contexts are different): o Constructors o Class declarations (statics) o Instance declarations (non-statics)  The context creates an instance  Limit code length – Create high-level Assert… methods Writing, maintaining and refactoring becomes much easier ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  11. 11. protected void AssertLinked( Invoice invoice, params object[] products ) { Assert.IsTrue(products.Length % 2 == 0, "Bad test!"); Assert.AreEquals( products.Count / 2, invoice.Count ); for( int i = 0; i < invoice.Count; i += 2 ) { Item item = invoice[ i / 2 ]; Assert.AreSame( products[i], item.Product ); Assert.AreEqual( products[i + 1], item.Quantity ); } } [TestMethod] public void Purchase() { … AssertLinked( invoice, product1, 2, product2, 4 ); } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  12. 12.  Think about the objective prior to coding it! – « What does it test? » o Which declaration? Of which class? – « Under which situation? » o States which influences the execution of the tested declaration?  An instance’s state  A class’ state  Other instances/class’ states  The environment’s state: files, DB… o What are the parameters during the invocation? ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  13. 13.  Non formalized objectives – « What does it test? »: name of a method! – « Under which situation? »: name of a method! Not easily manageable! Rests on the developer’s discipline ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  14. 14. Formalized objectives – « What does it test? »: indicate the tested declaration – « Under which situation? »: make use of criteria Knowledge of what is being tested and by which tests Compiler validation Missing tests notification at compilation Execution of tests from your tested declaration ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  15. 15. [TestClass] public class ProductTest { [TestMethod] public void PriceChange_SomeRandomValue() { … } [TestMethod] public void PriceChange_MinimumValue() { … } } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  16. 16. testclass for Product { test Price set when MinIncluded.IsAboveMin { … } test Price set when MinIncluded.IsMin { … } } // The compiler will warn you of a missing // test for MinIncluded.BelowMinCase ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  17. 17. Determine the test cases for the declaration – Normal cases (such as IsAboveMin, IsMin) – Error cases (such as BelowMinCase)  Détermine the cases for each parameter  Write a test for each possible combination – Combine valid cases – Isolate error cases ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  18. 18.  Test coverage – Technical, not logical! o Doesn’t indicate missing tests, only code that hasn’t been used by the tests o Doesn’t properly cover all cases  if( M1() && M2() ) with M1() always true, M2() variable – Must run all the tests to know which ones are missing – If the coverage isn’t complete: there’s a problem – If the coverage is complete: it doesn’t prove anything! ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  19. 19.  Using tests criteria – Definition of normal cases (IsAboveMin, IsMin) – Definition of error cases (BelowMinCase) – Logical definitions, not technical ones  Write a test per possible combination – Define case expressions for each test o Use the following operators &&, ||, => and () The compiler will warn you about the missing tests ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  20. 20. criteria MinIncludedCriteria { IsAboveMin, IsMin, [Error] BelowMinCase } criteria NotifyPropertyChangedCriteria { HasNoSubscriber, HasSubscribersSetSameValue, HasSubscribersSetOtherValue } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  21. 21. test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasNoSubscriber { … } test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasSubscribersSetOtherValue { … } test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasSubscribersSetSameValue { … } test Price set when MinIncluded.BelowMinCase { … } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  22. 22.  A couple of lines of code – « What does it test? » « Under which situation? » – « For what outcome? »  Separate it in 3 distinctive parts: – Execution: « What does it test? » o Write a single line of code – Preparation: « Under which situation? » o Write the most necessary code in the tests’ context o Validate the situation at the end of each preparation – Verification: « For what outcome? » o Correspond the assertions before and after the execution The effect of the tests code becomes evident ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  23. 23.  Context in two methods – A method before running the test – Another one after running the test Instance variables instead of local ones Impossible to protect the code o No try…catch or finally blocks. Impossible to repeat tests ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  24. 24.  Context defined in only one method – runtest keyword used like a placeholder for the test Local variables instead of instance variables Possibility to protect the code o try…catch or finally around runtest Easy repetition of tests o Use of multiple runtest o Loop over runtest o Define a new situation before each call to runtest ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  25. 25. [TestClass] public class ProductTest { private Product p; [TestInitialize] public void Setup() { p = new Product( "T#", 123 ); } … ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  26. 26. testclass for Product { private Product p; testcontext { p = new Product( "T#", 123 ); runtest; } … ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  27. 27.  3 parts well distinctive? – With comments in the code Discipline of each developer Assertions by methods – Lots of different Assert… – Not always natural o Expected value as first parameter o No automatic casting of types (MSTests) ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  28. 28.  Keyword: runtest – Execution: identified by the runtest keyword – Preparation: before runtest – Verification: after runtest Validation done by the compiler  Keyword: assert – Followed by a boolean expression – Whichever order of expected and actual values – Automatically generated error messages More natural to use ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  29. 29. [TestMethod] public void PriceChange_SomeValue_NoSubscribers() { // Preparation Assert.AreNotEqual( 234.0, p.Price ); // Execution p.Price = 234; // Verification Assert.AreEqual( 234.0, p.Price ); } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  30. 30. test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasNoSubscriber { assert p.Price != 234; runtest p.Price = 234; assert p.Price == 234; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  31. 31.  Verify all wanted effects – Each test has assertions for every wanted effect – Relative vs absolute assertions o Relative: « 1 more than before »  Clearly shows the change of value  More code, more complex, less maintenance o Absolute: « now equals to 1 »  Ensures fixed values  Less code, more simpler, more maintenance  Verify all unwanted effects – Each test has assertions for every unwanted effects ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  32. 32.  Verify all wanted effects – One Assert. … for each wanted effect – More complex scenarios: events o TONS of code Objective of the test is flooded in the code – Sometimes not well tests: exceptions o Makes use of attributes Test passes if any line of code raises an exception! Impossible to add further assertions ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  33. 33.  Verify all wanted effects – One assert keyword used for any verification o assert <expression booléenne> o assert [!] raised <event>  1 single line of code! o assert thrown <exception>  1 single line of code!  Assertion for the « Execution » part only  Other possible assertions o Automatic generated message (more detailed) ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  34. 34. [TestMethod] public void PriceChange_SaveValue_HasSubscriber() { // Preparation p.PropertyChanged += PropChanged; eventRaised = false; // Execution p.Price = p.Price; // Verification Assert.IsFalse( eventRaised ); } private bool eventRaised; private void PropChanged( object sender, EventArgs e ) { eventRaised = true; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  35. 35. test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasSubscriberSetSameValue { runtest p.Price = p.Price; assert !raised p.PropertyChanged; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  36. 36. [TestMethod] public void PriceChange_SomeValue_HasSubscriber() { // Preparation p.PropertyChanged += PropChanged; eventRaised = false; Assert.AreNotEqual( 234.0, p.Price ); // Execution p.Price = 234; // Verification Assert.IsTrue( eventRaised ); Assert.AreEqual( 234.0, p.Price ); } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  37. 37. test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasSubscriberSetOtherValue { assert p.Price != 234; runtest p.Price = 234; assert raised p.PropertyChanged; assert p.Price == 234; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  38. 38. [ExpectedException(typeof(ArgumentOutOfRangeException)] public void PriceChange_ImpossibleValue() { // Preparation // If the preparation code throws an expected // exception, the test passes!!! Assert.AreNotEqual( -1.0, p.Price ); // Execution p.Price = -1; // Verification // Instruction never reached (else: problem!) // Impossible to add further assertions // If reached and throws expected exception, // the test passes!!! } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  39. 39. test Price set when MinIncluded.BelowMinCase { // If the preparation code throws the expected //exception, the test fails assert p.Price != -1; runtest p.Price = -1; assert thrown ArgumentOutOfRangeException; // Add as many assertions as needed // If the verification code throws the expected // exception, the test fails } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  40. 40.  Verify all wanted effects…cont’d… – Relative assertions in 1 single line of code o assert changed <expression>  assert changed collection.Count++  assert changed collection.Count += 1  assert changed collection.Count = collection.Count + 1 ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  41. 41. [TestClass] public class InvoiceTest { private Invoice invoice; private Product product; … [TestMethod] public void Add_New_MultipleProducts_Test() { // Preparation int totalBefore = invoice.Total; Assert.AreEqual( 0, invoice.Count ); // Execution invoice.Add( product, 2 ); // Verification Assert.AreEqual( 1, invoice.Count ); Assert.AreEqual( totalBefore + 2 * product.Price, invoice.Total ); } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  42. 42. testclass for Invoice { private Invoice invoice; private Product product; … test Add( Product p, int qty ) when p: ValidReference.IsNotNull, qty: MinIncluded.IsAboveMin, Collection.IsEmpty && CollectionItem.IsNotInCollection { assert invoice.Count == 0; runtest invoice.Add( product, 2 ); assert changed invoice.Count++; assert changed invoice.Total += 2 * product.Price; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  43. 43.  Verify what you do not want Lots of code…blurry Code in « Preparation » to save Code in « Verification » to compare Objective of the test is flooded in the code ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  44. 44.  Verify what you do not want – Always the same keyword: assert !changed o For only public properties assert !changed obj.*; o For all variables assert !changed obj.-*; – No changes except…: …except… o Some properties (variables) assert !changed obj.* except Property; o An absolute assertion assert !… except Property == 1; o A relative assertion assert !… except changed Property++; ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  45. 45. [TestMethod] public void PriceChange_ImpossibleValue() { // Preparation double actualPrice = p.Price; Assert.AreNotEqual( -1.0, actualPrice ); // Execution try { p.Price = -1; // Verification Assert.Fail("ArgumentOutOfRangeException expected!"); } catch( ArgumentOutOfRangeException ) { Assert.AreEqual( actualPrice, p.Price ); } } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  46. 46. test Price set when MinIncluded.BelowMinCase { assert p.Price != -1; runtest p.Price = -1; assert thrown ArgumentOutOfRangeException; assert !changed p.*; // Verifies the constancy of Price, Number, // Description, etc. } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  47. 47. test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasSubscriberSetOtherValue { assert p.Price != 234; runtest p.Price = 234; assert raised p.PropertyChanged; assert !changed p.* except Price == 234; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  48. 48.  Prefer « black box » tests – Test only available services – Not the internal organization of classes  Not always thinkable – Simple invocation of the service, but complex to handle o A compiler has only one method: Compile(sourceFiles, destinationFiles) – Test hidden parts of higher level Changing code so that it can be testable is…detestable! ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  49. 49. • Test hidden parts  Don’t test them o Doesn’t correspond to our need! o Who is going to use those parts? For what outcome?  Use meta-programming in the test o typeof(…).GetMethod(…).Invoke(…) Blurs the test code Harder to read and understand Error at runtime instead of compile time – Hide the meta inside extension methods in the test module o Less bad solution! ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  50. 50. • Test hidden parts  .- operator o As simple as the . operator o Access any declaration, even private ones o Access any type, even internal ones o Doesn’t change the tested code o More effective than reflection ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  51. 51.  Pass the tests one by one – If you’re writing many tests, ignore all but one – Then make another one pass…  Instantly find tests regarding a declaration  Easily find what’s going on when a test fails ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  52. 52.  Find the tests – 1 class to test = many test classes – 1 declaration to test = many test methods  Find out what’s wrong – 3 states for a test: o Ignored: temporarily on hold o Correct: it passes, all is good  Uh…really? o Failed: the code is not good  Which code? The tested code? The test code? ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  53. 53.  Find the tests – Organize the tests according to the tested code – Execute the tests from the tested code  Find out what’s wrong – 4 states for a test: o Ignored: temporarily on hold o Correct: it passes, all is good o Invalid: the test code is not good (for sure) o Failed: the tested code is not good (probable) ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  54. 54. ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  55. 55. Pretty Objects – Visual T# Survey A good test is… Conclusion ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  56. 56.  Visual T# is free! http://forum.prettyobjects.com/forum2-t-download.aspx http://fr.wikipedia.org/wiki/Visual_T_Sharp  What are you waiting for to write good tests? ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  57. 57. Thanks! :) Pretty Objects Computers inc. 550 Sherbrooke Ouest, suite 100 Montreal, H3A 1B9 TÉL.: 514-840-1253 FAX: 514-840-1244 www.prettyOBJECTS.com info@prettyobjects.com ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009

×