T# @ Agile Tour Montreal 2009 En
Upcoming SlideShare
Loading in...5
×
 

T# @ Agile Tour Montreal 2009 En

on

  • 781 views

This presentation was shown at Agile Tour Montreal 2009.

This presentation was shown at Agile Tour Montreal 2009.

Statistics

Views

Total Views
781
Views on SlideShare
780
Embed Views
1

Actions

Likes
0
Downloads
5
Comments
0

1 Embed 1

http://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

T# @ Agile Tour Montreal 2009 En T# @ Agile Tour Montreal 2009 En Presentation Transcript

  • Ludovic Dubois ldubois@prettyOBJECTS.com www.prettyOBJECTS.com ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • Pretty Objects – Visual T#  Survey  A good test is…  Conclusion ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • • Mission: help OO developers (1993) • Three service offers: – Training(1993) o In partnership with CRIM o Trained over 10,000 participants – Consultation – Coaching (1994) o Object oriented – automated unit tests o Speciality: Microsoft.NET – Development of development tools(1995) o POCMock: VS addin for whitebox testing (2003) o T#: .NET language to simplify writing unit tests (2009) ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • • 2005: Idea to simplify writing tests, in particularly when using mocks • 2005: T# documentation • 2006: Beginning of development • 2007: C# compiler done (over 8,000 tests) Beginning of Visual Studio integration Complete rewrite of T# documentation • 2008: T# compiler done (over 12,000 tests), Visual Studio integration still going! • 05/2009: T# v1.0 (almost 18,000 tests) • 09/2009: T# v1.5 (almost 20,000 tests) • 10/2009: T# v1.6 (over 22.000 tests) ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • Pretty Objects – Visual T# Survey  A good test is…  Conclusion ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • • Who knows about unit testing theory? • Who develops unit tests? • Who uses NUnit? MSTests? Other? • Who has written over: – 100 tests? – 1,000 tests? – 10,000 tests? ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • Pretty Objects – Visual T# Survey A good test is…  Conclusion ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  No test = the worst of tests!  Unless it is…  … the test which verifies the contrary of what we want!  Test = a will to test – Is the good will to test sufficient?  Test written after the code – « Does this test actually tests something? »  Test written before the code – Think about the method’s objective prior to creating it ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Never fix a bug…  Write a test which proves the existence of the bug  Correct the code to make the test pass  Verify that the bug has disappeared from the original context  Never change a test…  Unless the business objective or need has changed  In case of a new bug: duplicate the test and change the copy ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Generalize whenever possible – Define general contexts for your tests – Separate tests in many classes (because the contexts are different): o Constructors o Class declarations (statics) o Instance declarations (non-statics)  The context creates an instance  Limit code length – Create high-level Assert… methods Writing, maintaining and refactoring becomes much easier ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • protected void AssertLinked( Invoice invoice, params object[] products ) { Assert.IsTrue(products.Length % 2 == 0, "Bad test!"); Assert.AreEquals( products.Count / 2, invoice.Count ); for( int i = 0; i < invoice.Count; i += 2 ) { Item item = invoice[ i / 2 ]; Assert.AreSame( products[i], item.Product ); Assert.AreEqual( products[i + 1], item.Quantity ); } } [TestMethod] public void Purchase() { … AssertLinked( invoice, product1, 2, product2, 4 ); } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Think about the objective prior to coding it! – « What does it test? » o Which declaration? Of which class? – « Under which situation? » o States which influences the execution of the tested declaration?  An instance’s state  A class’ state  Other instances/class’ states  The environment’s state: files, DB… o What are the parameters during the invocation? ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Non formalized objectives – « What does it test? »: name of a method! – « Under which situation? »: name of a method! Not easily manageable! Rests on the developer’s discipline ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • Formalized objectives – « What does it test? »: indicate the tested declaration – « Under which situation? »: make use of criteria Knowledge of what is being tested and by which tests Compiler validation Missing tests notification at compilation Execution of tests from your tested declaration ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • [TestClass] public class ProductTest { [TestMethod] public void PriceChange_SomeRandomValue() { … } [TestMethod] public void PriceChange_MinimumValue() { … } } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • testclass for Product { test Price set when MinIncluded.IsAboveMin { … } test Price set when MinIncluded.IsMin { … } } // The compiler will warn you of a missing // test for MinIncluded.BelowMinCase ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • Determine the test cases for the declaration – Normal cases (such as IsAboveMin, IsMin) – Error cases (such as BelowMinCase)  Détermine the cases for each parameter  Write a test for each possible combination – Combine valid cases – Isolate error cases ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Test coverage – Technical, not logical! o Doesn’t indicate missing tests, only code that hasn’t been used by the tests o Doesn’t properly cover all cases  if( M1() && M2() ) with M1() always true, M2() variable – Must run all the tests to know which ones are missing – If the coverage isn’t complete: there’s a problem – If the coverage is complete: it doesn’t prove anything! ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Using tests criteria – Definition of normal cases (IsAboveMin, IsMin) – Definition of error cases (BelowMinCase) – Logical definitions, not technical ones  Write a test per possible combination – Define case expressions for each test o Use the following operators &&, ||, => and () The compiler will warn you about the missing tests ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • criteria MinIncludedCriteria { IsAboveMin, IsMin, [Error] BelowMinCase } criteria NotifyPropertyChangedCriteria { HasNoSubscriber, HasSubscribersSetSameValue, HasSubscribersSetOtherValue } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasNoSubscriber { … } test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasSubscribersSetOtherValue { … } test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasSubscribersSetSameValue { … } test Price set when MinIncluded.BelowMinCase { … } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  A couple of lines of code – « What does it test? » « Under which situation? » – « For what outcome? »  Separate it in 3 distinctive parts: – Execution: « What does it test? » o Write a single line of code – Preparation: « Under which situation? » o Write the most necessary code in the tests’ context o Validate the situation at the end of each preparation – Verification: « For what outcome? » o Correspond the assertions before and after the execution The effect of the tests code becomes evident ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Context in two methods – A method before running the test – Another one after running the test Instance variables instead of local ones Impossible to protect the code o No try…catch or finally blocks. Impossible to repeat tests ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Context defined in only one method – runtest keyword used like a placeholder for the test Local variables instead of instance variables Possibility to protect the code o try…catch or finally around runtest Easy repetition of tests o Use of multiple runtest o Loop over runtest o Define a new situation before each call to runtest ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • [TestClass] public class ProductTest { private Product p; [TestInitialize] public void Setup() { p = new Product( "T#", 123 ); } … ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • testclass for Product { private Product p; testcontext { p = new Product( "T#", 123 ); runtest; } … ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  3 parts well distinctive? – With comments in the code Discipline of each developer Assertions by methods – Lots of different Assert… – Not always natural o Expected value as first parameter o No automatic casting of types (MSTests) ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Keyword: runtest – Execution: identified by the runtest keyword – Preparation: before runtest – Verification: after runtest Validation done by the compiler  Keyword: assert – Followed by a boolean expression – Whichever order of expected and actual values – Automatically generated error messages More natural to use ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • [TestMethod] public void PriceChange_SomeValue_NoSubscribers() { // Preparation Assert.AreNotEqual( 234.0, p.Price ); // Execution p.Price = 234; // Verification Assert.AreEqual( 234.0, p.Price ); } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasNoSubscriber { assert p.Price != 234; runtest p.Price = 234; assert p.Price == 234; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Verify all wanted effects – Each test has assertions for every wanted effect – Relative vs absolute assertions o Relative: « 1 more than before »  Clearly shows the change of value  More code, more complex, less maintenance o Absolute: « now equals to 1 »  Ensures fixed values  Less code, more simpler, more maintenance  Verify all unwanted effects – Each test has assertions for every unwanted effects ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Verify all wanted effects – One Assert. … for each wanted effect – More complex scenarios: events o TONS of code Objective of the test is flooded in the code – Sometimes not well tests: exceptions o Makes use of attributes Test passes if any line of code raises an exception! Impossible to add further assertions ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Verify all wanted effects – One assert keyword used for any verification o assert <expression booléenne> o assert [!] raised <event>  1 single line of code! o assert thrown <exception>  1 single line of code!  Assertion for the « Execution » part only  Other possible assertions o Automatic generated message (more detailed) ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • [TestMethod] public void PriceChange_SaveValue_HasSubscriber() { // Preparation p.PropertyChanged += PropChanged; eventRaised = false; // Execution p.Price = p.Price; // Verification Assert.IsFalse( eventRaised ); } private bool eventRaised; private void PropChanged( object sender, EventArgs e ) { eventRaised = true; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasSubscriberSetSameValue { runtest p.Price = p.Price; assert !raised p.PropertyChanged; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • [TestMethod] public void PriceChange_SomeValue_HasSubscriber() { // Preparation p.PropertyChanged += PropChanged; eventRaised = false; Assert.AreNotEqual( 234.0, p.Price ); // Execution p.Price = 234; // Verification Assert.IsTrue( eventRaised ); Assert.AreEqual( 234.0, p.Price ); } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasSubscriberSetOtherValue { assert p.Price != 234; runtest p.Price = 234; assert raised p.PropertyChanged; assert p.Price == 234; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • [ExpectedException(typeof(ArgumentOutOfRangeException)] public void PriceChange_ImpossibleValue() { // Preparation // If the preparation code throws an expected // exception, the test passes!!! Assert.AreNotEqual( -1.0, p.Price ); // Execution p.Price = -1; // Verification // Instruction never reached (else: problem!) // Impossible to add further assertions // If reached and throws expected exception, // the test passes!!! } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • test Price set when MinIncluded.BelowMinCase { // If the preparation code throws the expected //exception, the test fails assert p.Price != -1; runtest p.Price = -1; assert thrown ArgumentOutOfRangeException; // Add as many assertions as needed // If the verification code throws the expected // exception, the test fails } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Verify all wanted effects…cont’d… – Relative assertions in 1 single line of code o assert changed <expression>  assert changed collection.Count++  assert changed collection.Count += 1  assert changed collection.Count = collection.Count + 1 ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • [TestClass] public class InvoiceTest { private Invoice invoice; private Product product; … [TestMethod] public void Add_New_MultipleProducts_Test() { // Preparation int totalBefore = invoice.Total; Assert.AreEqual( 0, invoice.Count ); // Execution invoice.Add( product, 2 ); // Verification Assert.AreEqual( 1, invoice.Count ); Assert.AreEqual( totalBefore + 2 * product.Price, invoice.Total ); } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • testclass for Invoice { private Invoice invoice; private Product product; … test Add( Product p, int qty ) when p: ValidReference.IsNotNull, qty: MinIncluded.IsAboveMin, Collection.IsEmpty && CollectionItem.IsNotInCollection { assert invoice.Count == 0; runtest invoice.Add( product, 2 ); assert changed invoice.Count++; assert changed invoice.Total += 2 * product.Price; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Verify what you do not want Lots of code…blurry Code in « Preparation » to save Code in « Verification » to compare Objective of the test is flooded in the code ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Verify what you do not want – Always the same keyword: assert !changed o For only public properties assert !changed obj.*; o For all variables assert !changed obj.-*; – No changes except…: …except… o Some properties (variables) assert !changed obj.* except Property; o An absolute assertion assert !… except Property == 1; o A relative assertion assert !… except changed Property++; ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • [TestMethod] public void PriceChange_ImpossibleValue() { // Preparation double actualPrice = p.Price; Assert.AreNotEqual( -1.0, actualPrice ); // Execution try { p.Price = -1; // Verification Assert.Fail("ArgumentOutOfRangeException expected!"); } catch( ArgumentOutOfRangeException ) { Assert.AreEqual( actualPrice, p.Price ); } } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • test Price set when MinIncluded.BelowMinCase { assert p.Price != -1; runtest p.Price = -1; assert thrown ArgumentOutOfRangeException; assert !changed p.*; // Verifies the constancy of Price, Number, // Description, etc. } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • test Price set when MinIncluded.IsAboveMin && NotifyPropertyChanged.HasSubscriberSetOtherValue { assert p.Price != 234; runtest p.Price = 234; assert raised p.PropertyChanged; assert !changed p.* except Price == 234; } ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Prefer « black box » tests – Test only available services – Not the internal organization of classes  Not always thinkable – Simple invocation of the service, but complex to handle o A compiler has only one method: Compile(sourceFiles, destinationFiles) – Test hidden parts of higher level Changing code so that it can be testable is…detestable! ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • • Test hidden parts  Don’t test them o Doesn’t correspond to our need! o Who is going to use those parts? For what outcome?  Use meta-programming in the test o typeof(…).GetMethod(…).Invoke(…) Blurs the test code Harder to read and understand Error at runtime instead of compile time – Hide the meta inside extension methods in the test module o Less bad solution! ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • • Test hidden parts  .- operator o As simple as the . operator o Access any declaration, even private ones o Access any type, even internal ones o Doesn’t change the tested code o More effective than reflection ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Pass the tests one by one – If you’re writing many tests, ignore all but one – Then make another one pass…  Instantly find tests regarding a declaration  Easily find what’s going on when a test fails ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Find the tests – 1 class to test = many test classes – 1 declaration to test = many test methods  Find out what’s wrong – 3 states for a test: o Ignored: temporarily on hold o Correct: it passes, all is good  Uh…really? o Failed: the code is not good  Which code? The tested code? The test code? ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Find the tests – Organize the tests according to the tested code – Execute the tests from the tested code  Find out what’s wrong – 4 states for a test: o Ignored: temporarily on hold o Correct: it passes, all is good o Invalid: the test code is not good (for sure) o Failed: the tested code is not good (probable) ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • Pretty Objects – Visual T# Survey A good test is… Conclusion ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  •  Visual T# is free! http://forum.prettyobjects.com/forum2-t-download.aspx http://fr.wikipedia.org/wiki/Visual_T_Sharp  What are you waiting for to write good tests? ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009
  • Thanks! :) Pretty Objects Computers inc. 550 Sherbrooke Ouest, suite 100 Montreal, H3A 1B9 TÉL.: 514-840-1253 FAX: 514-840-1244 www.prettyOBJECTS.com info@prettyobjects.com ©2009 Pretty Objects Computers inc. All Rights Reserved. Agile Tour Montréal 2009