Your SlideShare is downloading. ×
0
Bad Tests, Good TestsTomek Kaczanowski
http://twitter.com/#!/devops_borat
Tomek Kaczanowski• Developer• Team lead• Blogger   • http://kaczanowscy.pl/tomek• Book author   • http://practicalunittest...
Before we begin• Most of the examples are real but:    Obfuscated       − to protect the innocents     Truncated       −...
A little bit of history
Please...                                           no more...http://thedogatemycareplan.wordpress.com
Before we begin• The tests were written in 2004-2006.• No automation, no CI.• Some tests do not compile.• In some tests yo...
We dont need no stinkin asserts!public void testAddChunks() {        System.out.println("*********************************...
Success is not an option...  /**    * Method testFailure.    */  public void testFailure() {       try {           Message...
What has happened? Well, it failed...public void testSimple() {        IData data = null;        IFormat format = null;   ...
What has happened? Well, it failed...public void testSimple() {        IData data = null;        IFormat format = null;   ...
Talk to me//wait for messagesdo {    input = "";    try {        System.out.print(">");        read = System.in.read(buf);...
Tests are boring – let us autogenerate them!/**                                                          protected void te...
Tests are boring – let us autogenerate them!public void testSetGetTimestamp() throws Exception {        // JUnitDoclet beg...
Bartosz, do you really work with such code?!
Conclusions• Automation!   • Running   • Verification• Tests are to be written not generated• You should be informed why y...
Few words about tests
Why bother with tests?• System works as expected• Changes do not hurt• Documentation                             http://tw...
Tests help to achieve quality                     Not sure when I saw this picture –                     probably in GOOS?
What happens if we do it wrong?• Angry clients• Depressed developers                         http://www.joshcanhelp.com
When I started out with unit tests, I wasenthralled with the promise of ease andsecurity that they would bring to myprojec...
http://chrispiascik.com/daily-drawings/express-yourself/
It is a full-time job
No smoke without testsclass SystemAdminSmokeTest extends GroovyTestCase {void testSmoke() {    def ds = new org.h2.jdbcx.J...
No smoke without testsclass SystemAdminSmokeTest extends GroovyTestCase {void testSmoke() {// do not remove below code// d...
Lets follow the leader!                  @Test                  public class ExampleTest {                        public v...
Uh-oh, I feel lonely...                   @Test                   public class ExampleTest {                          publ...
Flickering tests
Asking for troubles...LoggingPropertyConfigurator configurator        = mock(LoggingPropertyConfigurator.class);BaseServle...
Lets mock!
Mockem All!public String getUrl(User user, String timestamp) {         String name=user.getFullName();         String url=...
Mockem All!public String getUrl(User user, String timestamp) {         String name=user.getFullName();         String url=...
Use the front door@Testpublic void shouldAddTimestampToGeneratedUrl() {        //given        util = new ....        TimeP...
Mockem All!@Testpublic void shouldAddTimeZoneToModelAndView() {    //given    UserFacade userFacade = mock(UserFacade.clas...
Use the front door@Testpublic void shouldAddTimeZoneToModelAndView() {    //given    UserFacade userFacade = mock(UserFaca...
Single Responsibility Principle
SRP for testsA test should have one and only one reason to fail.P.S. This is definitely a good advice for unit tests, but ...
Testing two things at once@DataProviderpublic Object[][] data() {         return new Object[][] { {"48", true}, {"+48", tr...
Testing two things at once@DataProviderpublic Object[][] data() {         return new Object[][] { {"48", true}, {"+48", tr...
Testing two things at once@DataProviderpublic Object[][] data() {         return new Object[][] { {"48", true}, {"+48", tr...
Testing two things at once@DataProviderpublic Object[][] data() {       return new Object[][] { {"48", true}, {"+48", true...
Concentrate on one feature@DataProviderpublic Object[][] validQueries() {  return new Object[][] { {"48"}, {"48123"},     ...
Are you satisfied?
Happy pathtestSum() {       assertEquals(Math.sum(2,2), 4);}                            http://mw2.google.com/mw-panoramio...
Happy paths are for wimps2 + 22 + -22 + -50 + 22 + 0Integer.MAX_VALUE + somethingetc.                                http:...
Avoiding happy pathsStart with one:testSum() {       assertEquals(Math.sum(2,2), 4);}Do the simplest thing that works:sum(...
Avoiding happy paths sum(int x, int y) {       return 4; }You moron!Your test is so pathetic,that I can make it passby doi...
Readability is the king
Who the heck is “user_2” ?@DataProviderpublic static Object[][] usersPermissions() {    return new Object[][]{         {"u...
Ah, logged user can read and write...@DataProviderpublic static Object[][] usersPermissions() {    return new Object[][]{ ...
domain1, domain2, domain3, ...
domain1, domain2, domain3, ...
domain1, domain2, domain3, ...
Do not make me learn the API!  server = new MockServer(responseMap, true,              new URL(SERVER_ROOT).getPort(), fal...
Do not make me learn the API!   server = new MockServer(responseMap, true,               new URL(SERVER_ROOT).getPort(), f...
Do not make me learn the API!  server = new MockServer(responseMap, true,              new URL(SERVER_ROOT).getPort(), fal...
What is really important?
What is really important?@DataProviderpublic Object[][] snapshotArtifacts() {       return new Object[][]{            {"a"...
Only version matters@DataProviderpublic Object[][] snapshotVersions() {       return new Object[][]{            {"2.2-SNAP...
Test method names
Naming is really important
Test methods names are important• When test fails• Relation to focused tests
Test methods names are important• testFindTransactionsToAutoCharge()• testSystemSuccess()• testOperation()      @Test     ...
“should” is better than “test”•   testOperation()•   testQuery()•   testConstructor()•   testFindUsersWithFilter()•   shou...
“should” is better than “test”• Starting test method names  with “should” steers you in  the right direction.             ...
Test methods names are important@Testpublic void testQuery(){    when(q.getResultList()).thenReturn(null);    assertNull(d...
Assertions
Assertion part is freaking hugepublic void shouldPreDeployApplication() {          // given          Artifact artifact = m...
Just say itpublic void shouldPreDeployApplication() {      Artifact artifact = mock(Artifact.class);      when(artifact.ge...
Asserting using private methods@Testpublic void testChargeInRetryingState() throws Exception {    // given        TxDTO re...
Matchers vs. private methodsassertState(TxDTO txDTO, AndroidTransaction androidTransaction,         AndroidTransactionStat...
Matchers vs. private methods@Testpublic void testChargeInRetryingState() throws Exception {    // given    TxDTO request =...
What is asserted?@Testpublic void testCompile_32Bit_FakeSourceFile() {        CompilerSupport _32BitCompilerSupport       ...
What is asserted?@Testpublic void testCompile_32Bit_FakeSourceFile() {        CompilerSupport _32BitCompilerSupport       ...
Asserting everythingpublic void invalidTxShouldBeCanceled() {    String fileContent =       FileUtils.getContentOfFile("re...
Asserting everythingpublic void invalidTxShouldBeCanceled() {    String fileContent =       FileUtils.getContentOfFile("re...
Know your tool
Asynchronous Calls@Testpublic void updatesCustomerStatus() throws Exception {    // Publish an asynchronous event:    publ...
Expected exceptions@Test(expectedExceptions = SmsException.class)public void shouldThrowException() throws SmsException { ...
Expected exceptions@Test(expectedExceptions = SmsException.class)public void shouldThrowException() throws SmsException { ...
Expected exceptions (with catch-exception)@Testpublic void shouldThrowException() throws SmsException {        when(gutExt...
Running SUTs code concurrently@Test(threadPoolSize = 3, invocationCount = 10)public void testServer() {    // this method ...
Dependent test methods@Testpublic void shouldConnectToDB() {    // verifying that you can    // estabilish a connection wi...
Know your tools•   Unit testing framework           Additional libraries      Use of temporary file rule                ...
What do you really want to test?
What do you really want to test?  @Test  public void shouldAddAUser() {      User user = new User();      userService.save...
You wanted to see that the number increased @Test public void shouldAddAUser() {         int nb = dao.getNbOfUsers();     ...
Random
Doing it wrongpublic void myTest() {     SomeObject obj = new SomeObject(        a, b, c, productCode());     // testing o...
The dream of stronger, random-powered testspublic void myTest() {     SomeObject obj = new SomeObject(        randomName()...
The dream of stronger, random-powered testspublic void myTest() {     SomeObject obj = new SomeObject(        randomName()...
Conclusions
There is more to it• Integration / end-to-end tests which are not parametrized  (so they all try to set up jetty on port 8...
Test-last? No!• makes people not write tests at all• makes people do only happy-testing• tests reflect the implementation
Always TDD?For six or eight hours spread over the next few weeks Istruggled to get the first test written and running. Wri...
Treat tests as the first class citizens•    do it everyday or forget about it          •   make tests readable using match...
…questions?…rants?…hate speeches?…any other forms of expressing your ego?
Thank you!Thank you for watching theseslides! You can learn more aboutwirting high quality tests byreading my book – „Prac...
Confitura 2012 Bad Tests, Good Tests
Upcoming SlideShare
Loading in...5
×

Confitura 2012 Bad Tests, Good Tests

530

Published on

Slides from my Confitura 2012 presentation. The issues discussed during the talk will be described in my new (free!) ebook - see https://github.com/tomekkaczanowski/bad-tests-good-tests

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
530
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
11
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Confitura 2012 Bad Tests, Good Tests"

  1. 1. Bad Tests, Good TestsTomek Kaczanowski
  2. 2. http://twitter.com/#!/devops_borat
  3. 3. Tomek Kaczanowski• Developer• Team lead• Blogger • http://kaczanowscy.pl/tomek• Book author • http://practicalunittesting.com• Working at CodeWise (Krakow)  ...we are hiring, wanna join us?
  4. 4. Before we begin• Most of the examples are real but:  Obfuscated − to protect the innocents  Truncated − imagine much more complex domain objects• Asking questions is allowed  ...but being smarter than me is not ;)
  5. 5. A little bit of history
  6. 6. Please... no more...http://thedogatemycareplan.wordpress.com
  7. 7. Before we begin• The tests were written in 2004-2006.• No automation, no CI.• Some tests do not compile.• In some tests you can read a comment that "WARNING: test requires the divide param to be set to 20" but the code is so ugly, that there is no way to inject this value.• Some test data are available in form of serialized objects (*.ser) that can not be currently deserialized, because the classes have changed.• The project is now in maintenance. Courtesy of Bartosz http://twitter.com/#!/bocytko
  8. 8. We dont need no stinkin asserts!public void testAddChunks() { System.out.println("*************************************"); System.out.println("testAddChunks() ... "); ChunkMap cm = new ChunkMap(3); cm.addChunk(new Chunk("chunk")); List testList = cm.getChunks("chunk",null); if (testList.isEmpty()) fail("there should be at least one list!"); Chunk chunk = cm.getActualChunk("chunk",null); if (chunk.getElements().isEmpty()) fail("there should be at least one element!"); if (cm.getFinalChunkNr() != 1) fail("there should be at least one chunk!"); // iterate actual chunk for (Iterator it = chunk.getElements().iterator(); it.hasNext();) { Element element = (Element) it.next(); System.out.println("Element: " + element); } showChunks(cm); System.out.println("testAddChunks() OK ");}
  9. 9. Success is not an option... /** * Method testFailure. */ public void testFailure() { try { Message message = new Message(null,true); fail(); } catch(Exception ex) { ExceptionHandler.log(ExceptionLevel.ANY,ex); fail(); } }
  10. 10. What has happened? Well, it failed...public void testSimple() { IData data = null; IFormat format = null; LinkedList<String> attr = new LinkedList<String>(); attr.add("A"); attr.add("B"); try { format = new SimpleFormat("A"); data.setAmount(Amount.TEN); data.setAttributes(attr); IResult result = format.execute(); System.out.println(result.size()); Iterator iter = result.iterator(); while (iter.hasNext()) { IResult r = (IResult) iter.next(); System.out.println(r.getMessage()); ... } catch (Exception e) { fail(); }}
  11. 11. What has happened? Well, it failed...public void testSimple() { IData data = null; IFormat format = null; LinkedList<String> attr = new LinkedList<String>(); attr.add("A"); attr.add("B"); data is still null here. Ready or not, NPE is coming. try { format = new SimpleFormat("A"); data.setAmount(Amount.TEN); data.setAttributes(attr); IResult result = format.execute(); System.out.println(result.size()); Iterator iter = result.iterator(); while (iter.hasNext()) { IResult r = (IResult) iter.next(); System.out.println(r.getMessage()); ... } catch (Exception e) { fail(); }}
  12. 12. Talk to me//wait for messagesdo { input = ""; try { System.out.print(">"); read = System.in.read(buf); //convert characters to string input = new String(buf, 0, read - newline.length()); System.out.println(input); if (input.equals("end") || input.equals("exit") || input.equals("stop") || input.equals("quit")) { System.out.println("Terminating Test please wait..."); System.out.println("******* Test terminated *******"); toStop = true; } else { System.out.println("Commands:" + newline + "end, exit, stop or quit terminates this test "); } } catch (Exception e) { e.printStackTrace(); }} while (!toStop);
  13. 13. Tests are boring – let us autogenerate them!/** protected void tearDown() throws Exception {* Generated by JUnitDoclet, a tool provided by // JUnitDoclet begin method testcase.tearDown* ObjectFab GmbH under LGPL. adapter = null;* Please see www.junitdoclet.org, www.gnu.org super.tearDown();* and www.objectfab.de for informations about // JUnitDoclet end method testcase.tearDown* the tool, the licence and the authors.*/ public void testMain() throws Exception {public class AdapterTest // JUnitDoclet begin method testMain// JUnitDoclet begin extends_implements Adapter.main(new String [] {"ADAPTER"});extends TestCase // JUnitDoclet end method testMain// JUnitDoclet end extends_implements }{ // JUnitDoclet begin class Adapter adapter = null; /** // JUnitDoclet end class * JUnitDoclet moves marker to this method, if there is not match * for them in the regenerated code and if the marker is not empty. public AdapterTest(String name) { * This way, no test gets lost when regenerating after renaming. // JUnitDoclet begin method AdapterTest * Method testVault is supposed to be empty. super(name); */ // JUnitDoclet end method AdapterTest public void testVault() throws Exception { } // JUnitDoclet begin method testcase.testVault // JUnitDoclet end method testcase.testVault public Adapter createInstance() throws Exception { } // JUnitDoclet begin method testcase.createInstance return new Adapter(); public static void main(String[] args) { // JUnitDoclet end method testcase.createInstance // JUnitDoclet begin method testcase.main } junit.textui.TestRunner.run(AdapterTest.class); // JUnitDoclet end method testcase.main protected void setUp() throws Exception { } // JUnitDoclet begin method testcase.setUp } super.setUp(); adapter = createInstance(); // JUnitDoclet end method testcase.setUp }
  14. 14. Tests are boring – let us autogenerate them!public void testSetGetTimestamp() throws Exception { // JUnitDoclet begin method setTimestamp getTimestamp java.util.Calendar[] tests = {new GregorianCalendar(), null}; for (int i = 0; i < tests.length; i++) { adapter.setTimestamp(tests[i]); assertEquals(tests[i], adapter.getTimestamp()); } // JUnitDoclet end method setTimestamp getTimestamp }public void testSetGetParam() throws Exception { // JUnitDoclet begin method setParam getParam String[] tests = {"a", "aaa", "---", "23121313", "", null}; for (int i = 0; i < tests.length; i++) { adapter.setParam(tests[i]); assertEquals(tests[i], adapter.getParam()); } // JUnitDoclet end method setParam getParam}
  15. 15. Bartosz, do you really work with such code?!
  16. 16. Conclusions• Automation! • Running • Verification• Tests are to be written not generated• You should be informed why your test failed• Master your tools  …at least learn the basics!
  17. 17. Few words about tests
  18. 18. Why bother with tests?• System works as expected• Changes do not hurt• Documentation http://twitter.com/#!/devops_borat
  19. 19. Tests help to achieve quality Not sure when I saw this picture – probably in GOOS?
  20. 20. What happens if we do it wrong?• Angry clients• Depressed developers http://www.joshcanhelp.com
  21. 21. When I started out with unit tests, I wasenthralled with the promise of ease andsecurity that they would bring to myprojects. In practice, however, thetheory of sustainable software throughunit tests started to break down. Thisdifficulty continued to build up, until Ifinally threw my head back in anger anddeclared that "Unit Tests have becomemore trouble than they are worth." Llewellyn Falco and Michael Kennedy, Develop Mentor August 09
  22. 22. http://chrispiascik.com/daily-drawings/express-yourself/
  23. 23. It is a full-time job
  24. 24. No smoke without testsclass SystemAdminSmokeTest extends GroovyTestCase {void testSmoke() { def ds = new org.h2.jdbcx.JdbcDataSource( URL: jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;MODE=Oracle, user: sa, password: ) def jpaProperties = new Properties() jpaProperties.setProperty( hibernate.cache.use_second_level_cache, false) jpaProperties.setProperty( hibernate.cache.use_query_cache, false) def emf = new LocalContainerEntityManagerFactoryBean( dataSource: ds, persistenceUnitName: my-domain, jpaVendorAdapter: new HibernateJpaVendorAdapter( database: Database.H2, showSql: true, generateDdl: true), jpaProperties: jpaProperties)}
  25. 25. No smoke without testsclass SystemAdminSmokeTest extends GroovyTestCase {void testSmoke() {// do not remove below code// def ds = new org.h2.jdbcx.JdbcDataSource(// URL: jdbc:h2:mem:test;DB_CLOSE_DELAY=-1;MODE=Oracle,// user: sa, password: )//// def jpaProperties = new Properties()// jpaProperties.setProperty(// hibernate.cache.use_second_level_cache, false)// jpaProperties.setProperty(// hibernate.cache.use_query_cache, false)//// def emf = new LocalContainerEntityManagerFactoryBean(// dataSource: ds, persistenceUnitName: my-domain,// jpaVendorAdapter: new HibernateJpaVendorAdapter(// database: Database.H2, showSql: true,// generateDdl: true), jpaProperties: jpaProperties)…some more code below, all commented out :(}
  26. 26. Lets follow the leader! @Test public class ExampleTest { public void testExample() { assertTrue(true); } }
  27. 27. Uh-oh, I feel lonely... @Test public class ExampleTest { public void testExample() { assertTrue(true); } }
  28. 28. Flickering tests
  29. 29. Asking for troubles...LoggingPropertyConfigurator configurator = mock(LoggingPropertyConfigurator.class);BaseServletContextListener baseServletContextListener = new BaseServletContextListener(configurator); Should load some@Test public void shouldLoadConfigProperties() { default config baseServletContextListener.contextInitialized(); verify(configurator).configure(any(Properties.class));}@Test(expected = LoggingInitialisationException.class) Should load thispublic void shouldThrowLoggingException() { specific file System.setProperty("logConfig", "nonExistingFile"); baseServletContextListener.contextInitialized();}
  30. 30. Lets mock!
  31. 31. Mockem All!public String getUrl(User user, String timestamp) { String name=user.getFullName(); String url=baseUrl +"name="+URLEncoder.encode(name, "UTF-8") +"&timestamp="+timestamp; return url;}public String getUrl(User user) { Date date=new Date(); Long time=(date.getTime()/1000); //convert ms to seconds String timestamp=time.toString(); return getUrl(user, timestamp);}
  32. 32. Mockem All!public String getUrl(User user, String timestamp) { String name=user.getFullName(); String url=baseUrl +"name="+URLEncoder.encode(name, "UTF-8") +"&timestamp="+timestamp; return url;}public String getUrl(User user) { Date date=new Date(); Long time=(date.getTime()/1000); //convert ms to seconds String timestamp=time.toString(); return getUrl(user, timestamp);} @Test public void shouldUseTimestampMethod() { //given Util spyUtil = spy(util); //when spyUtil.getUrl(user); //then verify(spyUtil).getUrl(eq(user), anyString()); }
  33. 33. Use the front door@Testpublic void shouldAddTimestampToGeneratedUrl() { //given util = new .... TimeProvider timeProvider = mock(TimeProvider.class); when(timeProvider.getTime()).thenReturn("12345"); util.set(timeProvider); //when String url = util.getUrl(user); //then assertThat(url).contains("timestamp=12345");}
  34. 34. Mockem All!@Testpublic void shouldAddTimeZoneToModelAndView() { //given UserFacade userFacade = mock(UserFacade.class); ModelAndView modelAndView = mock(ModelAndView.class); given(userFacade.getTimezone()).willReturn("timezone X"); //when new UserDataInterceptor(userFacade) .postHandle(null, null, null, modelAndView); //then verify(modelAndView).addObject("timezone", "timezone X");}
  35. 35. Use the front door@Testpublic void shouldAddTimeZoneToModelAndView() { //given UserFacade userFacade = mock(UserFacade.class); ModelAndView modelAndView = new ModelAndView(); given(userFacade.getTimezone()).willReturn("timezone X"); //when new UserDataInterceptor(userFacade) .postHandle(null, null, null, modelAndView); //then assertThat(modelAndView).constains("timezone", "timezone X");}
  36. 36. Single Responsibility Principle
  37. 37. SRP for testsA test should have one and only one reason to fail.P.S. This is definitely a good advice for unit tests, but rather not validfor integration and end-to-end tests.
  38. 38. Testing two things at once@DataProviderpublic Object[][] data() { return new Object[][] { {"48", true}, {"+48", true}, {"++48", true}, {"+48503", true}, {"+4", false}, {"++4", false}, {"", false}, {null, false}, {" ", false}, };}@Test(dataProvider = "data")public void testQueryVerification(String query, boolean expected) { assertEquals(expected, FieldVerifier.isValidQuery(query));}
  39. 39. Testing two things at once@DataProviderpublic Object[][] data() { return new Object[][] { {"48", true}, {"+48", true}, Data {"++48", true}, {"+48503", true}, {"+4", false}, {"++4", false}, {"", false}, {null, false}, {" ", false}, };}@Test(dataProvider = "data") Algorithm / Logicpublic void testQueryVerification(String query, boolean expected) { assertEquals(expected, FieldVerifier.isValidQuery(query));}
  40. 40. Testing two things at once@DataProviderpublic Object[][] data() { return new Object[][] { {"48", true}, {"+48", true}, {"++48", true}, {"+48503", true}, {"+4", false}, {"++4", false}, {"", false}, {null, false}, {" ", false}, };}@Test(dataProvider = "data")public void testQueryVerification(String query, boolean expected) { assertEquals(expected, FieldVerifier.isValidQuery(query));} testQueryVerification1() { assertEquals(true, FieldVerifier.isValidQuery(„48”)); } testQueryVerification2() { assertEquals(true, FieldVerifier.isValidQuery(„+48”)); } testQueryVerification3() { assertEquals(true, FieldVerifier.isValidQuery(„++48”)); } testQueryVerification4() { assertEquals(true, FieldVerifier.isValidQuery(„+48503”)); } ...
  41. 41. Testing two things at once@DataProviderpublic Object[][] data() { return new Object[][] { {"48", true}, {"+48", true}, {"++48", true}, {"+48503", true}, {"+4", false}, {"++4", false}, {"", false}, {null, false}, {" ", false}, };}@Test(dataProvider = "data")public void testQueryVerification(String query, boolean expected) { assertEquals(expected, FieldVerifier.isValidQuery(query));}
  42. 42. Concentrate on one feature@DataProviderpublic Object[][] validQueries() { return new Object[][] { {"48"}, {"48123"}, {"+48"}, {"++48"}, {"+48503"}};}@Test(dataProvider = "validQueries")public void shouldRecognizeValidQueries( String validQuery) { assertTrue(FieldVerifier.isValidQuery(validQuery));}@DataProviderpublic Object[][] invalidQueries() { return new Object[][] { {"+4"}, {"++4"}, {""}, {null}, {" "} };}@Test(dataProvider = "invalidQueries")public void shouldRejectInvalidQueries( String invalidQuery) { assertFalse(FieldVerifier.isValidQuery(invalidQuery));}
  43. 43. Are you satisfied?
  44. 44. Happy pathtestSum() { assertEquals(Math.sum(2,2), 4);} http://mw2.google.com/mw-panoramio/photos/medium/68775332.jpg
  45. 45. Happy paths are for wimps2 + 22 + -22 + -50 + 22 + 0Integer.MAX_VALUE + somethingetc. http://kidskidskids.tumblr.com/post/1145294997
  46. 46. Avoiding happy pathsStart with one:testSum() { assertEquals(Math.sum(2,2), 4);}Do the simplest thing that works:sum(int x, int y) { return 4;}And then listen to your code.Because it tells you something. http://kidskidskids.tumblr.com/post/1145294997
  47. 47. Avoiding happy paths sum(int x, int y) { return 4; }You moron!Your test is so pathetic,that I can make it passby doing such a silly thing.Try harder! http://looneytunes09.files.wordpress.com/2010/07/lisa-yell.gif
  48. 48. Readability is the king
  49. 49. Who the heck is “user_2” ?@DataProviderpublic static Object[][] usersPermissions() { return new Object[][]{ {"user_1", Permission.READ}, {"user_1", Permission.WRITE}, {"user_1", Permission.REMOVE}, {"user_2", Permission.WRITE}, {"user_2", Permission.READ}, {"user_3", Permission.READ} };}
  50. 50. Ah, logged user can read and write...@DataProviderpublic static Object[][] usersPermissions() { return new Object[][]{ {ADMIN, Permission.READ}, {ADMIN, Permission.WRITE}, {ADMIN, Permission.REMOVE}, {LOGGED, Permission.WRITE}, {LOGGED, Permission.READ}, {GUEST, Permission.READ} };}
  51. 51. domain1, domain2, domain3, ...
  52. 52. domain1, domain2, domain3, ...
  53. 53. domain1, domain2, domain3, ...
  54. 54. Do not make me learn the API! server = new MockServer(responseMap, true, new URL(SERVER_ROOT).getPort(), false);
  55. 55. Do not make me learn the API! server = new MockServer(responseMap, true, new URL(SERVER_ROOT).getPort(), false);private static final boolean RESPONSE_IS_A_FILE = true;private static final boolean NO_SSL = false;server = new MockServer(responseMap, RESPONSE_IS_A_FILE, new URL(SERVER_ROOT).getPort(), NO_SSL);
  56. 56. Do not make me learn the API! server = new MockServer(responseMap, true, new URL(SERVER_ROOT).getPort(), false); server = new MockServerBuilder() .withResponse(responseMap) .withResponseType(FILE) .withUrl(SERVER_ROOT) .withoutSsl().create();
  57. 57. What is really important?
  58. 58. What is really important?@DataProviderpublic Object[][] snapshotArtifacts() { return new Object[][]{ {"a", "b", "2.2-SNAPSHOT", Artifact.JAR }, {"c", "d", "2.2.4.6-SNAPSHOT", Artifact.JAR}, {"e", "f", "2-SNAPSHOT", Artifact.JAR} };}@Test(dataProvider = "snapshotArtifacts")public void shouldRecognizeSnapshots( String groupId, String artifactId, String version, Type type) { Artifact artifact = new Artifact(groupId, artifactId, version, type); assertThat(artifact.isSnapshot()).isTrue();}
  59. 59. Only version matters@DataProviderpublic Object[][] snapshotVersions() { return new Object[][]{ {"2.2-SNAPSHOT"}, {"2.2.4.6-SNAPSHOT"}, {"2-SNAPSHOT"} };}@Test(dataProvider = "snapshotVersions")public void shouldRecognizeSnapshots(String version) { Artifact artifact = new Artifact(VALID_GROUP, VALID_ARTIFACT_ID, version, VALID_TYPE); assertThat(artifact.isSnapshot()).isTrue();}
  60. 60. Test method names
  61. 61. Naming is really important
  62. 62. Test methods names are important• When test fails• Relation to focused tests
  63. 63. Test methods names are important• testFindTransactionsToAutoCharge()• testSystemSuccess()• testOperation() @Test public void testOperation() { configureRequest("/validate") rc = new RequestContext(parser, request) assert rc.getConnector() == null assert rc.getOperation().equals("validate") }
  64. 64. “should” is better than “test”• testOperation()• testQuery()• testConstructor()• testFindUsersWithFilter()• shouldRejectInvalidRequests()• shouldSaveNewUserToDatabase()• constructorShouldFailWithNegativePrice()• shouldReturnOnlyUsersWithGivenName()
  65. 65. “should” is better than “test”• Starting test method names with “should” steers you in the right direction. http://jochopra.blogspot.com/• “test” prefix makes your test method a limitless bag where you throw everything worth testing http://www.greenerideal.com/
  66. 66. Test methods names are important@Testpublic void testQuery(){ when(q.getResultList()).thenReturn(null); assertNull(dao.findByQuery(Transaction.class, q, false)); assertNull(dao.findByQuery(Operator.class, q, false)); assertNull(dao.findByQuery(null, null, false)); List result = new LinkedList(); when(q.getResultList()).thenReturn(result); assertEquals(dao.findByQuery(Transaction.class, q, false), result); assertEquals(dao.findByQuery(Operator.class, q, false), result); assertEquals(dao.findByQuery(null, null, false), null); when(q.getSingleResult()).thenReturn(null); assertEquals(dao.findByQuery(Transaction.class, q, true).size(), 0); assertEquals(dao.findByQuery(Operator.class, q, true).size(), 0); assertEquals(dao.findByQuery(null, null, true), null); when(q.getSingleResult()).thenReturn(t); assertSame(dao.findByQuery(Transaction.class, q, true).get(0), t); when(q.getSingleResult()).thenReturn(o); assertSame(dao.findByQuery(Operator.class, q, true).get(0), o); when(q.getSingleResult()).thenReturn(null); assertSame(dao.findByQuery(null, null, true), null);}
  67. 67. Assertions
  68. 68. Assertion part is freaking hugepublic void shouldPreDeployApplication() { // given Artifact artifact = mock(Artifact.class); when(artifact.getFileName()).thenReturn("war-artifact-2.0.war"); ServerConfiguration config = new ServerConfiguration(ADDRESS, USER, KEY_FILE, TOMCAT_PATH, TEMP_PATH); Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config); String destDir = new File(".").getCanonicalPath() + SLASH + "target" + SLASH; new File(destDir).mkdirs(); // when tomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH)); //then JSch jsch = new JSch(); jsch.addIdentity(KEY_FILE); Session session = jsch.getSession(USER, ADDRESS, 22); session.setConfig("StrictHostKeyChecking", "no"); session.connect(); Channel channel = session.openChannel("sftp"); session.setServerAliveInterval(92000); channel.connect(); ChannelSftp sftpChannel = (ChannelSftp) channel; sftpChannel.get(TEMP_PATH + SLASH + artifact.getFileName(), destDir); sftpChannel.exit(); session.disconnect(); File downloadedFile = new File(destDir, artifact.getFileName()); assertThat(downloadedFile).exists().hasSize(WAR_FILE_LENGTH);}
  69. 69. Just say itpublic void shouldPreDeployApplication() { Artifact artifact = mock(Artifact.class); when(artifact.getFileName()) .thenReturn(ARTIFACT_FILE_NAME); ServerConfiguration config = new ServerConfiguration(ADDRESS, USER, KEY_FILE, TOMCAT_PATH, TEMP_PATH); Tomcat tomcat = new Tomcat(HTTP_TOMCAT_URL, config); tomcat.preDeploy(artifact, new FakeWar(WAR_FILE_LENGTH)); SSHServerAssert.assertThat(ARTIFACT_FILE_NAME) .existsOnServer(config).hasSize(WAR_FILE_LENGTH);}
  70. 70. Asserting using private methods@Testpublic void testChargeInRetryingState() throws Exception { // given TxDTO request = createTxDTO(RequestType.CHARGE); AndroidTransaction androidTransaction = ... // when final TxDTO txDTO = processor.processRequest(request); // then assertState(request, androidTransaction, CHARGED, CHARGE_PENDING, AS_ANDROID_TX_STATE, ClientMessage.SUCCESS, ResultCode.SUCCESS);}
  71. 71. Matchers vs. private methodsassertState(TxDTO txDTO, AndroidTransaction androidTransaction, AndroidTransactionState expectedAndroidState, AndroidTransactionState expectedPreviousAndroidState, ExtendedState expectedState, String expectedClientStatus, ResultCode expectedRequestResultCode) { final List<AndroidTransactionStep> steps = new ArrayList<>(androidTransaction.getTransactionSteps()); final boolean checkPreviousStep = expectedAndroidState != null; assertTrue(steps.size() >= (checkPreviousStep ? 3 : 2)); if (checkPreviousStep) { AndroidTransactionStep lastStep = steps.get(steps.size() - 2); assertEquals(lastStep.getTransactionState(), expectedPreviousAndroidState); } final AndroidTransactionStep lastStep = steps.get(steps.size() - 1); assertEquals(lastStep.getTransactionState(), expectedAndroidState); assertEquals(lastStep.getMessage(), expectedClientStatus); assertEquals(txDTO.getResultCode(), expectedRequestResultCode); assertEquals(androidTransaction.getState(), expectedAndroidState); assertEquals(androidTransaction.getExtendedState(), expectedState); if (expectedClientStatus == null) { verifyZeroInteractions(client); }}
  72. 72. Matchers vs. private methods@Testpublic void testChargeInRetryingState() throws Exception { // given TxDTO request = createTxDTO(CHARGE); AndroidTransaction androidTransaction = ... // when final TxDTO txDTO = processor.processRequest(request); // then assertThat(androidTransaction).hasState(CHARGED) .hasMessage(ClientMessage.SUCCESS) .hasPreviousState(CHARGE_PENDING) .hasExtendedState(null); assertEquals(txDTO.getResultCode(), ResultCode.SUCCESS);}
  73. 73. What is asserted?@Testpublic void testCompile_32Bit_FakeSourceFile() { CompilerSupport _32BitCompilerSupport = CompilerSupportFactory.getDefault32BitCompilerSupport(); testCompile_FakeSourceFile(_32BitCompilerSupport);}
  74. 74. What is asserted?@Testpublic void testCompile_32Bit_FakeSourceFile() { CompilerSupport _32BitCompilerSupport = CompilerSupportFactory.getDefault32BitCompilerSupport(); testCompile_FakeSourceFile(_32BitCompilerSupport);}private void testCompile_FakeSourceFile( CompilerSupport compilerSupport) { compiledFiles = compilerSupport.compile(new File[] { new File("fake") }); assertThat(compiledFiles, is(emptyArray()));}
  75. 75. Asserting everythingpublic void invalidTxShouldBeCanceled() { String fileContent = FileUtils.getContentOfFile("response.csv"); assertTrue(fileContent.contains( "CANCEL,123,123cancel,billing_id_123_cancel,SUCCESS,"));}
  76. 76. Asserting everythingpublic void invalidTxShouldBeCanceled() { String fileContent = FileUtils.getContentOfFile("response.csv"); assertTrue(fileContent.contains( "CANCEL,123,123cancel,billing_id_123_cancel,SUCCESS,"));}public void invalidTxShouldBeCanceled() { String fileContent = FileUtils.getContentOfFile("response.csv"); TxDTOAssert.assertThat(fileContent) .hasTransaction("123cancel").withResultCode(SUCCESS);}
  77. 77. Know your tool
  78. 78. Asynchronous Calls@Testpublic void updatesCustomerStatus() throws Exception { // Publish an asynchronous event: publishEvent(updateCustomerStatusEvent); // Awaitility lets you wait // until the asynchronous operation completes: await() .atMost(5, SECONDS) .until(costumerStatusIsUpdated()); ...} http://code.google.com/p/awaitility/
  79. 79. Expected exceptions@Test(expectedExceptions = SmsException.class)public void shouldThrowException() throws SmsException { try { String s = gutExtractor.extractGut(„invalid gut”); System.out.println(s); } catch (SmsException e) { e.printStackTrace(); throw e; }}
  80. 80. Expected exceptions@Test(expectedExceptions = SmsException.class)public void shouldThrowException() throws SmsException { String s = gutExtractor.extractGut(„invalid gut”);}
  81. 81. Expected exceptions (with catch-exception)@Testpublic void shouldThrowException() throws SmsException { when(gutExtractor.extractGut(„invalid gut”)); then(caughtException()) .isInstanceOf(SmsException.class) .hasMessage("Invalid gut") .hasNoCause();} http://code.google.com/p/catch-exception/
  82. 82. Running SUTs code concurrently@Test(threadPoolSize = 3, invocationCount = 10)public void testServer() { // this method will be run in parallel by 3 thread // 10 invocations (in total)}
  83. 83. Dependent test methods@Testpublic void shouldConnectToDB() { // verifying that you can // estabilish a connection with DB}@Test(dependsOnMethods = „shouldConnectToDB”)public void should…() { // some operations on DB}
  84. 84. Know your tools• Unit testing framework  Additional libraries  Use of temporary file rule  Hamcrest, FEST, Mockito,  Listeners catch-exception, awaitility, …  Concurrency • Build tool  @Before/@After  Parallel execution  Parametrized tests  CI  Test dependencies • IDE  Templates  Shortcuts
  85. 85. What do you really want to test?
  86. 86. What do you really want to test? @Test public void shouldAddAUser() { User user = new User(); userService.save(user); assertEquals(dao.getNbOfUsers(), 1); }
  87. 87. You wanted to see that the number increased @Test public void shouldAddAUser() { int nb = dao.getNbOfUsers(); User user = new User(); userService.save(user); assertEquals(dao.getNbOfUsers(), nb + 1); }
  88. 88. Random
  89. 89. Doing it wrongpublic void myTest() { SomeObject obj = new SomeObject( a, b, c, productCode()); // testing of obj here}private String productCode(){ String[] codes = {"Code A", "Code B", "Code C", "Code D"}; int index = rand.nextInt(codes.length); return codes[index];}
  90. 90. The dream of stronger, random-powered testspublic void myTest() { SomeObject obj = new SomeObject( randomName(), randomValue(), ....); // testing of obj here}Does it make your test stronger?
  91. 91. The dream of stronger, random-powered testspublic void myTest() { SomeObject obj = new SomeObject( randomName(), randomValue(), ....); // testing of obj here}Does it make your test stronger?...or does it only bring confusion?Test failedExpected SomeObject(„a”, „b”, ....)but got SomeObject(„*&O*$NdlF”, „#idSLNF”, ....)
  92. 92. Conclusions
  93. 93. There is more to it• Integration / end-to-end tests which are not parametrized (so they all try to set up jetty on port 8080),• Tests which should be really unit, but use Spring context to create objects,• Tests with a lot of dependencies between them (a nightmare to maintain!),• Tests which run slow• Tests which try to cover the deficiencies of production code and end up being a total mess,• Tests which verify methods instead of verifying responsibilities of a class• etc., etc.
  94. 94. Test-last? No!• makes people not write tests at all• makes people do only happy-testing• tests reflect the implementation
  95. 95. Always TDD?For six or eight hours spread over the next few weeks Istruggled to get the first test written and running. Writingtests for Eclipse plug-ins is not trivial, so it’s notsurprising I had some trouble. [...] In six or eight hoursof solid programming time, I can still make significantprogress. If I’d just written some stuff and verified it byhand, I would probably have the final answer to whethermy idea is actually worth money by now. Instead, all Ihave is a complicated test that doesn’t work, a pileof frustration, eight fewer hours in my life, and themotivation to write another essay. Kent Beck, Just Ship, Baby
  96. 96. Treat tests as the first class citizens• do it everyday or forget about it • make tests readable using matchers,• use the right tool for the job builders and good names • and learn to use it! • test behaviour not methods• do not live with broken windows • be pragmatic about the tests you write• respect KISS, SRP, DRY (?) • TDD always?• write good code, and you will also write • what is the best way to test it? good tests unit/integration/end-to-end ? • or rather write good tests and you • automate! will get good code for free • always concentrate on what is worth• code review your tests testing• do more than happy path testing • ask yourself questions like: is it really important that X should send• do not make the reader learn the API, message Y to Z? make it obvious • use the front door – state testing before• bad names lead to bad tests interaction testing (mockc)
  97. 97. …questions?…rants?…hate speeches?…any other forms of expressing your ego?
  98. 98. Thank you!Thank you for watching theseslides! You can learn more aboutwirting high quality tests byreading my book – „Practical UnitTesting with TestNG andMockito”.You can also participate inwriting of mynew (free!) e-book devoted tobad and good tests.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×