The document discusses best practices for testing software applications. It covers topics like writing testable code through techniques like dependency injection, using different types of tests like unit tests and functional tests, and strategies for testing like the four phase test approach of fixture setup, test execution, result verification, and fixture teardown. Fake objects and mocks are presented as ways to replace slow dependencies for isolated testing.

1. Best Practice Testing with Lime 2 Bernhard Schussek

2. Who am I? Web Developer since 7 years

3. Lead Developer at 2bePUBLISHED Development of symfony applications

4. Search Engine Marketing & Optimization

5. since 2007 Developer of Lime 2

6. Recently: Symfony Core Contributor

7. Part I The Interactive Stuff

8. What is important? Brainstorming

9. Things to Keep in Mind Write Tests

10. Things to Keep in Mind Write Tests

11. Test Frequently!

12. Things to Keep in Mind Write Tests

13. Test Frequently!

14. Performance

15. Things to Keep in Mind Write Tests

16. Test Frequently!

17. Performance

18. Reliability Make sure they do actually work...

19. Things to Keep in Mind Write Tests

20. Test Frequently!

21. Performance

22. Reliability Make sure they do actually work... Readability

23. Part II Testing Strategies

24. 4 Phase Test

25. The Test Fixture

26. Test Isolation

27. How to Write Testable Code?

28. Fake Objects

29. Functional vs. Unit Tests

30. Creation and Helper Functions

31. Cukeet

32. Testing Strategies - Cukeet Cukeet Web 3.0 recipe sharing website Business Model Recipes

33. Categories

34. Images

35. ...

36. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Our Tested Class

37. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Fixture Setup

38. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Test Execution

39. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Result Verification

40. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Fixture Teardown

41. Testing Strategies – 4 Phase Test 4 Phase Test Fixture setup

42. Test execution

43. Result Verification

44. Fixture teardown Goals: „ Leave the room how you entered it“

45. Make it obvious what you are testing

46. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail // setup copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); // test $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); // assertions $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); // teardown unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Use Comments!

47. The Test Fixture

48. Testing Strategies – The Test Fixture Everything that a test depends on Data in the database, objects, files, … Global Fixture

49. Fresh Fixture

50. Testing Strategies – The Test Fixture Global Fixture Fresh Fixture Fixture Setup test.png, new CukeetThumbnail() Test 1 Test 2 Test 3 Test 4 Fixture Setup test.png, new CukeetThumbnail() Test 1 Test 2 Fixture Setup test.png, new CukeetThumbnail() Fixture Setup

51. Test Isolation

52. Lime 2 Annotations

53. Testing Strategies – Annotations Annotations Control the program flow in the test Supported Annotations: @Test @Before

54. @After

55. @BeforeAll

56. @AfterAll

57. Testing Strategies – Annotations // @Before copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); // @After unlink( 'web/uploads/test.png' ); unset( $thumbnail ); // @Test: resize() resizes the thumbnail $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' );

58. Testing Strategies - Annotations Annotations Improve test isolation

59. Fixtures can be built and destroyed automatically

60. The chance of interacting tests is reduced

61. Testing Strategies – xUnit Style Test Class class CukeetThumbnailTest extends LimeTestCase { private $thumbnail ; public function setup() { copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $this ->thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); } public function teardown() { unlink( 'web/uploads/test.png' ); unset( $this ->thumbnail); } public function testTheThumbnailCanBeResized() {} } Same As With Annotations

62. How to Write Testable Code?

63. Testing Strategies – How to Write Testable Code class CukeetRecipe { public function save() { $user = sfContext::getInstance()->getUser(); if (! $user ->hasCredential( 'admin' )) { throw new RuntimeException( 'No permission!' ); } } }

64. Testing Strategies – How to Write Testable Code class CukeetRecipe { public function save() { $user = sfContext::getInstance()->getUser(); if (! $user ->hasCredential( 'admin' )) { throw new RuntimeException( 'No permission!' ); } } } Uh oh...

65. Expensive Operation

66. Unexpected Side-Effects

67. Lose Control

68. Testing Strategies – How to Write Testable Code? ? Test Tested Class Dependency

69. Dependency Injection

70. Testing Strategies – Dependency Injection Dependency Injection All dependencies of an object should be passed through method arguments Constructor Injection For required dependencies Setter Injection For optional dependencies Parameter Injection

71. Testing Strategies – Constructor Injection class CukeetRecipe { private $user ; public function __construct(sfBasicSecurityUser $user ) { $this ->user = $user ; } public function save() { if (! $this ->user->hasCredential( 'admin' )) { throw new RuntimeException( 'No permission!' ); } }

72. Testing Strategies – Dependency Injection // @Before $user = new sfBasicSecurityUser(); $recipe = new CukeetRecipe( $user ); // @Test: Users with credential 'admin' can save $user ->addCredential( 'admin' ); $recipe ->save( $user ); // @Test: Normal users cannot save $t ->expect( 'RuntimeException' ); $recipe ->save( $user );

73. Testing Strategies – How to Write Testable Code? Test Tested Class Dependency

74. Testing Strategies – How to Write Testable Code? sloooowwww sfContext, Database, ... Test Tested Class Dependency

75. How to Replace Slow Dependencies?

76. Fake Objects

77. Testing Strategies – How to Write Testable Code? Test Tested Class Fake Object

78. Testing Strategies – Slow Dependencies class CukeetRecipe { public function save(CukeetCategoryTable $categoryTable ) { if (is_null( $this ->Category)) { $this ->Category = $categoryTable ->findDefault(); } } }

79. Testing Strategies – Fake Objects // @Test: The default category is assigned, if empty // setup $defaultCategory = new CukeetCategory(); $categoryTable = new FakeCategoryTable( $defaultCategory ); // test $recipe = new CukeetRecipe(); $recipe ->Category = null ; $recipe ->save( $categoryTable ); // assertions $t ->is( $recipe ->Category, $defaultCategory , 'The default...' );

80. Testing Strategies – Fake Objects class FakeCategoryTable extends CukeetCategoryTable { protected $defaultCategory ; public function __construct(CukeetCategory $defaultCategory ) { $this ->defaultCategory = $defaultCategory ; } public function findDefault() { return $this ->defaultCategory; } }

81. Testing Strategies – Stub Objects Stub Provides fake output

82. Acts „as if“ it was the real object

83. Does not have any logic inside

84. Testing Strategies – Stub Objects // @Test: The default category is assigned, if empty // setup $defaultCategory = new CukeetCategory(); $categoryTable = $t ->stub( 'CukeetCategoryTable' ); $categoryTable ->findDefault()->returns( $defaultCategory ); $categoryTable ->replay(); // test $recipe = new CukeetRecipe(); $recipe ->Category = null ; $recipe ->save( $categoryTable ); // assertions $t ->is( $recipe ->Category, $defaultCategory , 'The default...' ); Stub Generation in Lime 2

85. Testing Strategies – Stub Objects // @Test: The default category is assigned, if empty // setup $defaultCategory = new CukeetCategory(); $categoryTable = $t ->stub( 'CukeetCategoryTable' ); $categoryTable ->findDefault()->returns( $defaultCategory ); $categoryTable ->replay(); // test $recipe = new CukeetRecipe(); $recipe ->Category = null ; $recipe ->save( $categoryTable ); // assertions $t ->is( $recipe ->Category, $defaultCategory , 'The default...' ); Stub Configuration

86. Testing Strategies – Stub Objects Attention! Don't replace entities (recipe, category, …)

87. Replace services (table, ...)

88. Testing Strategies – Stub Objects Test state after test execution Test Tested Class Stub

89. Testing Strategies – Mock Objects Test behaviour during test execution Test Tested Class Mock

90. Testing Strategies – Mock Objects Mock Behaviour verification

91. Monitors indirect input

92. Does the tested object call the right methods?

93. Testing Strategies – Mock Objects // @Test: Upon destruction all data is saved to the session // setup $storage = $t ->mock( 'CukeetSessionStorage' ); $storage ->write( 'Foo' , 'Bar' ); $storage ->replay(); $user = new CukeetUser( $storage ); // test $user ->setAttribute( 'Foo' , 'Bar' ); $user ->__destruct(); // assertions $storage ->verify();

94. Testing Strategies – Mock Objects Attention! Verifying behaviour leads to unflexible classes

95. Implementation changes break tests

96. Overuse of mocks can harm your health

97. Testing Strategies – Functional Tests $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end()

98. Functional vs. Unit Tests

99. Testing Strategies – Functional vs. Unit Tests Functional Tests „ Acceptance tests“ „ Integration Tests“

100. Test the system as a whole

101. Collaboration between classes/components

102. Are slow! They do not Thoroughly test a system for correctness

103. Test edge cases!

104. Testing Strategies – Functional vs. Unit Tests Unit Tests Test classes/components in isolation

105. Test edge cases

106. Are fast! The biggest part of an application should be tested in unit tests

107. Testing Strategies – Functional vs. Unit Tests $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end()

108. Testing Strategies – Test Edge Cases in Unit Tests // CukeetRecipeTableTest.php require_once dirname(__FILE__). '/../bootstrap/doctrine.php' ; $t ->comment( 'findRecent() returns recent recipes of a category' ); // setup ... // test $actual = $table ->findRecent( 2 , $category ); // assertions ... $t ->is( $actual , $expected , 'The correct recipes were returned' );

109. Testing Strategies – Test Edge Cases in Unit Tests // CukeetRecipeTableTest.php require_once dirname(__FILE__). '/../bootstrap/doctrine.php' ; $t ->comment( 'findRecent() returns recent recipes of a category' ); // setup ... // test $actual = $table ->findRecent( 2 , $category ); // assertions ... $t ->is( $actual , $expected , 'The correct recipes were returned' ); Bootstrap File

110. Testing Strategies – Test Edge Cases in Unit Tests // bootstrap/doctrine.php require_once dirname(__FILE__). '/unit.php' ; $database = new sfDoctrineDatabase( array ( 'name' => 'doctrine' , 'dsn' => 'sqlite::memory:' , )); // load missing model files and create tables Doctrine::createTablesFromModels(ROOT_DIR. '/lib/model' ); // load fixture data Doctrine::loadData( 'data/fixtures/fixtures.yml' ); Load YAML Fixtures

111. Testing Strategies – The Test Fixture Global Fixture Fixture Setup test.png, new CukeetThumbnail() Test 1 Test 2 Test 3 Test 4

112. Testing Strategies – Use Inline Fixtures // @Before reload(); $table = Doctrine::getTable( 'CukeetRecipe' ); // @Test: findRecent() returns recent recipes of a category // setup $category = new CukeetCategory(); $category ->fromArray( array ( 'name' => 'Desserts' , ...)); $recipe1 = new CukeetRecipe(); $recipe1 ->fromArray( array ( 'name' => 'Quatre Quarts' , ...)); $recipe1 ->Category = $category ; $recipe1 ->save(); ...

113. Creation and Helper Functions

114. Testing Strategies – Creation and Helper Functions // @Test: findRecent() returns recent recipes of a category // setup $category = createCategory(); $recipe1 = createRecipeInCategory( $category ); $recipe2 = createRecipeInCategory( $category ); $recipe3 = createRecipe(); $recipe4 = createRecipeInCategory( $category ); save( $recipe1 , $recipe2 , $recipe3 , $recipe4 ); // test $actual = $table ->findRecent( 2 , $category ); // assertions $expected = createCollection( $recipe4 , $recipe2 ); $t ->is( $actual , $expected , 'The correct recipes were returned' );

115. Testing Strategies – Creation and Helper Functions // bootstrap/doctrine.php function createRecipe( array $properties = array ()) { static $i = 0 ; $recipe = new CukeetRecipe(); $recipe ->fromArray(array_merge( array ( 'name' => 'Recipe ' . ++ $i , 'created_at' => date( 'Y-m-d H:m:i' , $i ), ), $properties )); return $recipe ; }

116. Testing Strategies – Functional Test Revisited (1) $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end( )

117. Testing Strategies – Functional Test Revisited (2) $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end()

118. Let's Summarize

119. Testing Strategies - Summary Su mmary 4 Phase Test

120. Test Isolation

121. Annotations

122. Dependency Injection

123. Stubs

124. Mocks

125. Functional vs. Unit Tests

126. Creation and Helper Functions

127. Part III Whatz Nu in Lime 2

128. Annotation Support

129. What Nu in Lime 2 – Annotation Support // @Before copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); // @After unlink( 'web/uploads/test.png' ); unset( $thumbnail ); // @Test: resize() resizes the thumbnail $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' );

130. Stub & Mock Generation

131. Whatz Nu in Lime 2 – Mock Objects // @Test: Upon destruction all data is saved to the session // setup $storage = $t ->mock( 'SessionStorage' ); $storage ->write( 'Foo' , 'Bar' )->returns( true )->once(); $storage ->flush()->atLeastOnce(); $storage ->any( 'read' )->throws( 'BadMethodCallException' ); $storage ->replay(); // test ...

132. Improved Error Messages

133. What Nu in Lime 2 – Improved Error Messages # got: object(CukeetRecipe) ( # ... # '_data' => array ( # ... # 'name' => 'Quatre Quarts', # ... # ), # ) # expected: object(CukeetRecipe) ( # ... # '_data' => array ( # ... # 'name' => 'Kamikaze Burger', # ... # ), # )

134. Multi-Processing Support

135. Whatz Nu in Lime 2 – Multi-Processing Support Test Suite in 1 Process Total 3:20 minutes

136. One CPU core at 100%

137. One CPU core at 20% Test Suite in 8 Processes Total 1:40 minutes

138. Both CPU cores at 100% Up To 100% Faster!

139. Extensibility

140. Whatz Nu in Lime 2 - Extensibility Extensible Testers override is(), like() etc. for specific types class LimeTesterException extends LimeTesterObject { // don't compare stack trace protected $properties = array ( 'message' , 'code' , 'file' , 'line' ); } LimeTester::register( 'Exception' , 'LimeTesterException' ); $exception1 = new RuntimeException( 'Foobar' , 10 ); $exception2 = new RuntimeException( 'Foobar' , 10 ); $t ->is( $exception1 , $exception2 , 'Hurray!' )

141. Whatz Nu in Lime 2 - Extensibility Extensible Outputs Implement custom outputs for the tests

142. Log-Output

143. Email-Output

144. ...

145. Integration With CI-Tools sismo php Under Control

146. ?