Best Practice Testing with Lime 2

  • 5,032 views
Uploaded on

 

More in: Technology , Business
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • Excellent presentation. It represent some good practices for testing!
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
5,032
On Slideshare
0
From Embeds
0
Number of Embeds
5

Actions

Shares
Downloads
161
Comments
1
Likes
14

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Best Practice Testing with Lime 2 Bernhard Schussek
  • 2. Who am I?
    • Web Developer since 7 years
    • 3. Lead Developer at 2bePUBLISHED
      • Development of symfony applications
      • 4. Search Engine Marketing & Optimization
      • 5. since 2007
    • Developer of Lime 2
    • 6. Recently: Symfony Core Contributor
  • 7. Part I The Interactive Stuff
  • 8. What is important? Brainstorming
  • 9. Things to Keep in Mind
    • Write Tests
  • 10. Things to Keep in Mind
    • Write Tests
    • 11. Test Frequently!
  • 12. Things to Keep in Mind
  • 15. Things to Keep in Mind
    • Write Tests
    • 16. Test Frequently!
    • 17. Performance
    • 18. Reliability
      • Make sure they do actually work...
  • 19. Things to Keep in Mind
    • Write Tests
    • 20. Test Frequently!
    • 21. Performance
    • 22. Reliability
      • Make sure they do actually work...
    • Readability
  • 23. Part II Testing Strategies
  • 24. 4 Phase Test
  • 25. The Test Fixture
  • 26. Test Isolation
  • 27. How to Write Testable Code?
  • 28. Fake Objects
  • 29. Functional vs. Unit Tests
  • 30. Creation and Helper Functions
  • 31. Cukeet
  • 32. Testing Strategies - Cukeet
    • Cukeet
      • Web 3.0 recipe sharing website
    • Business Model
  • 36. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Our Tested Class
  • 37. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Fixture Setup
  • 38. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Test Execution
  • 39. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Result Verification
  • 40. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Fixture Teardown
  • 41. Testing Strategies – 4 Phase Test
    • 4 Phase Test
      • Fixture setup
      • 42. Test execution
      • 43. Result Verification
      • 44. Fixture teardown
    • Goals:
      • „ Leave the room how you entered it“
      • 45. Make it obvious what you are testing
  • 46. Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail // setup copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); // test $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); // assertions $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); // teardown unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Use Comments!
  • 47. The Test Fixture
  • 48. Testing Strategies – The Test Fixture
    • Everything that a test depends on
      • Data in the database, objects, files, …
    • Global Fixture
    • 49. Fresh Fixture
  • 50. Testing Strategies – The Test Fixture Global Fixture Fresh Fixture Fixture Setup test.png, new CukeetThumbnail() Test 1 Test 2 Test 3 Test 4 Fixture Setup test.png, new CukeetThumbnail() Test 1 Test 2 Fixture Setup test.png, new CukeetThumbnail() Fixture Setup
  • 51. Test Isolation
  • 52. Lime 2 Annotations
  • 53. Testing Strategies – Annotations
    • Annotations
      • Control the program flow in the test
    • Supported Annotations:
      • @Test
  • 57. Testing Strategies – Annotations // @Before copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); // @After unlink( 'web/uploads/test.png' ); unset( $thumbnail ); // @Test: resize() resizes the thumbnail $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' );
  • 58. Testing Strategies - Annotations
    • Annotations
      • Improve test isolation
      • 59. Fixtures can be built and destroyed automatically
      • 60. The chance of interacting tests is reduced
  • 61. Testing Strategies – xUnit Style Test Class class CukeetThumbnailTest extends LimeTestCase { private $thumbnail ; public function setup() { copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $this ->thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); } public function teardown() { unlink( 'web/uploads/test.png' ); unset( $this ->thumbnail); } public function testTheThumbnailCanBeResized() {} } Same As With Annotations
  • 62. How to Write Testable Code?
  • 63. Testing Strategies – How to Write Testable Code class CukeetRecipe { public function save() { $user = sfContext::getInstance()->getUser(); if (! $user ->hasCredential( 'admin' )) { throw new RuntimeException( 'No permission!' ); } } }
  • 64. Testing Strategies – How to Write Testable Code class CukeetRecipe { public function save() { $user = sfContext::getInstance()->getUser(); if (! $user ->hasCredential( 'admin' )) { throw new RuntimeException( 'No permission!' ); } } } Uh oh...
  • 65. Expensive Operation
  • 66. Unexpected Side-Effects
  • 67. Lose Control
  • 68. Testing Strategies – How to Write Testable Code? ? Test Tested Class Dependency
  • 69. Dependency Injection
  • 70. Testing Strategies – Dependency Injection
    • Dependency Injection
      • All dependencies of an object should be passed through method arguments
    • Constructor Injection
      • For required dependencies
    • Setter Injection
      • For optional dependencies
    • Parameter Injection
  • 71. Testing Strategies – Constructor Injection class CukeetRecipe { private $user ; public function __construct(sfBasicSecurityUser $user ) { $this ->user = $user ; } public function save() { if (! $this ->user->hasCredential( 'admin' )) { throw new RuntimeException( 'No permission!' ); } }
  • 72. Testing Strategies – Dependency Injection // @Before $user = new sfBasicSecurityUser(); $recipe = new CukeetRecipe( $user ); // @Test: Users with credential 'admin' can save $user ->addCredential( 'admin' ); $recipe ->save( $user ); // @Test: Normal users cannot save $t ->expect( 'RuntimeException' ); $recipe ->save( $user );
  • 73. Testing Strategies – How to Write Testable Code? Test Tested Class Dependency
  • 74. Testing Strategies – How to Write Testable Code? sloooowwww sfContext, Database, ... Test Tested Class Dependency
  • 75. How to Replace Slow Dependencies?
  • 76. Fake Objects
  • 77. Testing Strategies – How to Write Testable Code? Test Tested Class Fake Object
  • 78. Testing Strategies – Slow Dependencies class CukeetRecipe { public function save(CukeetCategoryTable $categoryTable ) { if (is_null( $this ->Category)) { $this ->Category = $categoryTable ->findDefault(); } } }
  • 79. Testing Strategies – Fake Objects // @Test: The default category is assigned, if empty // setup $defaultCategory = new CukeetCategory(); $categoryTable = new FakeCategoryTable( $defaultCategory ); // test $recipe = new CukeetRecipe(); $recipe ->Category = null ; $recipe ->save( $categoryTable ); // assertions $t ->is( $recipe ->Category, $defaultCategory , 'The default...' );
  • 80. Testing Strategies – Fake Objects class FakeCategoryTable extends CukeetCategoryTable { protected $defaultCategory ; public function __construct(CukeetCategory $defaultCategory ) { $this ->defaultCategory = $defaultCategory ; } public function findDefault() { return $this ->defaultCategory; } }
  • 81. Testing Strategies – Stub Objects
    • Stub
      • Provides fake output
      • 82. Acts „as if“ it was the real object
      • 83. Does not have any logic inside
  • 84. Testing Strategies – Stub Objects // @Test: The default category is assigned, if empty // setup $defaultCategory = new CukeetCategory(); $categoryTable = $t ->stub( 'CukeetCategoryTable' ); $categoryTable ->findDefault()->returns( $defaultCategory ); $categoryTable ->replay(); // test $recipe = new CukeetRecipe(); $recipe ->Category = null ; $recipe ->save( $categoryTable ); // assertions $t ->is( $recipe ->Category, $defaultCategory , 'The default...' ); Stub Generation in Lime 2
  • 85. Testing Strategies – Stub Objects // @Test: The default category is assigned, if empty // setup $defaultCategory = new CukeetCategory(); $categoryTable = $t ->stub( 'CukeetCategoryTable' ); $categoryTable ->findDefault()->returns( $defaultCategory ); $categoryTable ->replay(); // test $recipe = new CukeetRecipe(); $recipe ->Category = null ; $recipe ->save( $categoryTable ); // assertions $t ->is( $recipe ->Category, $defaultCategory , 'The default...' ); Stub Configuration
  • 86. Testing Strategies – Stub Objects
    • Attention!
      • Don't replace entities (recipe, category, …)
      • 87. Replace services (table, ...)
  • 88. Testing Strategies – Stub Objects Test state after test execution Test Tested Class Stub
  • 89. Testing Strategies – Mock Objects Test behaviour during test execution Test Tested Class Mock
  • 90. Testing Strategies – Mock Objects
    • Mock
      • Behaviour verification
      • 91. Monitors indirect input
      • 92. Does the tested object call the right methods?
  • 93. Testing Strategies – Mock Objects // @Test: Upon destruction all data is saved to the session // setup $storage = $t ->mock( 'CukeetSessionStorage' ); $storage ->write( 'Foo' , 'Bar' ); $storage ->replay(); $user = new CukeetUser( $storage ); // test $user ->setAttribute( 'Foo' , 'Bar' ); $user ->__destruct(); // assertions $storage ->verify();
  • 94. Testing Strategies – Mock Objects
    • Attention!
      • Verifying behaviour leads to unflexible classes
      • 95. Implementation changes break tests
      • 96. Overuse of mocks can harm your health
  • 97. Testing Strategies – Functional Tests $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end()
  • 98. Functional vs. Unit Tests
  • 99. Testing Strategies – Functional vs. Unit Tests
    • Functional Tests
      • „ Acceptance tests“
      • „ Integration Tests“
      • 100. Test the system as a whole
      • 101. Collaboration between classes/components
      • 102. Are slow!
    • They do not
      • Thoroughly test a system for correctness
      • 103. Test edge cases!
  • 104. Testing Strategies – Functional vs. Unit Tests
    • Unit Tests
      • Test classes/components in isolation
      • 105. Test edge cases
      • 106. Are fast!
    • The biggest part of an application should be tested in unit tests
  • 107. Testing Strategies – Functional vs. Unit Tests $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end()
  • 108. Testing Strategies – Test Edge Cases in Unit Tests // CukeetRecipeTableTest.php require_once dirname(__FILE__). '/../bootstrap/doctrine.php' ; $t ->comment( 'findRecent() returns recent recipes of a category' ); // setup ... // test $actual = $table ->findRecent( 2 , $category ); // assertions ... $t ->is( $actual , $expected , 'The correct recipes were returned' );
  • 109. Testing Strategies – Test Edge Cases in Unit Tests // CukeetRecipeTableTest.php require_once dirname(__FILE__). '/../bootstrap/doctrine.php' ; $t ->comment( 'findRecent() returns recent recipes of a category' ); // setup ... // test $actual = $table ->findRecent( 2 , $category ); // assertions ... $t ->is( $actual , $expected , 'The correct recipes were returned' ); Bootstrap File
  • 110. Testing Strategies – Test Edge Cases in Unit Tests // bootstrap/doctrine.php require_once dirname(__FILE__). '/unit.php' ; $database = new sfDoctrineDatabase( array ( 'name' => 'doctrine' , 'dsn' => 'sqlite::memory:' , )); // load missing model files and create tables Doctrine::createTablesFromModels(ROOT_DIR. '/lib/model' ); // load fixture data Doctrine::loadData( 'data/fixtures/fixtures.yml' ); Load YAML Fixtures
  • 111. Testing Strategies – The Test Fixture Global Fixture Fixture Setup test.png, new CukeetThumbnail() Test 1 Test 2 Test 3 Test 4
  • 112. Testing Strategies – Use Inline Fixtures // @Before reload(); $table = Doctrine::getTable( 'CukeetRecipe' ); // @Test: findRecent() returns recent recipes of a category // setup $category = new CukeetCategory(); $category ->fromArray( array ( 'name' => 'Desserts' , ...)); $recipe1 = new CukeetRecipe(); $recipe1 ->fromArray( array ( 'name' => 'Quatre Quarts' , ...)); $recipe1 ->Category = $category ; $recipe1 ->save(); ...
  • 113. Creation and Helper Functions
  • 114. Testing Strategies – Creation and Helper Functions // @Test: findRecent() returns recent recipes of a category // setup $category = createCategory(); $recipe1 = createRecipeInCategory( $category ); $recipe2 = createRecipeInCategory( $category ); $recipe3 = createRecipe(); $recipe4 = createRecipeInCategory( $category ); save( $recipe1 , $recipe2 , $recipe3 , $recipe4 ); // test $actual = $table ->findRecent( 2 , $category ); // assertions $expected = createCollection( $recipe4 , $recipe2 ); $t ->is( $actual , $expected , 'The correct recipes were returned' );
  • 115. Testing Strategies – Creation and Helper Functions // bootstrap/doctrine.php function createRecipe( array $properties = array ()) { static $i = 0 ; $recipe = new CukeetRecipe(); $recipe ->fromArray(array_merge( array ( 'name' => 'Recipe ' . ++ $i , 'created_at' => date( 'Y-m-d H:m:i' , $i ), ), $properties )); return $recipe ; }
  • 116. Testing Strategies – Functional Test Revisited (1) $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end( )
  • 117. Testing Strategies – Functional Test Revisited (2) $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end()
  • 118. Let's Summarize
  • 119. Testing Strategies - Summary
  • 127. Part III Whatz Nu in Lime 2
  • 128. Annotation Support
  • 129. What Nu in Lime 2 – Annotation Support // @Before copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); // @After unlink( 'web/uploads/test.png' ); unset( $thumbnail ); // @Test: resize() resizes the thumbnail $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' );
  • 130. Stub & Mock Generation
  • 131. Whatz Nu in Lime 2 – Mock Objects // @Test: Upon destruction all data is saved to the session // setup $storage = $t ->mock( 'SessionStorage' ); $storage ->write( 'Foo' , 'Bar' )->returns( true )->once(); $storage ->flush()->atLeastOnce(); $storage ->any( 'read' )->throws( 'BadMethodCallException' ); $storage ->replay(); // test ...
  • 132. Improved Error Messages
  • 133. What Nu in Lime 2 – Improved Error Messages # got: object(CukeetRecipe) ( # ... # '_data' => array ( # ... # 'name' => 'Quatre Quarts', # ... # ), # ) # expected: object(CukeetRecipe) ( # ... # '_data' => array ( # ... # 'name' => 'Kamikaze Burger', # ... # ), # )
  • 134. Multi-Processing Support
  • 135. Whatz Nu in Lime 2 – Multi-Processing Support
    • Test Suite in 1 Process
      • Total 3:20 minutes
      • 136. One CPU core at 100%
      • 137. One CPU core at 20%
    • Test Suite in 8 Processes
      • Total 1:40 minutes
      • 138. Both CPU cores at 100%
    • Up To 100% Faster!
  • 139. Extensibility
  • 140. Whatz Nu in Lime 2 - Extensibility
    • Extensible Testers
      • override is(), like() etc. for specific types
    class LimeTesterException extends LimeTesterObject { // don't compare stack trace protected $properties = array ( 'message' , 'code' , 'file' , 'line' ); } LimeTester::register( 'Exception' , 'LimeTesterException' ); $exception1 = new RuntimeException( 'Foobar' , 10 ); $exception2 = new RuntimeException( 'Foobar' , 10 ); $t ->is( $exception1 , $exception2 , 'Hurray!' )
  • 141. Whatz Nu in Lime 2 - Extensibility
    • Extensible Outputs
  • 145. Integration With CI-Tools sismo php Under Control
  • 146. ?