Best Practice Testing with Lime 2

Best Practice Testing with Lime 2 Bernhard Schussek

Who am I?
Web Developer since 7 years
Lead Developer at 2bePUBLISHED
Development of symfony applications
Search Engine Marketing & Optimization
since 2007
Developer of Lime 2
Recently: Symfony Core Contributor

Part I The Interactive Stuff

What is important? Brainstorming

Things to Keep in Mind
Write Tests

Write Tests
Test Frequently!

Write Tests
Test Frequently!
Performance

Write Tests
Test Frequently!
Performance
Reliability
Make sure they do actually work...

Write Tests
Test Frequently!
Performance
Reliability
Make sure they do actually work...
Readability

Part II Testing Strategies

4 Phase Test

The Test Fixture

Test Isolation

How to Write Testable Code?

Fake Objects

Functional vs. Unit Tests

Creation and Helper Functions

Cukeet

Testing Strategies - Cukeet
Cukeet
Web 3.0 recipe sharing website
Business Model
Recipes
Categories
Images
...

Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Our Tested Class

Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Fixture Setup

Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Test Execution

Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Result Verification

Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Fixture Teardown

Testing Strategies – 4 Phase Test
4 Phase Test
Fixture setup
Test execution
Result Verification
Fixture teardown
Goals:
„ Leave the room how you entered it“
Make it obvious what you are testing

Testing Strategies – 4 Phase Test // @Test: resize() resizes the thumbnail // setup copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); // test $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); $thumbnail ->resize( 100 , 100 ); // assertions $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' ); // teardown unlink( 'web/uploads/test.png' ); unset( $thumbnail ); Use Comments!

The Test Fixture

Testing Strategies – The Test Fixture
Everything that a test depends on
Data in the database, objects, files, …
Global Fixture
Fresh Fixture

Testing Strategies – The Test Fixture Global Fixture Fresh Fixture Fixture Setup test.png, new CukeetThumbnail() Test 1 Test 2 Test 3 Test 4 Fixture Setup test.png, new CukeetThumbnail() Test 1 Test 2 Fixture Setup test.png, new CukeetThumbnail() Fixture Setup

Test Isolation

Lime 2 Annotations

Testing Strategies – Annotations
Annotations
Control the program flow in the test
Supported Annotations:
@Test
@Before
@After
@BeforeAll
@AfterAll

Testing Strategies – Annotations // @Before copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); // @After unlink( 'web/uploads/test.png' ); unset( $thumbnail ); // @Test: resize() resizes the thumbnail $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' );

Testing Strategies - Annotations
Annotations
Improve test isolation
Fixtures can be built and destroyed automatically
The chance of interacting tests is reduced

Testing Strategies – xUnit Style Test Class class CukeetThumbnailTest extends LimeTestCase { private $thumbnail ; public function setup() { copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $this ->thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); } public function teardown() { unlink( 'web/uploads/test.png' ); unset( $this ->thumbnail); } public function testTheThumbnailCanBeResized() {} } Same As With Annotations

How to Write Testable Code?

Testing Strategies – How to Write Testable Code class CukeetRecipe { public function save() { $user = sfContext::getInstance()->getUser(); if (! $user ->hasCredential( 'admin' )) { throw new RuntimeException( 'No permission!' ); } } }

Testing Strategies – How to Write Testable Code class CukeetRecipe { public function save() { $user = sfContext::getInstance()->getUser(); if (! $user ->hasCredential( 'admin' )) { throw new RuntimeException( 'No permission!' ); } } } Uh oh...

Expensive Operation

Unexpected Side-Effects

Lose Control

Testing Strategies – How to Write Testable Code? ? Test Tested Class Dependency

Dependency Injection

Testing Strategies – Dependency Injection
All dependencies of an object should be passed through method arguments
Constructor Injection
For required dependencies
Setter Injection
For optional dependencies
Parameter Injection

Testing Strategies – Constructor Injection class CukeetRecipe { private $user ; public function __construct(sfBasicSecurityUser $user ) { $this ->user = $user ; } public function save() { if (! $this ->user->hasCredential( 'admin' )) { throw new RuntimeException( 'No permission!' ); } }

Testing Strategies – Dependency Injection // @Before $user = new sfBasicSecurityUser(); $recipe = new CukeetRecipe( $user ); // @Test: Users with credential 'admin' can save $user ->addCredential( 'admin' ); $recipe ->save( $user ); // @Test: Normal users cannot save $t ->expect( 'RuntimeException' ); $recipe ->save( $user );

Testing Strategies – How to Write Testable Code? Test Tested Class Dependency

Testing Strategies – How to Write Testable Code? sloooowwww sfContext, Database, ... Test Tested Class Dependency

How to Replace Slow Dependencies?

Fake Objects

Testing Strategies – How to Write Testable Code? Test Tested Class Fake Object

Testing Strategies – Slow Dependencies class CukeetRecipe { public function save(CukeetCategoryTable $categoryTable ) { if (is_null( $this ->Category)) { $this ->Category = $categoryTable ->findDefault(); } } }

Testing Strategies – Fake Objects // @Test: The default category is assigned, if empty // setup $defaultCategory = new CukeetCategory(); $categoryTable = new FakeCategoryTable( $defaultCategory ); // test $recipe = new CukeetRecipe(); $recipe ->Category = null ; $recipe ->save( $categoryTable ); // assertions $t ->is( $recipe ->Category, $defaultCategory , 'The default...' );

Testing Strategies – Fake Objects class FakeCategoryTable extends CukeetCategoryTable { protected $defaultCategory ; public function __construct(CukeetCategory $defaultCategory ) { $this ->defaultCategory = $defaultCategory ; } public function findDefault() { return $this ->defaultCategory; } }

Testing Strategies – Stub Objects
Stub
Provides fake output
Acts „as if“ it was the real object
Does not have any logic inside

Testing Strategies – Stub Objects // @Test: The default category is assigned, if empty // setup $defaultCategory = new CukeetCategory(); $categoryTable = $t ->stub( 'CukeetCategoryTable' ); $categoryTable ->findDefault()->returns( $defaultCategory ); $categoryTable ->replay(); // test $recipe = new CukeetRecipe(); $recipe ->Category = null ; $recipe ->save( $categoryTable ); // assertions $t ->is( $recipe ->Category, $defaultCategory , 'The default...' ); Stub Generation in Lime 2

Testing Strategies – Stub Objects // @Test: The default category is assigned, if empty // setup $defaultCategory = new CukeetCategory(); $categoryTable = $t ->stub( 'CukeetCategoryTable' ); $categoryTable ->findDefault()->returns( $defaultCategory ); $categoryTable ->replay(); // test $recipe = new CukeetRecipe(); $recipe ->Category = null ; $recipe ->save( $categoryTable ); // assertions $t ->is( $recipe ->Category, $defaultCategory , 'The default...' ); Stub Configuration

Testing Strategies – Stub Objects
Attention!
Don't replace entities (recipe, category, …)
Replace services (table, ...)

Testing Strategies – Stub Objects Test state after test execution Test Tested Class Stub

Testing Strategies – Mock Objects Test behaviour during test execution Test Tested Class Mock

Testing Strategies – Mock Objects
Mock
Behaviour verification
Monitors indirect input
Does the tested object call the right methods?

Testing Strategies – Mock Objects // @Test: Upon destruction all data is saved to the session // setup $storage = $t ->mock( 'CukeetSessionStorage' ); $storage ->write( 'Foo' , 'Bar' ); $storage ->replay(); $user = new CukeetUser( $storage ); // test $user ->setAttribute( 'Foo' , 'Bar' ); $user ->__destruct(); // assertions $storage ->verify();

Testing Strategies – Mock Objects
Attention!
Verifying behaviour leads to unflexible classes
Implementation changes break tests
Overuse of mocks can harm your health

Testing Strategies – Functional Tests $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end()

Testing Strategies – Functional vs. Unit Tests
Functional Tests
„ Acceptance tests“
„ Integration Tests“
Test the system as a whole
Collaboration between classes/components
Are slow!
They do not
Thoroughly test a system for correctness
Test edge cases!

Testing Strategies – Functional vs. Unit Tests
Unit Tests
Test classes/components in isolation
Test edge cases
Are fast!
The biggest part of an application should be tested in unit tests

Testing Strategies – Functional vs. Unit Tests $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end()

Testing Strategies – Test Edge Cases in Unit Tests // CukeetRecipeTableTest.php require_once dirname(__FILE__). '/../bootstrap/doctrine.php' ; $t ->comment( 'findRecent() returns recent recipes of a category' ); // setup ... // test $actual = $table ->findRecent( 2 , $category ); // assertions ... $t ->is( $actual , $expected , 'The correct recipes were returned' );

Testing Strategies – Test Edge Cases in Unit Tests // CukeetRecipeTableTest.php require_once dirname(__FILE__). '/../bootstrap/doctrine.php' ; $t ->comment( 'findRecent() returns recent recipes of a category' ); // setup ... // test $actual = $table ->findRecent( 2 , $category ); // assertions ... $t ->is( $actual , $expected , 'The correct recipes were returned' ); Bootstrap File

Testing Strategies – Test Edge Cases in Unit Tests // bootstrap/doctrine.php require_once dirname(__FILE__). '/unit.php' ; $database = new sfDoctrineDatabase( array ( 'name' => 'doctrine' , 'dsn' => 'sqlite::memory:' , )); // load missing model files and create tables Doctrine::createTablesFromModels(ROOT_DIR. '/lib/model' ); // load fixture data Doctrine::loadData( 'data/fixtures/fixtures.yml' ); Load YAML Fixtures

Testing Strategies – The Test Fixture Global Fixture Fixture Setup test.png, new CukeetThumbnail() Test 1 Test 2 Test 3 Test 4

Testing Strategies – Use Inline Fixtures // @Before reload(); $table = Doctrine::getTable( 'CukeetRecipe' ); // @Test: findRecent() returns recent recipes of a category // setup $category = new CukeetCategory(); $category ->fromArray( array ( 'name' => 'Desserts' , ...)); $recipe1 = new CukeetRecipe(); $recipe1 ->fromArray( array ( 'name' => 'Quatre Quarts' , ...)); $recipe1 ->Category = $category ; $recipe1 ->save(); ...

Testing Strategies – Creation and Helper Functions // @Test: findRecent() returns recent recipes of a category // setup $category = createCategory(); $recipe1 = createRecipeInCategory( $category ); $recipe2 = createRecipeInCategory( $category ); $recipe3 = createRecipe(); $recipe4 = createRecipeInCategory( $category ); save( $recipe1 , $recipe2 , $recipe3 , $recipe4 ); // test $actual = $table ->findRecent( 2 , $category ); // assertions $expected = createCollection( $recipe4 , $recipe2 ); $t ->is( $actual , $expected , 'The correct recipes were returned' );

Testing Strategies – Creation and Helper Functions // bootstrap/doctrine.php function createRecipe( array $properties = array ()) { static $i = 0 ; $recipe = new CukeetRecipe(); $recipe ->fromArray(array_merge( array ( 'name' => 'Recipe ' . ++ $i , 'created_at' => date( 'Y-m-d H:m:i' , $i ), ), $properties )); return $recipe ; }

Testing Strategies – Functional Test Revisited (1) $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end( )

Testing Strategies – Functional Test Revisited (2) $browser ->info( '1 - Recipes are displayed in each category' ) ->info( ' 1.1 - The two most recent recipes are displayed' ) ->click( 'Desserts' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Deluxe Apples/' ) ->checkElement( '#recipes li:last' , '/Quatre Quarts/' ) ->checkElement( '#recipes li' , 2 ) ->end() ->click( 'Burgers' ) ->with( 'response' )->begin() ->checkElement( '#recipes li:first' , '/Kamikaze Burgers/' ) ->checkElement( '#recipes li:last' , '/Surprise Burgers/' ) ->checkElement( '#recipes li' , 2 ) ->end()

Let's Summarize

Testing Strategies - Summary
Su mmary
4 Phase Test
Test Isolation
Annotations
Stubs
Mocks

Part III Whatz Nu in Lime 2

Annotation Support

What Nu in Lime 2 – Annotation Support // @Before copy( 'data/fixtures/test.png' , 'web/uploads/test.png' ); $thumbnail = new CukeetThumbnail( 'web/uploads/test.png' ); // @After unlink( 'web/uploads/test.png' ); unset( $thumbnail ); // @Test: resize() resizes the thumbnail $thumbnail ->resize( 100 , 100 ); $size = getimagesize( $thumbnail ->getPath()); $t ->is( $size , array ( 100 , 100 ), 'The image has been resized' );

Stub & Mock Generation

Whatz Nu in Lime 2 – Mock Objects // @Test: Upon destruction all data is saved to the session // setup $storage = $t ->mock( 'SessionStorage' ); $storage ->write( 'Foo' , 'Bar' )->returns( true )->once(); $storage ->flush()->atLeastOnce(); $storage ->any( 'read' )->throws( 'BadMethodCallException' ); $storage ->replay(); // test ...

Improved Error Messages

What Nu in Lime 2 – Improved Error Messages # got: object(CukeetRecipe) ( # ... # '_data' => array ( # ... # 'name' => 'Quatre Quarts', # ... # ), # ) # expected: object(CukeetRecipe) ( # ... # '_data' => array ( # ... # 'name' => 'Kamikaze Burger', # ... # ), # )

Multi-Processing Support

Whatz Nu in Lime 2 – Multi-Processing Support
Test Suite in 1 Process
Total 3:20 minutes
One CPU core at 100%
One CPU core at 20%
Test Suite in 8 Processes
Total 1:40 minutes
Both CPU cores at 100%
Up To 100% Faster!

Extensibility

Whatz Nu in Lime 2 - Extensibility
Extensible Testers
override is(), like() etc. for specific types
class LimeTesterException extends LimeTesterObject { // don't compare stack trace protected $properties = array ( 'message' , 'code' , 'file' , 'line' ); } LimeTester::register( 'Exception' , 'LimeTesterException' ); $exception1 = new RuntimeException( 'Foobar' , 10 ); $exception2 = new RuntimeException( 'Foobar' , 10 ); $t ->is( $exception1 , $exception2 , 'Hurray!' )

Whatz Nu in Lime 2 - Extensibility
Extensible Outputs
Implement custom outputs for the tests
Log-Output
Email-Output
...

Integration With CI-Tools sismo php Under Control

http://webmozarts.com	224
http://www.symfony.es	103
http://www.slideshare.net	27
http://www.sfexception.com	2
http://coderwall.com	2
http://sfdaycgn.sknuell.lando.local	1
http://static.slidesharecdn.com	1
http://slideclip.b-prep.com	1

Best Practice Testing with Lime 2

by Bernhard Schussek on Sep 04, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

8 Embeds 361

Statistics

Best Practice Testing with Lime 2 — Presentation Transcript