From typing the test to testing the type

From typing the test to testing the type Wim Godden Cu.be Solutions User meeting Aug 25, 2010

About me
Wim Godden
PHP developer since 1997 (PHP/FI 2.0)
ZCE and ZFCE
Developer of OpenX
Owner of Cu.be Solutions
Currently : PHP Architect @ NMBS

About you
Developers ? Managers ?
OOP experience ?
Testing :
What is unit testing ?
Have you written any ?

Brief agenda
What's unit testing ?
How does it work ?
When and how to test
A complete example (with some pitfalls)
Some hints & tips along the way
Testing the type

This is not...
a full tutorial on unit testing
a complete step-by-step guide
a typical unit testing presentation (as you'll notice...)
-> Plenty of online resources, tutorials, conference talks, etc. about the regular stuff

What's unit testing ?
<Wikipedia> Unit testing is a software design and development method where the programmer gains confidence that individual units of source code are fit for use. </Wikipedia>

What's a unit ? <?php class Fridge { protected $beerBottles ; public function getBeerSupply() { return $this ->beerBottles; } public function addBeer( $bottles ) { $this ->beerBottles += $bottles ; return true ; } public function drinkBeer( $bottles ) { $this ->beerBottles -= $bottles ; return true ; } }

Why unit test ?
Write code
Add code
Add more code
At some point you will break something
Misspelling
Typos (= instead of == in an if-statement)
Backwards logic (== instead of !=, < instead of >)
etc.
So what do you do ?
-> You look for the bug !

Why unit test ?
So what happens when you find that bug ?
You fix it of course...
Only to find that you just caused 6 other things to break

Why unit test ?
You test the smallest piece of code
-> if a test fails, you know exactly where to look
Early detection of bugs
Tests are :
automated
repeatable
consistent
Functional tests -> “something's wrong” -> but where ?
Unit tests -> “this is what's wrong in this piece of code”

A simple example <?php class Fridge { protected $beerBottles ; public function addBeer( $bottles ) { $this ->beerBottles += $bottles ; return true ; } public function drinkBeer( $bottles ) { if ( $bottles >= $this ->beerBottles) { $this ->beerBottles -= $bottles ; return true ; } else { // We ran out of beer ! // $wive->sendToStore($beer, 'a lot'); return false ; } } public function getBeerSupply() { return $this ->beerBottles; } } <?php class FridgeTest extends PHPUnit_Framework_TestCase { private $fridge ; public function testAddBeer() { $this ->fridge = new Fridge(); $this ->assertTrue( $this ->fridge->addBeer( 15 )); } public function testDrinkBeer() { $this ->fridge = new Fridge(); $this ->assertTrue( $this ->fridge->drinkBeer( 15 )); } public function testGetBeerSupply() { $this ->fridge = new Fridge(); $this ->assertType( 'integer' , $this ->fridge->getBeerSupply() ); } }

How does it work ?

Running the test
> phpunit <test-filename.php>
Output :
...
Time: 0 seconds
OK (3 tests, 3 assertions)

Assertions
AssertEquals
AssertTrue
AssertFalse
AssertType
AssertGreaterThan
AssertNotNull
…
You can also add your own !

Test case environment
Each test must be run in an identical environment
Tests should run independent of the result of a previous test
How ?  Fixtures
setUp() : creates the initial state
tearDown() : destroys the state

Fixtures
Test1
Starts with empty fridge
Fill the fridge with 15 beers
Run the test (take 14 beers)
Test2
Starts with 1 beer in fridge
Run the test (see if there's 15 beers in the fridge)
-> FAIL !

Fixtures
setUp()
Test1
Run the test (take 14 beers)
tearDown()
Drink all the beer left
in the fridge
setUp()
Test2
Run the test (see if there's 15 beers in the fridge)
-> SUCCESS !
tearDown()

Fixtures example
Running the tests results in :
setUp()
testAddBeer()
tearDown()
setUp()
teatDrinkBeer()
tearDown()
setUp()
testGetBeerSupply()
tearDown()
class FridgeTest extends PHPUnit_Framework_TestCase { private $fridge ; protected function setUp() { $this ->fridge = new Fridge(); } protected function tearDown() { $this ->fridge->drinkBeer( $this ->fridge->getBeerSupply() ); } public function testAddBeer() { $this ->assertTrue( $this ->fridge->addBeer( 15 )); } public function testDrinkBeer() { $this ->assertTrue( $this ->fridge->drinkBeer( 15 )); } public function testGetBeerSupply() { $this ->assertType( 'integer' , $this ->fridge->getBeerSupply() ); } }

The do's and dont's class User { protected $age ; public function getAge() { return $this ->age; } public function setAge( $newAge ) { if ( $newAge ) { if ( $newAge >= 0 && $newAge <= 150 ) { $this ->age = $newAge ; return true ; } else { return false ; } } else { return false ; } } } class UserTest extends PHPUnit_Framework_TestCase { private $User ; protected function setUp() { $this ->User = new User(); } public function testGetAge() { $this ->assertType( 'integer' , $this ->User->getAge()); } protected static function provideAges() { return array ( array (rand( 0 , 150 )), array (rand( 0 , 150 )), array (rand( 0 , 150 )) ); } /** * @dataProvider provideAges */ public function testSetAge( $age ) { $this ->assertTrue( $this ->User->setAge( $age )); }

Problems with this test
What about ages < 0 and > 150 ?
What about non-integers ?
What about empty string ? Null ?
protected static function provideInvalidAges() { return array ( array (- 1 ), array ( 151 ), array ( "not an int" ), array ( "" ), array (NULL) ); } /** * @dataProvider provideInvalidAges */ public function testSetAgeInvalid( $age ) { $this ->assertFalse( $this ->User->setAge( $age )); }

Good test ? protected static function provideAges() { return array ( array (rand( 0 , 150 )), array (rand( 0 , 150 )), array (rand( 0 , 150 )) ); } /** * @dataProvider provideAges */ public function testSetAge( $age ) { $this ->assertTrue( $this ->User->setAge( $age )); } protected static function provideInvalidAges() { return array ( array (- 1 ), array ( 151 ), array ( "not an int" ), array ( "" ), array (NULL) ); } /** * @dataProvider provideInvalidAges */ public function testSetAgeInvalid( $age ) { $this ->assertFalse( $this ->User->setAge( $age )); }

Erratic test results !

Bug in the test or the code ? public function setAge( $newAge ) { if ( $newAge ) { if ( $newAge >= 0 && $newAge <= 150 ) { $this ->age = $newAge ; return true ; } else { return false ; } } else { return false ; } } protected static function provideAges() { return array ( array (rand( 0 , 150 )), array (rand( 0 , 150 )), array (rand( 0 , 150 )) ); } /** * @dataProvider provideAges */ public function testSetAge( $age ) { $this ->assertTrue( $this ->User->setAge( $age )); }

Bug in the code ! public function setAge( $newAge ) { if (is_int( $newAge )) { if ( $newAge >= 0 && $newAge <= 150 ) { $this ->age = $newAge ; return true ; } else { return false ; } } else { return false ; } } protected static function provideAges() { return array ( array (rand( 0 , 150 )), array (rand( 0 , 150 )), array (rand( 0 , 150 )) ); } /** * @dataProvider provideAges */ public function testSetAge( $age ) { $this ->assertTrue( $this ->User->setAge( $age )); }

The code was buggy... what about the test ?

Good tests...
always test minimum and maximum values
always test edge cases
So... good test ? We're testing :
valid parameters
invalid parameters
edge cases
But : are we actually testing setAge() ?
-> We're just checking if setAge() returns true or false !
protected static function provideAges() { return array ( array ( 0 ), array (rand( 1 , 149 )), array (rand( 1 , 149 )), array (rand( 1 , 149 )), array ( 150 ) ); }

Good tests... public function testSetAgeActuallySets() { $newAge = mt_rand( 0 , 150 ); $this ->User->setAge( $newAge ); $this ->assertEquals( $newAge , $this ->User->getAge() ); } Good tests don't just cover code... they test the functionality of the code !

Testing code – not always easy
Some things are hard to test :
Private methods
Final methods
Long methods
Singletons / static methods
Deep inheritance
Too many conditional statements
The secret to writing good tests, is to write testable code

How NOT to test try { $billingAddress = new Address("Meir 12", 2000, "Antwerpen", "BE"); $officeAddress = new Address("Groenplaats 50", 2000, "Antwerpen", "BE"); $customer = new Customer(402, "Jan", "De Man", 32, $billingAddress, $officeAddress); $product = new Product(92, "Speculaas", 5.95); $invoice = new Invoice($customer); $invoice.addItemQuantity($product, 5); $lineItems = $invoice.getLineitems(); if ($lineItems.size() == 1) { $actualLineItem = $lineItems.get(0); $this->assertEquals($invoice, $actualLineItem.getInvoice()); $this->assertEquals($product, $actualLineItem.getProduct()); $this->assertEquals($quantity, $actualLineItem, getQuantity()); $this->assertEquals(5.95, $actualLineItem.getUnitPrice()); } else { $this->assertTrue(false, 'Invoice should have exactly one line item'); } deleteObject($invoice); deleteObject($product); deleteObject($customer); deleteObject($billingAddress); deleteObject($officeAddress); } catch (Exception $e) { // Whatever }

How NOT to test try { $billingAddress = new Address("Meir 12", 2000, "Antwerpen", "BE"); $officeAddress = new Address("Groenplaats 50", 2000, "Antwerpen", "BE"); $customer = new Customer(402, "Jan", "De Man", 32, $billingAddress, $officeAddress); $product = new Product(92, "Speculaas", 5.95); $invoice = new Invoice($customer); $invoice.addItemQuantity($product, 5); $lineItems = $invoice.getLineitems(); $this->assertEquals(1, $lineItems.size()); $actualLineItem = $lineItems.get(0); $this->assertEquals($invoice, $actualLineItem.getInvoice()); $this->assertEquals($product, $actualLineItem.getProduct()); $this->assertEquals($quantity, $actualLineItem, getQuantity()); $this->assertEquals(5.95, $actualLineItem.getUnitPrice()); deleteObject($invoice); deleteObject($product); deleteObject($customer); deleteObject($billingAddress); deleteObject($officeAddress); } catch (Exception $e) { // Whatever }

How NOT to test try { $billingAddress = new Address("Meir 12", 2000, "Antwerpen", "BE"); $officeAddress = new Address("Groenplaats 50", 2000, "Antwerpen", "BE"); $customer = new Customer(402, "Jan", "De Man", 32, $billingAddress, $officeAddress); $product = new Product(92, "Speculaas", 5.95); $invoice = new Invoice($customer); $invoice.addItemQuantity($product, 5); $lineItems = $invoice.getLineitems(); $this->assertEquals(1, $lineItems.size()); $actualLineItem = $lineItems.get(0); $this->assertEquals($invoice, $actualLineItem.getInvoice()); $this->assertEquals($product, $actualLineItem.getProduct()); $this->assertEquals($quantity, $actualLineItem, getQuantity()); $this->assertEquals(5.95, $actualLineItem.getUnitPrice()); try { deleteObject($invoice); } catch (Exception $e) { // Whatever } try { deleteObject($product); } catch (Exception $e) { // Whatever } try { deleteObject($customer); } catch (Exception $e) { // Whatever } try { deleteObject($billingAddress); } catch (Exception $e) { // Whatever } try { deleteObject($officeAddress); } catch (Exception $e) { // Whatever } } catch (Exception $e) { // Whatever }

Make a testObject array protected $testObjects; public function testInvoice() { $billingAddress = new Address("Meir 12", 2000, "Antwerpen", "BE"); $this->addTestObject($billingAddress); $officeAddress = new Address("Groenplaats 50", 2000, "Antwerpen", "BE"); $this->addTestObject($officeAddress); $customer = new Customer(402, "Jan", "De Man", 32, $billingAddress, $officeAddress); $this->addTestObject($customer); $product = new Product(92, "Speculaas", 5.95); $this->addTestObject($product); $invoice = new Invoice($customer); $this->addTestObject($invoice); $invoice.addItemQuantity($product, 5); $lineItems = $invoice.getLineitems(); $this->assertEquals(1, $lineItems.size()); $actualLineItem = $lineItems.get(0); $this->assertEquals($invoice, $actualLineItem.getInvoice()); $this->assertEquals($product, $actualLineItem.getProduct()); $this->assertEquals($quantity, $actualLineItem, getQuantity()); $this->assertEquals(5.95, $actualLineItem.getUnitPrice()); } public function tearDown() { $this->deleteTestObjects(); }
Note : this is a design decision from the very start of your project, since all objects need the same method for deletion

Replace the unneeded data protected $testObjects; public function testInvoice() { $billingAddress = new TestAddress(); $this->addTestObject($billingAddress); $officeAddress = new TestAddress(); $this->addTestObject($officeAddress); $customer = new TestCustomer($billingAddress, $officeAddress); $this->addTestObject($customer); $product = new Product(92, "Speculaas", 5.95); $this->addTestObject($product); $invoice = new Invoice($customer); $this->addTestObject($invoice); $invoice.addItemQuantity($product, 5); $lineItems = $invoice.getLineitems(); $this->assertEquals(1, $lineItems.size()); $actualLineItem = $lineItems.get(0); $this->assertEquals($invoice, $actualLineItem.getInvoice()); $this->assertEquals($product, $actualLineItem.getProduct()); $this->assertEquals($quantity, $actualLineItem, getQuantity()); $this->assertEquals(5.95, $actualLineItem.getUnitPrice()); } public function tearDown() { $this->deleteTestObjects(); }

Instantiating objects (1/3)
Looks like a standard test-class relationship, with invoice being a 'leaf' class
Until you look at the Invoice class

Create a seam which separates your class from other classes

Don't do 'new Customer' in Invoice, instead do it in the test and pass it as a parameter
Advantages :
Objects can be tested separately
Hard-to-test classes can be mocked (replaced by a fake class)
Important : you have to keep this in mind when writing your code !

Testable code
Good object orientation
Dependency injection
Uses TDD

Test-Driven Development
Reverse programming logic :
Write the test
then
Write the code
If code is too complex -> hard to test
Write the test first -> code will do what the test dictates
It's a habit...
and it's addicting !

Random thoughts...
# lines test ≈ # lines code
Why ? Because you need to test invalid cases, edge cases, etc.
You can add your own assertions
Multi-developer projects :
If you launch tests simultaneously -> unexpected (and unrepeatable) results -> agree on a token
Agree on 'punishment' for who breaks the build (buy a beer, go get coffee, …)
-> makes people test locally before committing code

We typed the test... Now let's test the type !

What is it ?
A PHPUnit addon / patch
Allows type validation during unit testing
Time it took to build : a few days
Time it takes for developers to use : none
Just add --check-param-types to your PHPUnit call
See the magic happen

How it works
It analyzes the docblock
When the method is called from our test, it compares the type of each parameter with the type in the docblock :
The result :
/** * Cat constructor * @param string $name The name of the cat * @param int $lives The number of lives this cat has left */ PHPUnit 3.5 by Sebastian Bergmann. FF... Time: 1 second, Memory: 3.25Mb There were 2 failures: 1) CatTest::test__constructWithNegativeLives Invalid type calling Cat->__construct : parameter 1 ($name) should be of type string but got bool instead in /home/dev/paramtest/tests/CatTest.php 2) CatTest::test__constructWithNegativeLives Invalid type calling Cat->__construct : parameter 2 ($lives) should be of type int but got string instead in /home/dev/paramtest/tests/CatTest.php FAILURES! Tests: 5, Assertions: 8, Failures: 2. $cat = new Cat(true, "9");

Why it's useful
PHP's dynamic typing is great, but it can be a pain !
If you write a library :
You want to be sure people pass the right parameter types
You don't want to do is_int(), is_bool(), ... all the time
If you modify your function's parameters :
If you forget to modify the callee : warning will be shown
If you forget to update your docblock : warning will be shown
Parameter type validation was designed to help your data be consistent and of the type expected during design !
(and it forces you to keep the docblock updated,
which is good for API documentation quality !)

How to install and run
Go to http://github.com/wimg/phpunit
Click on 'Download Source'
Run it :
> php <path-to-phpunit.php> --check-param-types <your-test.php>
Be amazed at how many type issues you have ;-)

Warning
Work in progress
Slows down unit tests by factor of 2.5
MySQL = trouble
(but we're looking into it)

Let's refresh
Tests should always pass or fail, but never behave erratically
-> Consistency is the key to code quality
-> Consistency gives the developer confidence in testing
Always test valid and invalid cases
Always test the edge cases
Don't rely on completion of other tests
-> Use fixtures
Use docblocks
Test your types for consistency
-> Unit tests take a developer from :
This code should work
to
I can prove it works

Cu.be Solutions
Founded in 2010
Spinoff of FirstLink Networks (founded in 2000)
High-quality open source solutions
Centered around PHP
Contribute to open source projects
(ZF, Xdebug, PHPUnit, OpenX, ...)
We're hiring !
-> http://cu.be/jobs
-> info@cu.be

Questions ?

Thanks !

Contact
Web http://joind.in/talk/view/xxx
http://cu.be
http://techblog.wimgodden.be
Slides http://www.slideshare.net/wimg
Twitter @wimgtr
E-mail [email_address]

From typing the test to testing the type

by Wim Godden on Sep 16, 2010

Accessibility

Categories

Upload Details

Usage Rights

1 Embed 1

Statistics

From typing the test to testing the type Presentation Transcript