Automated Unit Testing

Automated Unit Testing Getting Started The 2008 DC PHP Conference June 2nd, 2008

Hello!

Mike Lively

Lead Developer Selling Source, Inc.

PHPUnit Contributor

PHPUnit Database Extension

A frequent inhabitant of #phpc

http://digitalsandwich.com

Automated Unit Testing

What is unit testing?

Why unit test?

What do I test?

When do I test?

How do I test it?

Why unit test?

What do I test?

When do I test?

How do I test it?

What is Unit Testing

From Wikipedia: Unit testing is a procedure used to validate that individual units of source code are working properly.

Tests should test as little as possible.

Each test should validate a single unit.

Unit testing is just a single aspect of quality assurance.

Unit Testing

Functional Testing

System Tests

Acceptance Tests

Code Review

Etc.

Why unit test?

What do I test?

When do I test?

How do I test it?

Why should I unit test?

PHP is being used more and more for business critical applications.

There are two periods in which bugs will be found: during development and after development. When would you rather find your bugs?

Regression testing - make it safer to change code.

Why should I unit test?

Makes integration testing and functional testing less stressful.

Provides built in examples of code use.

Can be used effectively as a design validation tool.

Code that is hard to test is often hard to reuse.

Tests will expose difficult to use APIs.

Why unit test?

What do I test?

When do I test?

How do I test it?

What Should I Test?

Options

Test everything

Test critical code

Test frequently changing code

Test buggy code

Coverage has to be balanced with project timelines

Unit testing can save time on debugging

If you can't test everything make sure your testing time is well spent

Why unit test?

What do I test?

When do I test?

How do I test it?

When do I Test?

Option 1: Test Driven Development

Tests before code

Tests are written to fail, code is written to fix the tests

Works for some people, but just like cake, some people like it...some people prefer pie.

Option 2: Test Last

Tests are written on completed code.

When projects are rushed this tends to be the testing method

Tests risk being biased

Option 3: Test During

Test problem areas after they are identified

When do I Test?

Test as soon as you can in your process

Always create regression tests for new bugs

Don't become a victim of testing exhaustion

start small, get comfortable

Why unit test?

What do I test?

When do I test?

How do I test it?

How do I Test

Tests should be independant of the code being tested.

Tests should be easy to write

Tests should be easy to run

Tests should be easy to understand

PHPUnit

Created by Sebastian Bergmann

JUnit Port

Very Rich Feature Set

Mock Objects

Selenium Integration

Database Testing

Much Much More

Alternatives

SimpleTest

PHPT

test-more.php

lime

PHPUnit - Creating a Test Case <?php require_once( 'PHPUnit/Framework/TestCase.php' ); class SubStrTest extends PHPUnit_Framework_TestCase { public function testSubstr () { $this -> assertEquals ( 'o wo' , substr ( 'Hello world!' , 4 , 4 )); } } ?>

PHPUnit - Fixtures

Each test needs an established, predictable environment.

Tests should run completely independent of the results of previous tests.

setUp() - Common setup required for each test in the test case.

tearDown() - Hide your tracks!

PHPUnit - Call Order PHPUnit_Framework_TestCase::setUp(); PHPUnit_Framework_TestCase::testMyCode1(); PHPUnit_Framework_TestCase::tearDown(); PHPUnit_Framework_TestCase::setUp(); PHPUnit_Framework_TestCase::testMyCode2(); PHPUnit_Framework_TestCase::tearDown();

PHPUnit - Fixtures <?php class BankAccountTest extends PHPUnit_Framework_TestCase { protected $ba ; protected function setUp () { $this -> ba = new BankAccount ; } public function testBalanceIsInitiallyZero () { $this -> assertEquals ( 0 , $this -> ba -> getBalance ()); } protected function tearDown () { $this -> ba = NULL ; } } ?>

PHPUnit - Testing Code

The basis of unit testing is making assertions regarding the expected results of a function.

assertEquals(), assertSame(), assertLessThan(), assertContains(), the list goes on and on and on and on ...

PHPUnit - Testing Code <?php $this -> assertEquals ( 'o wo' , substr ( 'hello world' , 4 , 4 )); $this -> assertGreaterThan ( 3 , 4 ); $this -> assertTrue ( TRUE ); $this -> assertNULL ( NULL ); $this -> assertContains ( 'value1' , array( 'value1' , 'value2' , 'value3' )); $this -> assertFileExists ( '/tmp/testfile' ); $this -> assertType ( 'int' , 20 ); $this -> assertRegExp ( '/d{3}-d{2}-d{4}/' , '123-45-6789' ); ?>

PHPUnit - Testing Code

The basis of unit testing is making assertions regarding the expected results of a function.

assertEquals(), assertSame(), assertLessThan(), assertContains(), the list goes on and on and on and on ...

A significantly fuller list: http://www.phpunit.de/pocket_guide/3.2/en/api.html#api.assert

Testing for Failure is also important: setExpectedException()

PHPUnit - Testing Code <?php require_once 'PHPUnit/Framework.php' ; class ExceptionTest extends PHPUnit_Framework_TestCase { public function testException () { $this -> setExpectedException ( 'Exception' , 'exception message' , 100 ); } } ?>

PHPUnit - Running a Test Usage: phpunit [switches] UnitTest [UnitTest.php] --log-graphviz <file> Log test execution in GraphViz markup. --log-json <file> Log test execution in JSON format. --log-tap <file> Log test execution in TAP format to file. --log-xml <file> Log test execution in XML format to file. --coverage-html <dir> Generate code coverage report in HTML format. --coverage-xml <file> Write code coverage information in XML format. --test-db-dsn <dsn> DSN for the test database. --test-db-log-rev <r> Revision information for database logging. --test-db-prefix ... Prefix that should be stripped from filenames. --test-db-log-info ... Additional information for database logging. --filter <pattern> Filter which tests to run. --group ... Only runs tests from the specified group(s). --exclude-group ... Exclude tests from the specified group(s). --loader <loader> TestSuiteLoader implementation to use. --repeat <times> Runs the test(s) repeatedly. --tap Report test execution progress in TAP format. --testdox Report test execution progress in TestDox format.

PHPUnit - Running a Test Usage: phpunit [switches] UnitTest [UnitTest.php] --no-syntax-check Disable syntax check of test source files. --stop-on-failure Stop execution upon first error or failure. --verbose Output more verbose information. --wait Waits for a keystroke after each test. --skeleton Generate skeleton UnitTest class for Unit in Unit.php. --help Prints this usage information. --version Prints the version and exits. --configuration <file> Read configuration from XML file. -d key[=value] Sets a php.ini value.

PHPUnit - Test Suites

When you start amassing tests you need a way to organize them

Test Suites

Allow you to group test cases together so you can run one, all or a specific area's tests

Effective but somewhat cumbersome to set up

PHPUnit - Test Suites <?php if (! defined ( 'PHPUnit_MAIN_METHOD' )) define ( 'PHPUnit_MAIN_METHOD' , 'AllTests::main' ); require_once 'PHPUnit/Framework.php' ; require_once 'PHPUnit/TextUI/TestRunner.php' ; require_once 'MyTest.php' ; require_once 'MyOtherTest.php' ; class AllTests { public static function main () { PHPUnit_TextUI_TestRunner :: run ( self :: suite ()); } public static function suite () { $suite = new PHPUnit_Framework_TestSuite ( 'My Test Suite' ); $suite -> addTestSuite ( 'MyTest' ); $suite -> addTestSuite ( 'MyOtherTest' ); return $suite ; } } if ( PHPUnit_MAIN_METHOD == 'AllTests::main' ) AllTests :: main (); ?>

PHPUnit - Test Suites

Using the xml config

You can specify a directory and a file mask

Test Case class names must follow specific format

Groups can still be set up (we'll talk about that soon)

Not as flexible as test suites

Incredibly simple to add new tests: Just drop the test in the appropriate directory

PHPUnit - XML Configuration Running a list of tests <phpunit> <testsuite name="My Test Suite"> <directory suffix="Test.php"> mytests/directory </directory> <file> mytests/dirtory/fileTest.php </file> </testsuite> </phpunit>

PHPUnit - XML Configuration

So, how can I group tests at will when using XML to set up the test suite?

@group test annotation

--group, --exclude-group command line options

PHPUnit - XML Configuration <?php require_once( 'PHPUnit/Framework/TestCase.php' ); class SubStrTest extends PHPUnit_Framework_TestCase { /** * @group builtins */ public function testSubstr () { $this -> assertEquals ( 'o wo' , substr ( 'Hello world!' , 4 , 4 )); } } ?> >phpunit --configuration tests.xml --group builtins

PHPUnit - Annotations

Annotations can be an incredible time saver!

@test - Marks a particular method in a test case as a test

@dataProvider - Allow running the same tests using different data.

@expectedException - a convenient shortcut for setExpectedException

@assert - Helps in automatically generating tests

@test - Marks a particular method in a test case as a test

<?php require_once( 'PHPUnit/Framework/TestCase.php' ); class SubStrTestAnnote extends PHPUnit_Framework_TestCase { /** * @test */ public function subStr1 () { $this -> assertEquals ( 'o wo' , substr ( 'Hello world!' , 4 , 4 )); } } ?>

@dataProvider - Allow running the same tests using different data.

<?php class DataTest extends PHPUnit_Framework_TestCase { public static function provider () { return array( array( 0 , 0 , 0 ), array( 0 , 1 , 1 ), array( 1 , 1 , 3 ) ); } /** * @dataProvider provider */ public function testAdd ( $a , $b , $c ) { $this -> assertEquals ( $c , $a + $b ); } } ?>

@expectedException - a convenient shortcut for setExpectedException

<?php require_once 'PHPUnit/Framework.php' ; class ExceptionTest extends PHPUnit_Framework_TestCase { /** * @expectedException Exception */ public function testException () { } } ?>

PHPUnit - Skeleton Generator

When creating simple tests this can be incredibly useful.

<?php class Calculator { public function add ( $a , $b ) { return $a + $b ; } } ?>

> phpunit --skeleton Calculator

PHPUnit 3.2.19 by Sebastian Bergmann. Wrote test class skeleton for "Calculator" to "CalculatorTest.php".

Result? /** * @todo Implement testAdd(). */ public function testAdd () { // Remove the following lines when you implement this test. $this -> markTestIncomplete ( 'This test has not been implemented yet.' ); }

PHPUnit - Skeleton Generator > phpunit --verbose CalculatorTest PHPUnit 3.2.19 by Sebastian Bergmann. CalculatorTest I Time: 0 seconds There was 1 incomplete test: 1) testAdd(CalculatorTest) This test has not been implemented yet. /home/mike/phpunit/CalculatorTest.php:65 OK, but incomplete or skipped tests! Tests: 1, Incomplete: 1.

Save some more time!

<?php class Calculator { /** * @assert (0, 0) == 0 * @assert (1, 2) == 3 * @assert (2, 5) == 7 * @assert (3, 3) == 6 * @assert (4, 8) == 12 */ public function add ( $a , $b ) { return $a + $b ; } } ?>

Result

/** * Generated from @assert (0, 0) == 0. */ public function testAdd () { $this -> assertEquals ( 0 , $this -> object -> add ( 0 , 0 ) ); } /** * Generated from @assert (1, 2) == 3. */ public function testAdd2 () { $this -> assertEquals ( 3 , $this -> object -> add ( 1 , 2 ) ); } // ...

PHPUnit - Skeleton Generator > phpunit --verbose CalculatorTest PHPUnit 3.2.19 by Sebastian Bergmann. CalculatorTest ..... Time: 0 seconds OK (5 tests)

PHPUnit - Skeleton Generator @assert (...) == X > assertEquals(X, method(...)) @assert (...) != X > assertNotEquals(X, method(...)) @assert (...) === X > assertSame(X, method(...)) @assert (...) !== X > assertNotSame(X, method(...)) @assert (...) > X > assertGreaterThan(X, method(...)) @assert (...) >= X > assertGreaterThanOrEqual(X, method(...)) @assert (...) < X > assertLessThan(X, method(...)) @assert (...) <= X > assertLessThanOrEqual(X, method(...)) @assert (...) throws X > @expectedException X

PHPUnit - Incomplete Tests TODO: finish slide

PHPUnit - Incomplete Tests Slide TODO: finish slide don't worry...just a little irony

PHPUnit - Incomplete Tests <?php require_once 'PHPUnit/Framework.php' ; class SampleTest extends PHPUnit_Framework_TestCase { public function testSomething () { // Optional: Test anything here, if you want. $this -> assertTrue ( TRUE , 'This should already work.' ); // Stop here and mark this test as incomplete. $this -> markTestIncomplete ( 'This test has not been implemented yet.' ); } } ?>

PHPUnit - SkippedTests <?php require_once 'PHPUnit/Framework.php' ; class DatabaseTest extends PHPUnit_Framework_TestCase { protected function setUp () { if (! extension_loaded ( 'mysqli' )) { $this -> markTestSkipped ( 'The MySQLi extension is not available.' ); } } } ?>

Advanced Features Mock Objects Database Testing Selenium RC PHPUnderControl Come see me again! Same room, same day, 3:30 PM

Thanks http://phpun.it http://planet.phpunit.de http://digitalsandwich.com http://dev.sellingsource.com Questions???