Testing PHP/Web Applications with PHPUnit and Selenium

Welcome!

Testing PHP/Web Applications

with PHPUnit 3 and Selenium

Sebastian Bergmann

http://sebastian-bergmann.de/

May 16 th 2007

Who I Am

Sebastian Bergmann

29 years old

Contributor to the PHP Project since 2000

Creator of PHPUnit

Author, Consultant, Trainer

Developer for eZ Systems AS

Who Are You?

Your Background

PHP 5?

OOP?

Testing?

PHPUnit?

Selenium?

The Need for Testing

PHP has evolved from a niche language for small websites to a powerful tool making strong inroads into large-scale, business-critical Web-based systems.

For instance, financial institutions use PHP to develop and maintain solutions for Basel II Credit Rating.

Critical business logic like this needs to work correctly.

But how do you ensure that it does?

You test it, of course!

What Needs Testing?

Web Application

Backend

Business Logic

Reusable Components

Frontend

Form Processing and Template Rendering

Rich Interfaces with AJAX, JSON, ...

Feeds, Web Services, ...

And How Do We Test It?

Web Application

Backend

Functional Testing of Business Logic with Unit Tests

Reusable Components

Third-Party components hopefully come with their own tests

Frontend

Testing with Acceptance Tests or System Tests that “run in the browser”

Testing of Feeds, Web Services, etc. with Unit Tests

Compatibility Testing for Browser / OS combinations

Performance Testing and Security Testing

Testing Software Systems

Unit Testing

Runnable code fragments, so-called Unit Tests , test the correctness of parts, units , of the software.

System Testing

Conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements (from Wikipedia).

Non-Functional Testing

Performance, Stability, Security, Usability, ...

Testing Software Systems

Developer Tests

Ensure that the code works correctly.

Acceptance Tests

Ensure that the code does what the customer wants.

Tool Support for Testing

To make (code) testing viable, good tool support is needed.

This is where a testing framework such as PHPUnit comes into play.

Requirements

Reusable test environment

Strict separation of production code and test code

Automatic execution of test code

Analysis of the result

Easy to learn to use and easy to use

Tools for PHP (and Web) Testing

Unit Testing

PHPUnit

SimpleTest

System Testing

Selenium

PHPUnit + Selenium

Non-Functional Testing

Perfomance, Load, Stress, Reliability, Availability

ab, httperf, JMeter, Grinder, OpenSTA, ...

Security

Chorizo

PHPUnit + Chorizo

Gateway Drug to Test Infection <?php class Calculator { public function add ( $a , $b ) { return $a + $b ; } } ?>

Gateway Drug to Test Infection <?php class Calculator { /** * @assert (0,0) == 0 * @assert (0,1) == 1 * @assert (1,0) == 1 * @assert (1,1) == 2 * @assert (1,2) != 3 */ public function add ( $a , $b ) { return $a + $b ; } } ?>

Gateway Drug to Test Infection

Back to the Beginning

We just used PHPUnit's Skeleton Generator to automatically (generate and) run tests from source code annotations.

While this is handy for simple test cases such as the ones in the previous example, it does not cater well to more complex test scenarios.

So let us take a step back and start from the beginning ...

Writing a Test Case <?php require_once 'PHPUnit/Framework.php' ; require_once 'Calculator.php' ; class CalculatorTest extends PHPUnit_Framework_TestCase { public function testAdd () { $calculator = new Calculator ; $this -> assertEquals ( 0 , $calculator -> add ( 0 , 0 )); $this -> assertEquals ( 1 , $calculator -> add ( 0 , 1 )); $this -> assertEquals ( 1 , $calculator -> add ( 1 , 0 )); $this -> assertEquals ( 2 , $calculator -> add ( 1 , 1 )); $this -> assertNotEquals ( 3 , $calculator -> add ( 1 , 2 )); } } ?>

Running a Test Case

Writing a Test Case <?php require_once 'PHPUnit/Framework.php' ; require_once 'Calculator.php' ; class CalculatorTest extends PHPUnit_Framework_TestCase { protected $calculator ; protected function setUp () { $this -> calculator = new Calculator ; } public function testAdd () { $this -> assertEquals ( 0 , $this -> calculator -> add ( 0 , 0 )); } // ... ?>

Running a Test Case

Organizing Tests into Suites

Application/

Package/

Class (Application/Package/Class.php)

...

...

Tests/

AllTests.php

Package/

AllTests.php

ClassTest (Application/Tests/Package/ClassTest.php)

Organizing Tests into Suites <?php require_once 'PHPUnit/Framework.php' ; require_once 'Application/Tests/Package/AllTests.php' ; class AllTests { public static function suite () { $suite = new PHPUnit_Framework_TestSuite ( 'Project' ); $suite -> addTest ( Package_AllTests :: suite ()); return $suite ; } } ?>

Organizing Tests into Suites <?php require_once 'PHPUnit/Framework.php' ; require_once 'Application/Tests/Package/ClassTest.php' ; class Package_AllTests { public static function suite () { $suite = new PHPUnit_Framework_TestSuite ( 'Package' ); $suite -> addTestSuite ( 'Package_ClassTest' ); return $suite ; } } ?>

Testing Practices

You can always write more tests.

However, you will quickly find that only a fraction of the tests you can imagine are actually useful.

What you want is to write tests that fail even though you think they should work, or tests that succeed even though you think they should fail.

Another way to think of it is in cost / benefit terms.

You want to write tests that will pay you back with information.

This slide contains material by Erich Gamma.

Testing Practices

During Development

A test suite is invaluable while refactoring.

During Debugging

Verify that you can reproduce the defect.

Find the smallest-scale demonstration of the defect in the code.

Write an automated test that fails now but will succeed when the defect is fixed.

Fix the defect.

Test-Driven Development

Tests drive the development.

Tests written before code.

No code without tests.

This slide contains material by Brady Kelly.

Method of designing software, not just a method of testing software.

Test

What do we want X to do?

How do we want to tell X to do it?

How will we know when X has done it?

Code

How does X do it?

Test-Driven Development The BankAccount Example

Let's implement a class that represents a bank account using TDD.

API

Retrieve the current balance.

Set the balance.

Deposit money.

Withdraw money.

Rules

Balance is initially zero.

Balance cannot become negative.

Test-Driven Development The BankAccount Example <?php require_once 'PHPUnit/Framework.php' ; require_once 'BankAccount.php' ; class BankAccountTest extends PHPUnit_Framework_TestCase { private $ba ; protected function setUp () { $this -> ba = new BankAccount ; } public function testBalanceIsInitiallyZero () { $this -> assertEquals ( 0 , $this -> ba -> getBalance ()); } } ?>

Test-Driven Development The BankAccount Example <?php require_once 'PHPUnit/Framework.php' ; require_once 'BankAccount.php' ; class BankAccountTest extends PHPUnit_Framework_TestCase { // ... public function testBalanceCannotBecomeNegative () { try { $this -> ba -> withdrawMoney ( 1 ); } catch ( BankAccountException $e ) { return; } $this -> fail (); } } ?>

Test-Driven Development The BankAccount Example <?php require_once 'PHPUnit/Framework.php' ; require_once 'BankAccount.php' ; class BankAccountTest extends PHPUnit_Framework_TestCase { // ... public function testBalanceCannotBecomeNegative2 () { try { $this -> ba -> depositMoney (- 1 ); } catch ( BankAccountException $e ) { return; } $this -> fail (); } } ?>

Test-Driven Development The BankAccount Example <?php require_once 'PHPUnit/Framework.php' ; require_once 'BankAccount.php' ; class BankAccountTest extends PHPUnit_Framework_TestCase { // ... public function testBalanceCannotBecomeNegative3 () { try { $this -> ba -> setBalance (- 1 ); } catch ( BankAccountException $e ) { return; } $this -> fail (); } } ?>

Test-Driven Development The BankAccount Example <?php require_once 'BankAccountException.php' ; class BankAccount { private $balance = 0 ; public function getBalance () { return $this -> balance ; } public function setBalance ( $balance ) { if ( $balance >= 0 ) { $this -> balance = $balance ; } else { throw new BankAccountException ; } } } ?>

Test-Driven Development The BankAccount Example <?php require_once 'BankAccountException.php' ; class BankAccount { // ... public function depositMoney ( $balance ) { $this -> setBalance ( $this -> getBalance () + $balance ); return $this -> getBalance (); } public function withdrawMoney ( $balance ) { $this -> setBalance ( $this -> getBalance () - $balance ); return $this -> getBalance (); } } ?>

Test-Driven Development The BankAccount Example

Code Coverage The BankAccount Example

Test-Driven Development The BankAccount Example <?php require_once 'PHPUnit/Framework.php' ; require_once 'BankAccount.php' ; class BankAccountTest extends PHPUnit_Framework_TestCase { // ... public function testDepositWithdrawMoney () { $this -> assertEquals ( 0 , $this -> ba -> getBalance ()); $this -> ba -> depositMoney ( 1 ); $this -> assertEquals ( 1 , $this -> ba -> getBalance ()); $this -> ba -> withdrawMoney ( 1 ); $this -> assertEquals ( 0 , $this -> ba -> getBalance ()); } } ?>

Code Coverage The BankAccount Example

Test-Driven Development

Microsoft Case Study

TDD project had twice the code quality.

Writing the tests required 15% more time.

IBM Case Study

40% fewer defects.

No impact on the team's productivity.

John Deere / Ericsson Case Study

TDD produces higher quality code.

Impact of 16% on the team's productivity.

But the comparison team did not test at all.

The Testivus Manifestivus

Less testing dogma, more testing karma.

Any tests are better than no tests.

Testing beats debugging.

Test first, during, or after – whatever works best for you.

If a method, technique, or tool gives you more or better tests, use it.

Alberto Savoia on DeveloperTesting.com

PHPUnit Features

Test Framework

Member of the xUnit family of test frameworks

Mock Objects

Integration

Selenium RC

CruiseControl

Even More Cool Stuff :-)

Code Coverage

Test Database

Mutation Testing (GSoC07 project)

Selenium

Selenium

Test tool for web applications

Browser Compatibility Testing

System Functional Testing

Runs in the browser

Selenium IDE

IDE for Selenium tests

Record, edit, debug tests in the browser

Selenium

Selenium RC

Automated execution of Selenium tests

Tests can be specified in any programming language

PHP bindings exist

PHPUnit can talk to a Selenium RC server directly

With the right setup, a test can be tested in different browsers on different operating systems

Selenium RC

The End

Thank you for your interest!

These slides will be available shortly on http://sebastian-bergmann.de/talks/.

Bibliography

T. Bhat, N. Nagappan, “Evaluating the Efficacy of Test-Driven Development: Industrial Case Studies”, ISESE'06, September 21–22, 2006, Rio de Janeiro, Brazil.

L. Williams, E. M. Maximilien, and M. Vouk, “Test-Driven Development as a Defect-Reduction Practice”, Proceedings of IEEE International Symposium on Software Reliability Engineering, Denver, CO, pp. 34-45, 2003.

B. George and L. Williams, “A Structured Experiment of Test-Driven Development”, Information and Software Technology (IST), 46(5), pp. 337-342, 2003.

License

This presentation material is published under the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported license.

You are free:

to Share – to copy, distribute and transmit the work.

Under the following conditions:

Attribution. You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work).