The State of
QA Tools for PHP
Sebastian Bergmann
October 31st 2009

Sebastian Bergmann
 Co-Founder and
Principal Consultant
with thePHP.cc
 Creator of PHPUnit
 Involved in the PHP
project since 2000

This presentation is about tools
that help you build better software.

Tools solve problems.
Problems you should think about before it is too late.

When things go wrong in software projects,
the team has to work overtime and cancel vacations.

More often than not,
deadlines and quality goals are missed nevertheless.

You will ask yourself: ”Why did I not test this?”

Does my code work?
 Be confident in your code!
 Do not be afraid of changing your code!

Does my code work?
Classic Approach
 Write a test program
 Run the test program
 Manually verify the output
 Delete the test program

Lets focus on a more modern approach.

Does my code work?
Agile Approach
Write a Unit Test
 Executable specification
 Automatic evaluation
 Simple test environment
 Instant feedback
 Keep as regression test

Unit Tests improve the confidence in your code
as they detect problems as early as possible.

phpunit
 De-facto standard for the unit testing
of PHP applications
 Member of the xUnit family
 Inspired by
 JUnit
 TestNG
 JUnitour
 JExample
 …

phpunit
Show
Me
The
Code!

phpunit
<?php
require_once 'BankAccount.php';
class BankAccountTest extends PHPUnit_Framework_TestCase {
}

phpunit
<?php
require_once 'BankAccount.php';
class BankAccountTest extends PHPUnit_Framework_TestCase {
public function testBalanceIsInitiallyZero() {
}
}

phpunit
<?php
require_once 'BankAccount.php';
class BankAccountTest extends PHPUnit_Framework_TestCase {
public function testBalanceIsInitiallyZero() {
$ba = new BankAccount;
$this->assertEquals(0, $ba->getBalance());
}
}

phpunit
<?php
require_once 'BankAccount.php';
class BankAccountTest extends PHPUnit_Framework_TestCase {
protected $ba;
protected function setUp()
{
$this->ba = new BankAccount;
}
public function testBalanceIsInitiallyZero() {
$this->assertEquals(0, $this->ba->getBalance());
}
}

phpunit
<?php
require_once 'BankAccount.php';
class BankAccountTest extends PHPUnit_Framework_TestCase {
public function testBalanceIsInitiallyZero() {
$ba = new BankAccount;
$this->assertEquals(0, $ba->getBalance());
return $ba;
}
/**
* @depends testBalanceIsInitiallyZero
* @expectedException RuntimeException
*/
public function testBalanceCannotBecomeNegative(BankAccount $ba) {
$ba->withdrawMoney(1);
}
}

phpunit
<?php
require_once 'BankAccount.php';
class BankAccountTest extends PHPUnit_Framework_TestCase {
public function testBalanceIsInitiallyZero() {
$ba = new BankAccount;
$this->assertEquals(0, $ba->getBalance());
return $ba;
}
/**
* @depends testBalanceIsInitiallyZero
* @expectedException RuntimeException
*/
public function testBalanceCannotBecomeNegative(BankAccount $ba) {
$ba->withdrawMoney(1);
}
/**
* @depends testBalanceIsInitiallyZero
* @expectedException RuntimeException
*/
public function testBalanceCannotBecomeNegative2(BankAccount $ba) {
$ba->depositMoney(-1);
}
// ...
}

phpunit
<?php
require_once 'BankAccount.php';
class BankAccountTest extends PHPUnit_Framework_TestCase {
// ...
/**
* @depends testBalanceIsInitiallyZero
*/
public function testDepositingMoneyWorks(BankAccount $ba) {
$ba->depositMoney(1);
$this->assertEquals(1, $ba->getBalance());
return $ba;
}
}

phpunit
<?php
require_once 'BankAccount.php';
class BankAccountTest extends PHPUnit_Framework_TestCase {
// ...
/**
* @depends testBalanceIsInitiallyZero
*/
public function testDepositingMoneyWorks(BankAccount $ba) {
$ba->depositMoney(1);
$this->assertEquals(1, $ba->getBalance());
return $ba;
}
/**
* @depends testDepositingMoneyWorks
*/
public function testWithdrawingMoneyWorks(BankAccount $ba) {
$ba->withdrawMoney(1);
$this->assertEquals(0, $ba->getBalance());
}
}

phpunit
sb@thinkpad ~ % phpunit BankAccountTest
PHPUnit 3.4.2 by Sebastian Bergmann.
.....
Time: 0 seconds
OK (5 tests, 5 assertions)

phpunit
Tests can serve as an executable specification.

phpunit
Tests can serve as an executable specification.
sb@thinkpad ~ % phpunit --testdox BankAccountTest
PHPUnit 3.4.2 by Sebastian Bergmann.
BankAccount
[x] Balance is initially zero
[x] Balance cannot become negative
[x] Depositing money works
[x] Withdrawing money works

phpunit
Lets break something …
<?php
class BankAccount {
protected $balance = 1;
public function getBalance() {
return $this->balance;
}
protected function setBalance($balance) {
if ($balance >= 0) {
$this->balance = $balance;
} else {
throw new RuntimeException;
}
}
public function depositMoney($balance) {
$this->setBalance($this->getBalance() + $balance);
}
public function withdrawMoney($balance) {
$this->setBalance($this->getBalance() - $balance);
}
}
?>

phpunit
… and see how tests fail.
sb@thinkpad ~ % phpunit --verbose BankAccountTest
PHPUnit 3.4.2 by Sebastian Bergmann.
FSSSS
Time: 0 seconds
There was 1 failure:
1) BankAccountTest::testBalanceIsInitiallyZero
Failed asserting that <integer:1> matches expected <integer:0>.
/home/sb/BankAccountTest.php:7
There were 4 skipped tests:
1) BankAccountTest::testBalanceCannotBecomeNegative
This test depends on "BankAccountTest::testBalanceIsInitiallyZero" to pass.
2) BankAccountTest::testBalanceCannotBecomeNegative2
This test depends on "BankAccountTest::testBalanceIsInitiallyZero" to pass.
3) BankAccountTest::testDepositingMoneyWorks
This test depends on "BankAccountTest::testBalanceIsInitiallyZero" to pass.
4) BankAccountTest::testWithdrawingMoneyWorks
This test depends on "BankAccountTest::testDepositingMoneyWorks" to pass.
FAILURES!
Tests: 1, Assertions: 1, Failures: 1, Skipped: 4.

A unit test should run in less than 1 ms.

Software Testing Categorization
 Small: Unit Tests
 Check conditional logic in the code
 A debugger should not be required in case of
failure
 Runs in less than 1 ms
 Medium: Functional Tests
 Check whether the interfaces between
classes abide by their contracts
 Large: End-to-End Tests
 Check for ”wiring bugs”
This slide contains material by Miško Hevery

Software Metrics
Measuring Software and Software Quality
„You cannot control what
you cannot measure.“
– Tom DeMarco

Software Metrics
Code Coverage
 Which statements, branches, and paths
are executed when the tests run?
 Statement Coverage
 Branch Coverage
 Path Coverage
 100% Code Coverage is a required,
but not a sufficient criteria for test
completeness

Software Metrics
Code Coverage
sb@thinkpad ~ % phpunit --coverage-html /tmp/report BankAccountTest
PHPUnit 3.4.2 by Sebastian Bergmann.
.....
Time: 0 seconds
OK (5 tests, 5 assertions)
Generating code coverage report, this may take a moment.

Software Metrics
Code Analysis

Software Metrics
Lines of Code
 Text-based metric for code size
 Various definitions
 Lines of Code (LOC)
 Comment Lines of Code (CLOC)
 Non-Comment Lines of Code (NCLOC)
 Executable Lines of Code (ELOC)
 Ratios can be interesting
 CLOC / (E)LOC

Software Metrics
Lines of Code
sb@thinkpad ~ % phploc --count-tests /usr/local/src/php-object-freezer
phploc 1.3.0 by Sebastian Bergmann.
Directories: 10
Files: 30
Lines of Code (LOC): 4032
Cyclomatic Complexity / Lines of Code: 0.06
Executable Lines of Code (ELOC): 1218
Comment Lines of Code (CLOC): 2007
Non-Comment Lines of Code (NCLOC): 2025
Interfaces: 2
Classes: 19
Abstract: 1 (5.26%)
Concrete: 18 (94.74%)
Lines of Code / Number of Classes: 153
Methods: 51
Scope:
Non-Static: 48 (94.12%)
Static: 3 (5.88%)
Visibility:
Public: 35 (68.63%)
Non-Public: 16 (31.37%)
Lines of Code / Number of Methods: 57
Cyclomatic Complexity / Number of Methods: 2.41
Functions: 0
Constants: 1
Global constants: 0
Class constants: 1
Tests:
Classes: 9
Methods: 76

Software Metrics
Code Duplication
 Two sequences of code are duplicate
when they are
 textually identical
 token for token identical
 functionally identical
 Problems
 Duplicate code contradicts code reuse
 Co-Evolution of clones hinders maintenance

Software Metrics
Code Duplication
sb@thinkpad ~ % phpcpd /usr/local/src/phpunit/trunk/PHPUnit
phpcpd 1.2.0 by Sebastian Bergmann.
Found 4 exact clones with 131 duplicated lines in 7 files:
- Extensions/Database/DataSet/AbstractTable.php:156-190
Extensions/Database/DataSet/ReplacementTable.php:172-206
- Samples/BankAccountDB/BankAccountDBTest.php:84-128
Samples/BankAccountDB/BankAccountDBTestMySQL.php:84-128
- Tests/Extensions/Database/DataSet/XmlDataSetsTest.php:71-98
Tests/Extensions/Database/DataSet/YamlDataSetTest.php:70-97
- Tests/Extensions/Database/DataSet/XmlDataSetsTest.php:71-97
Tests/Extensions/Database/DataSet/CsvDataSetTest.php:70-96
0.21% duplicated lines out of 61720 total lines of code.

Software Metrics
Code Complexity
 Cyclomatic Complexity
 Counts the number of branching points
if, for, foreach, while, case, catch, &&, ||,
ternary operator (?:)
 NPath Complexity
 Counts the number execution paths
 Interpretation
 Higher complexity leads to more errors
 Higher complexity makes testing harder

Software Metrics
pdepend
 Static Analysis of PHP Code
 Software Metrics
 Software Visualization
 Helps to identify parts of an application that
should be refactored

Software Metrics
phpmd
 Static Analysis of PHP Code
 Based on pdepend
 Reports violations of rules that operate on raw
software metrics data

Software Metrics
phpcs
 Static Analysis of PHP Code
 Based on ext/tokenizer
 ”Sniffs”
 Coding Standard
 Software Metrics
 Bug Patterns
 Performance Patterns
 ...

Software Metrics
bytekit-cli
 Static Analysis of PHP Code
 Based on ext/bytekit
 Disassembles PHP bytecode
 Visualizes PHP bytecode
 Scans PHP bytecode
 … for disallowed opcode sequences
 … for direct output of variables
 …

Software Metrics
bytekit-cli
1 <?php
2 if (TRUE) {
3 print '*';
4 }
5 ?>
sb@thinkpad ~ % bytekit if.php
bytekit-cli 1.0.0 by Sebastian Bergmann.
Filename: /home/sb/if.php
Function: main
Number of oplines: 8
line # opcode result operands
-----------------------------------------------------------------------------
2 0 EXT_STMT
1 JMPZ true, ->6
3 2 EXT_STMT
3 PRINT ~0 '*'
4 FREE ~0
4 5 JMP ->6
6 6 EXT_STMT
7 RETURN 1

Software Metrics
bytekit-cli
1 <?php
2 if (TRUE) {
3 print '*';
4 }
5 ?>
sb@thinkpad ~ % bytekit --graph /tmp --format svg if.php

Software Metrics
bytekit-cli
sb@thinkpad ~ % bytekit --rule DisallowedOpcodes:EVAL
/usr/local/src/phpunit/trunk/PHPUnit
bytekit-cli 1.0.0 by Sebastian Bergmann.
- Disallowed opcode "EVAL"
in /usr/local/src/phpunit/trunk/PHPUnit/Framework/TestCase.php:1160
- Disallowed opcode "EVAL"
in /usr/local/src/phpunit/trunk/PHPUnit/Framework/TestCase.php:1061
- Disallowed opcode "EVAL"
in /usr/local/src/phpunit/trunk/PHPUnit/Extensions/PhptTestCase.php:223
- Disallowed opcode "EVAL"
in /usr/local/src/phpunit/trunk/PHPUnit/TextUI/Command.php:169

QA Tools for PHP
Overview
http://phpqatools.org/

Build Automation
Automate {error­prone|recurring|tedious} tasks

Build Automation
Apache Ant
 Java-based build tool
 Kind of like make, without make's
wrinkles
 Build files are XML-based, calling out a
target tree where various tasks get
executed

Build Automation
Apache Ant
<project name="php-object-freezer" default="build" basedir=".">
<target name="clean">
<!-- Clean up -->
<delete dir="build"/>
<!-- Create build directories -->
<mkdir dir="build/api"/>
<mkdir dir="build/coverage"/>
<mkdir dir="build/logs"/>
<mkdir dir="build/pdepend"/>
<!-- Generate dummy PMD logfile -->
<echoxml file="build/logs/pmd.xml">
<pmd version="dummy"/>
</echoxml>
</target>
<!-- Update checkout -->
<target name="update">
<exec dir="${basedir}/source" executable="git">
<arg line="pull"/>
</exec>
</target>

Build Automation
Apache Ant
<!-- Run unit tests and generate junit.xml and clover.xml -->
<target name="phpunit">
<exec dir="${basedir}/source" executable="phpunit" failonerror="true">
<arg line="--log-junit ${basedir}/build/logs/junit.xml
--coverage-clover ${basedir}/build/logs/clover.xml
--coverage-html ${basedir}/build/coverage
Tests" />
</exec>
</target>
<!-- Run pdepend, phpcpd, phpcs, and phpdoc in parallel -->
<target name="parallelTasks">
<parallel>
<antcall target="pdepend"/>
<antcall target="phpcpd"/>
<antcall target="phpcs"/>
<antcall target="phpdoc"/>
</parallel>
</target>
<!-- Generate pmd-cpd.xml -->
<target name="phpcpd">
<exec dir="${basedir}/source" executable="phpcpd">
<arg line="--log-pmd ${basedir}/build/logs/pmd-cpd.xml Object" />
</exec>
</target>

Build Automation
Apache Ant
<!-- Generate jdepend.xml and software metrics charts -->
<target name="pdepend">
<exec dir="${basedir}" executable="pdepend">
<arg line="--jdepend-xml=${basedir}/build/logs/jdepend.xml
--jdepend-chart=${basedir}/build/pdepend/08-dependencies.svg
--overview-pyramid=${basedir}/build/pdepend/09-overview.svg
source/Object" />
</exec>
</target>
<!-- Generate checkstyle.xml -->
<target name="phpcs">
<exec dir="${basedir}" executable="phpcs">
<arg line="--report=checkstyle
--report-file=${basedir}/build/logs/checkstyle.xml
--standard=Sebastian
source/Object" />
</exec>
</target>

Build Automation
Apache Ant
<!-- Generate API documentation -->
<target name="phpdoc">
<exec dir="${basedir}/source/Object" executable="phpdoc">
<arg line="-q -ct type -ue on
-tb /usr/share/pear/data/phpUnderControl/data/phpdoc
-o HTML:Phpuc:phpuc
-t ${basedir}/build/api
-d ." />
</exec>
</target>
<target name="build" depends="clean,update,parallelTasks,phpunit"/>
</project>

Build Automation
Apache Ant
sb@thinkpad php-object-freezer % ant
Buildfile: build.xml
clean:
[delete] Deleting directory /usr/local/cruisecontrol/projects/php-object-freezer/build
[mkdir] Created dir: /usr/local/cruisecontrol/projects/php-object-freezer/build/api
[mkdir] Created dir: /usr/local/cruisecontrol/projects/php-object-freezer/build/coverage
[mkdir] Created dir: /usr/local/cruisecontrol/projects/php-object-freezer/build/logs
[mkdir] Created dir: /usr/local/cruisecontrol/projects/php-object-freezer/build/pdepend
update:
[exec] Already up-to-date.
pdepend:
[exec] PHP_Depend 0.9.7 by Manuel Pichler
[exec]
[exec] Parsing source files:
[exec] .......... 10
[exec]
[exec] Executing Inheritance-Analyzer:
[exec] 16
[exec]
[exec] Executing NodeCount-Analyzer:
[exec] .. 51
[exec]
[exec] Executing Coupling-Analyzer:
[exec] ... 76
[exec]
[exec] Executing Dependency-Analyzer:
[exec] .. 58
[exec]
[exec] Executing NodeLoc-Analyzer:
[exec] ... 61
[exec]
[exec] Executing CyclomaticComplexity-Analyzer:
[exec] ... 70
[exec]
[exec] Generating pdepend log files, this may take a moment.
[exec]
[exec] Time: 00:01; Memory: 10.00Mb
phpcpd:
[exec] phpcpd 1.2.0 by Sebastian Bergmann.
[exec]
[exec] 0.00% duplicated lines out of 1686 total lines of code.
phpcs:
[exec] Result: 1
phpdoc:
phpunit:
[exec] PHPUnit 3.4.2 by Sebastian Bergmann.
[exec]
[exec] ............................................................ 60 / 78
[exec] ..................
[exec]
[exec] Time: 3 seconds
[exec]
[exec] OK (78 tests, 166 assertions)
[exec]
[exec] Writing code coverage data to XML file, this may take a moment.
[exec]
[exec] Generating code coverage report, this may take a moment.
build:
BUILD SUCCESSFUL
Total time: 9 seconds

Continuous Integration
Feel the pulse of your project!

Continuous Integration
Software development practice where members
of a team integrate their work frequently
 Usually each person integrates at least daily,
leading to multiple integrations per day
 Each integration is verified by an automated
build (including test) to detect integration
errors as quickly as possible
 Leads to significantly reduced integration
problems and allows a team to develop
cohesive software more rapidly

Continuous Integration
CruiseControl
Framework for a continuous build process
 Includes, but is not limited to, plugins for
email notification, Apache Ant, Phing, and
various source control tools
 A web interface is provided to view the
details of the current and previous builds

Continuous Integration
phpUnderControl
Customization of CruiseControl that caters to
the needs of PHP projects
 PHPUnit
 PHPDocumentor
 PHP_CodeSniffer
 (PHP_Depend)
 (phpmd)
 (phpcpd)

Show
Me
A
Demo!

Continuous Integration
Alternatives
 Hudson
 See http://blog.jepamedia.org/2009/10/28/continuous-integration-for-php-with-hudson/
 See http://www.flickr.com/photos/sebastian_bergmann/sets/72157622541690849/
 Atlassian Bamboo
 See http://www.flickr.com/photos/sebastian_bergmann/sets/72157621193562929/

The End
Thank you for your interest!
These slides will be posted on
http://slideshare.net/sebastian_bergmann
I am co-authoring a book on
Quality Assurance in PHP Projects
http://phpqabook.com/

License
This presentation material is published under the Attribution-Share Alike 3.0 Unported
license.
You are free:
✔ to Share – to copy, distribute and transmit the work.
✔ to Remix – to adapt the work.
Under the following conditions:
● Attribution. You must attribute the work in the manner specified by the author or
licensor (but not in any way that suggests that they endorse you or your use of the
work).
● Share Alike. If you alter, transform, or build upon this work, you may distribute the
resulting work only under the same, similar or a compatible license.
For any reuse or distribution, you must make clear to others the license terms of this
work.
Any of the above conditions can be waived if you get permission from the copyright
holder.
Nothing in this license impairs or restricts the author's moral rights.