The State of QA Tools for PHP

The State of QA Tools for PHP Sebastian Bergmann August 22nd 2009

Sebastian Bergmann  Co-Founder and Principal Consultant with thePHP.cc  Creator of PHPUnit  Involved in the PHP project since 2000

This presentation is about tools that help you build better software.

Tools solve problems. Problems you should think about before it is too late.

When things go wrong in software projects, the team has to work overtime and cancel vacations.

More often than not, deadlines and quality goals are missed nevertheless.

You will ask yourself: ”Why did I not test this?”

Does my code work?  Be confident in your code!  Do not be afraid of changing your code!

Does my code work? Classic Approach  Write a test program  Run the test program  Manually verify the output  Delete the test program

Lets focus on a more modern approach.

Does my code work? Agile Approach Write a Unit Test  Executable specification  Automatic evaluation  Simple test environment  Instant feedback  Keep as regression test

Unit Tests improve the confidence in your code as they detect problems as early as possible.

phpunit  De-facto standard for the unit testing of PHP applications  Member of the xUnit family  Inspired by  JUnit  TestNG  JUnitour  JExample  …

phpunit Show Me The Code!

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { public function testBalanceIsInitiallyZero() { } }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { public function testBalanceIsInitiallyZero() { $ba = new BankAccount; $this->assertEquals(0, $ba->getBalance()); } }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { protected $ba; protected function setUp() { $this->ba = new BankAccount; } public function testBalanceIsInitiallyZero() { $this->assertEquals(0, $this->ba->getBalance()); } }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { public function testBalanceIsInitiallyZero() { $ba = new BankAccount; $this->assertEquals(0, $ba->getBalance()); return $ba; } /** * @depends testBalanceIsInitiallyZero * @expectedException RuntimeException */ public function testBalanceCannotBecomeNegative(BankAccount $ba) { $ba->withdrawMoney(1); } }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { public function testBalanceIsInitiallyZero() { $ba = new BankAccount; $this->assertEquals(0, $ba->getBalance()); return $ba; } /** * @depends testBalanceIsInitiallyZero * @expectedException RuntimeException */ public function testBalanceCannotBecomeNegative(BankAccount $ba) { $ba->withdrawMoney(1); } /** * @depends testBalanceIsInitiallyZero * @expectedException RuntimeException */ public function testBalanceCannotBecomeNegative2(BankAccount $ba) { $ba->depositMoney(-1); } // ... }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { // ... /** * @depends testBalanceIsInitiallyZero */ public function testDepositingMoneyWorks(BankAccount $ba) { $ba->depositMoney(1); $this->assertEquals(1, $ba->getBalance()); return $ba; } }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { // ... /** * @depends testBalanceIsInitiallyZero */ public function testDepositingMoneyWorks(BankAccount $ba) { $ba->depositMoney(1); $this->assertEquals(1, $ba->getBalance()); return $ba; } /** * @depends testDepositingMoneyWorks */ public function testWithdrawingMoneyWorks(BankAccount $ba) { $ba->withdrawMoney(1); $this->assertEquals(0, $ba->getBalance()); } }

phpunit sb@thinkpad ~ % phpunit BankAccountTest PHPUnit 3.4.0 by Sebastian Bergmann. ..... Time: 0 seconds OK (5 tests, 5 assertions)

phpunit Tests can serve as an executable specification.

phpunit Tests can serve as an executable specification. sb@thinkpad ~ % phpunit --testdox BankAccountTest PHPUnit 3.4.0 by Sebastian Bergmann. BankAccount [x] Balance is initially zero [x] Balance cannot become negative [x] Depositing money works [x] Withdrawing money works

phpunit Lets break something … <?php class BankAccount { protected $balance = 1; public function getBalance() { return $this->balance; } protected function setBalance($balance) { if ($balance >= 0) { $this->balance = $balance; } else { throw new RuntimeException; } } public function depositMoney($balance) { $this->setBalance($this->getBalance() + $balance); } public function withdrawMoney($balance) { $this->setBalance($this->getBalance() - $balance); } } ?>

phpunit … and see how tests fail. sb@thinkpad ~ % phpunit --verbose BankAccountTest PHPUnit 3.4.0 by Sebastian Bergmann. FSSSS Time: 0 seconds There was 1 failure: 1) BankAccountTest::testBalanceIsInitiallyZero Failed asserting that <integer:1> matches expected <integer:0>. /home/sb/BankAccountTest.php:7 There were 4 skipped tests: 1) BankAccountTest::testBalanceCannotBecomeNegative This test depends on "BankAccountTest::testBalanceIsInitiallyZero" to pass. 2) BankAccountTest::testBalanceCannotBecomeNegative2 This test depends on "BankAccountTest::testBalanceIsInitiallyZero" to pass. 3) BankAccountTest::testDepositingMoneyWorks This test depends on "BankAccountTest::testBalanceIsInitiallyZero" to pass. 4) BankAccountTest::testWithdrawingMoneyWorks This test depends on "BankAccountTest::testDepositingMoneyWorks" to pass. FAILURES! Tests: 1, Assertions: 1, Failures: 1, Skipped: 4.

A unit test should run in less than 1 ms.

Software Testing Categorization  Small: Unit Tests  Check conditional logic in the code  A debugger should not be required in case of failure  Runs in less than 1 ms  Medium: Functional Tests  Check whether the interfaces between classes abide by their contracts  Large: End-to-End Tests  Check for ”wiring bugs” This slide contains material by Miško Hevery

Software Metrics Measuring Software and Software Quality „You cannot control what you cannot measure.“ – Tom DeMarco

Software Metrics Code Coverage  Which statements, branches, and paths are executed when the tests run?  Statement Coverage  Branch Coverage  Path Coverage  100% Code Coverage is a required, but not a sufficient criteria for test completeness

Software Metrics Code Coverage sb@thinkpad ~ % phpunit --coverage-html /tmp/report BankAccountTest PHPUnit 3.4.0 by Sebastian Bergmann. ..... Time: 0 seconds OK (5 tests, 5 assertions) Generating code coverage report, this may take a moment.

Software Metrics Code Analysis

Software Metrics Lines of Code  Text-based metric for code size  Various definitions  Lines of Code (LOC)  Comment Lines of Code (CLOC)  Non-Comment Lines of Code (NCLOC)  Executable Lines of Code (ELOC)  Ratios can be interesting  CLOC / (E)LOC

Software Metrics Lines of Code sb@thinkpad ~ % phploc /usr/local/src/ezcomponents/trunk/Workflow/src phploc 1.2.0 by Sebastian Bergmann. Directories: 11 Files: 82 Lines of Code (LOC): 9576 Executable Lines of Code (ELOC): 2962 Comment Lines of Code (CLOC): 4923 Non-Comment Lines of Code (NCLOC): 4653 Interfaces: 7 Abstract Classes: 12 Classes: 62 Non-Static Methods: 274 Static Methods: 22 Functions: 0 Constants: 0 Class constants: 10

Software Metrics Code Duplication  Two sequences of code are duplicate when they are  textually identical  token for token identical  functionally identical  Problems  Duplicate code contradicts code reuse  Co-Evolution of clones hinders maintenance

Software Metrics Code Duplication sb@thinkpad ~ % phpcpd /usr/local/src/phpunit/trunk/PHPUnit phpcpd 1.1.0 by Sebastian Bergmann. Found 4 exact clones with 131 duplicated lines in 7 files: - Extensions/Database/DataSet/AbstractTable.php:156-190 Extensions/Database/DataSet/ReplacementTable.php:172-206 - Samples/BankAccountDB/BankAccountDBTest.php:84-128 Samples/BankAccountDB/BankAccountDBTestMySQL.php:84-128 - Tests/Extensions/Database/DataSet/XmlDataSetsTest.php:71-98 Tests/Extensions/Database/DataSet/YamlDataSetTest.php:70-97 - Tests/Extensions/Database/DataSet/XmlDataSetsTest.php:71-97 Tests/Extensions/Database/DataSet/CsvDataSetTest.php:70-96 0.21% duplicated lines out of 61720 total lines of code.

Software Metrics Code Complexity  Cyclomatic Complexity  Counts the number of branching points if, for, foreach, while, case, catch, &&, ||, ternary operator (?:)  NPath Complexity  Counts the number execution paths  Interpretation  Higher complexity leads to more errors  Higher complexity makes testing harder

Software Metrics pdepend  Static Analysis of PHP Code  Software Metrics  Software Visualization  Helps to identify parts of an application that should be refactored

Software Metrics phpmd  Static Analysis of PHP Code  Based on pdepend  Reports violations of rules that operate on raw software metrics data

Software Metrics phpcs  Static Analysis of PHP Code  Based on ext/tokenizer  ”Sniffs”  Coding Standard  Software Metrics  Bug Patterns  Performance Patterns  ...

Software Metrics bytekit-cli  Static Analysis of PHP Code  Based on ext/bytekit  Disassembles PHP bytecode  Visualizes PHP bytecode  Scans PHP bytecode  … for disallowed opcode sequences  … for direct output of variables  …

Software Metrics bytekit-cli 1 <?php 2 if (TRUE) { 3 print '*'; 4 } 5 ?> sb@thinkpad ~ % bytekit if.php bytekit-cli 1.0.0 by Sebastian Bergmann. Filename: /home/sb/if.php Function: main Number of oplines: 8 line # opcode result operands ----------------------------------------------------------------------------- 2 0 EXT_STMT 1 JMPZ true, ->6 3 2 EXT_STMT 3 PRINT ~0 '*' 4 FREE ~0 4 5 JMP ->6 6 6 EXT_STMT 7 RETURN 1

Software Metrics bytekit-cli 1 <?php 2 if (TRUE) { 3 print '*'; 4 } 5 ?> sb@thinkpad ~ % bytekit --graph /tmp --format svg if.php

Software Metrics bytekit-cli sb@thinkpad ~ % bytekit --rule DisallowedOpcodes:EVAL /usr/local/src/phpunit/trunk/PHPUnit bytekit-cli 1.0.0 by Sebastian Bergmann. - Disallowed opcode "EVAL" in /usr/local/src/phpunit/trunk/PHPUnit/Framework/TestCase.php:1160 - Disallowed opcode "EVAL" in /usr/local/src/phpunit/trunk/PHPUnit/Framework/TestCase.php:1061 - Disallowed opcode "EVAL" in /usr/local/src/phpunit/trunk/PHPUnit/Extensions/PhptTestCase.php:223 - Disallowed opcode "EVAL" in /usr/local/src/phpunit/trunk/PHPUnit/TextUI/Command.php:169

QA Tools for PHP Overview

Build Automation Automate {errorprone|recurring|tedious} tasks

Build Automation Apache Ant  Java-based build tool  Kind of like make, without make's wrinkles  Build files are XML-based, calling out a target tree where various tasks get executed

Build Automation Apache Ant <project name="Money" default="build"> <target name="clean"> <delete dir="${basedir}/build"/> </target> <target name="prepare"> <mkdir dir="${basedir}/build/logs"/> </target> <target name="phpcs"> <exec dir="${basedir}" executable="phpcs" output="${basedir}/build/logs/checkstyle.xml" failonerror="false"> <arg line="--report=checkstyle ."/> </exec> </target> <target name="phpmd"> <exec dir="${basedir}" executable="phpmd" failonerror="false"> <arg line=". xml codesize --reportfile ${basedir}/build/logs/pmd.xml"/> </exec> </target>

Build Automation Apache Ant <target name="phpcpd"> <exec dir="${basedir}" executable="phpcpd" failonerror="false"> <arg line="--log-pmd=${basedir}/build/logs/pmd-cpd.xml ."/> </exec> </target> <target name="pdepend"> <exec dir="${basedir}" executable="pdepend" failonerror="false"> <arg line="--jdepend-xml=${basedir}/build/logs/jdepend.xml ."/> </exec> </target> <target name="phpunit"> <exec dir="${basedir}" executable="phpunit" failonerror="true"> <arg line="--log-xml ${basedir}/build/logs/junit.xml --coverage-clover ${basedir}/build/logs/clover.xml MoneyTest"/> </exec> </target> <target name="build" depends="clean,prepare,phpcs,phpmd,phpcpd,pdepend,phpunit"/> </project>

Build Automation Apache Ant sb@ubuntu Money % ant Buildfile: build.xml clean: [delete] Deleting directory /home/sb/Money/build prepare: [mkdir] Created dir: /home/sb/Money/build/logs phpcs: phpmd: phpcpd: [exec] phpcpd 1.1.0 by Sebastian Bergmann. [exec] [exec] 0.00% duplicated lines out of 722 total lines of code. pdepend: [exec] PHP_Depend 0.9.4 by Manuel Pichler [exec] [exec] Executing Dependency-Analyzer: [exec] 16 [exec] [exec] Generating pdepend log files, this may take a moment. phpunit: [exec] PHPUnit 3.4.0 by Sebastian Bergmann. [exec] [exec] ...................... [exec] [exec] Time: 0 seconds [exec] [exec] OK (22 tests, 34 assertions) [exec] [exec] Writing code coverage data to XML file, this may take a moment. build: BUILD SUCCESSFUL Total time: 4 seconds

Continuous Integration Feel the pulse of your project!

Continuous Integration Software development practice where members of a team integrate their work frequently  Usually each person integrates at least daily, leading to multiple integrations per day  Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible  Leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly

Continuous Integration CruiseControl Framework for a continuous build process  Includes, but is not limited to, plugins for email notification, Apache Ant, Phing, and various source control tools  A web interface is provided to view the details of the current and previous builds

Continuous Integration phpUnderControl Customization of CruiseControl that caters to the needs of PHP projects  PHPUnit  PHPDocumentor  PHP_CodeSniffer  (PHP_Depend)  (phpmd)  (phpcpd)

The End Thank you for your interest! These slides will be posted on http://slideshare.net/sebastian_bergmann I am co-authoring a book on Quality Assurance in PHP Projects http://phpqabook.com/

License This presentation material is published under the Attribution-Share Alike 3.0 Unported license. You are free: ✔ to Share – to copy, distribute and transmit the work. ✔ to Remix – to adapt the work. Under the following conditions: ● Attribution. You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). ● Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.

http://qualityassuranceinphpprojects.com	11029
http://www.phphatesme.com	62
http://jujo00obo2o234ungd3t8qjfcjrs3o6k-a-sites-opensocial.googleusercontent.com	52
http://hinikato.blogspot.com	28
http://www.slideshare.net	22
http://translate.googleusercontent.com	17
http://webcache.googleusercontent.com	8
http://www.estudios-aqua.com	6
http://localhost	4
http://sanjaysurani.blogspot.com	1
http://fof-dev	1
http://static.slidesharecdn.com	1
http://gdp.axiome.fr	1
file://	1
http://livebuzz.com.br	1

The State of QA Tools for PHP

by Sebastian Bergmann on Aug 22, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

15 Embeds 11,234

Statistics

The State of QA Tools for PHP — Presentation Transcript