The State of Quality Assurance Tools for PHP

The State of Quality Assurance Tools for PHP Sebastian Bergmann November 18th 2009

Sebastian Bergmann  Co-Founder and Principal Consultant with thePHP.cc  Creator of PHPUnit  Involved in the PHP project since 2000

This presentation is about tools that help you build better software.

Tools solve problems. Problems you should think about before it is too late.

When things go wrong in software projects, the team has to work overtime and cancel vacations.

More often than not, deadlines and quality goals are missed nevertheless.

You will ask yourself: ”Why did I not test this?”

Does my code work?  Be confident in your code!  Do not be afraid of changing your code!

Does my code work? Classic Approach  Write a test program  Run the test program  Manually verify the output  Delete the test program

Lets focus on a more modern approach.

Does my code work? Agile Approach Write a Unit Test  Executable specification  Automatic evaluation  Simple test environment  Instant feedback  Keep as regression test

Unit Tests improve the confidence in your code as they detect problems as early as possible.

phpunit  De-facto standard for the unit testing of PHP applications  Member of the xUnit family  Inspired by  JUnit  TestNG  JUnitour  JExample  …

phpunit Show Me The Code!

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { public function testBalanceIsInitiallyZero() { } }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { public function testBalanceIsInitiallyZero() { $ba = new BankAccount; $this->assertEquals(0, $ba->getBalance()); } }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { protected $ba; protected function setUp() { $this->ba = new BankAccount; } public function testBalanceIsInitiallyZero() { $this->assertEquals(0, $this->ba->getBalance()); } }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { public function testBalanceIsInitiallyZero() { $ba = new BankAccount; $this->assertEquals(0, $ba->getBalance()); return $ba; } /** * @depends testBalanceIsInitiallyZero * @expectedException RuntimeException */ public function testBalanceCannotBecomeNegative(BankAccount $ba) { $ba->withdrawMoney(1); } }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { public function testBalanceIsInitiallyZero() { $ba = new BankAccount; $this->assertEquals(0, $ba->getBalance()); return $ba; } /** * @depends testBalanceIsInitiallyZero * @expectedException RuntimeException */ public function testBalanceCannotBecomeNegative(BankAccount $ba) { $ba->withdrawMoney(1); } /** * @depends testBalanceIsInitiallyZero * @expectedException RuntimeException */ public function testBalanceCannotBecomeNegative2(BankAccount $ba) { $ba->depositMoney(-1); } // ... }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { // ... /** * @depends testBalanceIsInitiallyZero */ public function testDepositingMoneyWorks(BankAccount $ba) { $ba->depositMoney(1); $this->assertEquals(1, $ba->getBalance()); return $ba; } }

phpunit <?php require_once 'BankAccount.php'; class BankAccountTest extends PHPUnit_Framework_TestCase { // ... /** * @depends testBalanceIsInitiallyZero */ public function testDepositingMoneyWorks(BankAccount $ba) { $ba->depositMoney(1); $this->assertEquals(1, $ba->getBalance()); return $ba; } /** * @depends testDepositingMoneyWorks */ public function testWithdrawingMoneyWorks(BankAccount $ba) { $ba->withdrawMoney(1); $this->assertEquals(0, $ba->getBalance()); } }

phpunit sb@thinkpad ~ % phpunit BankAccountTest PHPUnit 3.4.2 by Sebastian Bergmann. ..... Time: 0 seconds OK (5 tests, 5 assertions)

phpunit Tests can serve as an executable specification.

phpunit Tests can serve as an executable specification. sb@thinkpad ~ % phpunit --testdox BankAccountTest PHPUnit 3.4.2 by Sebastian Bergmann. BankAccount [x] Balance is initially zero [x] Balance cannot become negative [x] Depositing money works [x] Withdrawing money works

phpunit Lets break something … <?php class BankAccount { protected $balance = 1; public function getBalance() { return $this->balance; } protected function setBalance($balance) { if ($balance >= 0) { $this->balance = $balance; } else { throw new RuntimeException; } } public function depositMoney($balance) { $this->setBalance($this->getBalance() + $balance); } public function withdrawMoney($balance) { $this->setBalance($this->getBalance() - $balance); } } ?>

phpunit … and see how tests fail. sb@thinkpad ~ % phpunit --verbose BankAccountTest PHPUnit 3.4.2 by Sebastian Bergmann. FSSSS Time: 0 seconds There was 1 failure: 1) BankAccountTest::testBalanceIsInitiallyZero Failed asserting that <integer:1> matches expected <integer:0>. /home/sb/BankAccountTest.php:7 There were 4 skipped tests: 1) BankAccountTest::testBalanceCannotBecomeNegative This test depends on "BankAccountTest::testBalanceIsInitiallyZero" to pass. 2) BankAccountTest::testBalanceCannotBecomeNegative2 This test depends on "BankAccountTest::testBalanceIsInitiallyZero" to pass. 3) BankAccountTest::testDepositingMoneyWorks This test depends on "BankAccountTest::testBalanceIsInitiallyZero" to pass. 4) BankAccountTest::testWithdrawingMoneyWorks This test depends on "BankAccountTest::testDepositingMoneyWorks" to pass. FAILURES! Tests: 1, Assertions: 1, Failures: 1, Skipped: 4.

A unit test should run in less than 1 ms.

Software Testing Categorization  Small: Unit Tests  Check conditional logic in the code  A debugger should not be required in case of failure  Runs in less than 1 ms  Medium: Functional Tests  Check whether the interfaces between classes abide by their contracts  Large: End-to-End Tests  Check for ”wiring bugs” This slide contains material by Miško Hevery

Software Metrics Measuring Software and Software Quality „You cannot control what you cannot measure.“ – Tom DeMarco

Software Metrics Code Coverage  Which statements, branches, and paths are executed when the tests run?  Statement Coverage  Branch Coverage  Path Coverage  100% Code Coverage is a required, but not a sufficient criteria for test completeness

Software Metrics Code Coverage sb@thinkpad ~ % phpunit --coverage-html /tmp/report BankAccountTest PHPUnit 3.4.2 by Sebastian Bergmann. ..... Time: 0 seconds OK (5 tests, 5 assertions) Generating code coverage report, this may take a moment.

Software Metrics Code Analysis

Software Metrics Lines of Code  Text-based metric for code size  Various definitions  Lines of Code (LOC)  Comment Lines of Code (CLOC)  Non-Comment Lines of Code (NCLOC)  Executable Lines of Code (ELOC)  Ratios can be interesting  CLOC / (E)LOC

Software Metrics Lines of Code sb@thinkpad ~ % phploc --count-tests /usr/local/src/php-object-freezer phploc 1.3.2 by Sebastian Bergmann. Directories: 10 Files: 30 Lines of Code (LOC): 4032 Cyclomatic Complexity / Lines of Code: 0.06 Executable Lines of Code (ELOC): 1218 Comment Lines of Code (CLOC): 2007 Non-Comment Lines of Code (NCLOC): 2025 Interfaces: 2 Classes: 19 Abstract: 1 (5.26%) Concrete: 18 (94.74%) Lines of Code / Number of Classes: 153 Methods: 51 Scope: Non-Static: 48 (94.12%) Static: 3 (5.88%) Visibility: Public: 35 (68.63%) Non-Public: 16 (31.37%) Lines of Code / Number of Methods: 57 Cyclomatic Complexity / Number of Methods: 2.41 Functions: 0 Constants: 1 Global constants: 0 Class constants: 1 Tests: Classes: 9 Methods: 76

Software Metrics Code Duplication  Two sequences of code are duplicate when they are  textually identical  token for token identical  functionally identical  Problems  Duplicate code contradicts code reuse  Co-Evolution of clones hinders maintenance

Software Metrics Code Duplication sb@thinkpad ~ % phpcpd /usr/local/src/phpunit/trunk/PHPUnit phpcpd 1.2.2 by Sebastian Bergmann. Found 4 exact clones with 131 duplicated lines in 7 files: - Extensions/Database/DataSet/AbstractTable.php:156-190 Extensions/Database/DataSet/ReplacementTable.php:172-206 - Samples/BankAccountDB/BankAccountDBTest.php:84-128 Samples/BankAccountDB/BankAccountDBTestMySQL.php:84-128 - Tests/Extensions/Database/DataSet/XmlDataSetsTest.php:71-98 Tests/Extensions/Database/DataSet/YamlDataSetTest.php:70-97 - Tests/Extensions/Database/DataSet/XmlDataSetsTest.php:71-97 Tests/Extensions/Database/DataSet/CsvDataSetTest.php:70-96 0.21% duplicated lines out of 61720 total lines of code.

Software Metrics Code Complexity  Cyclomatic Complexity  Counts the number of branching points if, for, foreach, while, case, catch, &&, ||, ternary operator (?:)  NPath Complexity  Counts the number execution paths  Interpretation  Higher complexity leads to more errors  Higher complexity makes testing harder

Software Metrics pdepend  Static Analysis of PHP Code  Software Metrics  Software Visualization  Helps to identify parts of an application that should be refactored

Software Metrics phpmd  Static Analysis of PHP Code  Based on pdepend  Reports violations of rules that operate on raw software metrics data

Software Metrics phpcs  Static Analysis of PHP Code  Based on ext/tokenizer  ”Sniffs”  Coding Standard  Software Metrics  Bug Patterns  Performance Patterns  ...

Software Metrics bytekit-cli  Static Analysis of PHP Code  Based on ext/bytekit  Disassembles PHP bytecode  Visualizes PHP bytecode  Scans PHP bytecode  … for disallowed opcode sequences  … for direct output of variables  …

Software Metrics bytekit-cli 1 <?php 2 if (TRUE) { 3 print '*'; 4 } 5 ?> sb@thinkpad ~ % bytekit if.php bytekit-cli 1.1.1 by Sebastian Bergmann. Filename: /home/sb/if.php Function: main Number of oplines: 8 line # opcode result operands ----------------------------------------------------------------------------- 2 0 EXT_STMT 1 JMPZ true, ->6 3 2 EXT_STMT 3 PRINT ~0 '*' 4 FREE ~0 4 5 JMP ->6 6 6 EXT_STMT 7 RETURN 1

Software Metrics bytekit-cli 1 <?php 2 if (TRUE) { 3 print '*'; 4 } 5 ?> sb@thinkpad ~ % bytekit --graph /tmp --format svg if.php

Software Metrics bytekit-cli sb@thinkpad ~ % bytekit --rule DisallowedOpcodes:EVAL /usr/local/src/phpunit/trunk/PHPUnit bytekit-cli 1.1.1 by Sebastian Bergmann. - Disallowed opcode "EVAL" in /usr/local/src/phpunit/trunk/PHPUnit/Framework/TestCase.php:1160 - Disallowed opcode "EVAL" in /usr/local/src/phpunit/trunk/PHPUnit/Framework/TestCase.php:1061 - Disallowed opcode "EVAL" in /usr/local/src/phpunit/trunk/PHPUnit/Extensions/PhptTestCase.php:223 - Disallowed opcode "EVAL" in /usr/local/src/phpunit/trunk/PHPUnit/TextUI/Command.php:169

QA Tools for PHP Overview http://phpqatools.org/

The End Thank you for your interest! These slides will be posted on http://slideshare.net/sebastian_bergmann I am co-authoring a book on Quality Assurance in PHP Projects http://phpqabook.com/

License This presentation material is published under the Attribution-Share Alike 3.0 Unported license. You are free: ✔ to Share – to copy, distribute and transmit the work. ✔ to Remix – to adapt the work. Under the following conditions: ● Attribution. You must attribute the work in the manner specified by the author or licensor (but not in any way that suggests that they endorse you or your use of the work). ● Share Alike. If you alter, transform, or build upon this work, you may distribute the resulting work only under the same, similar or a compatible license. For any reuse or distribution, you must make clear to others the license terms of this work. Any of the above conditions can be waived if you get permission from the copyright holder. Nothing in this license impairs or restricts the author's moral rights.

http://www.mostofreddy.com.ar	175
http://www.3wstudio.com.ar	32
http://www.slideshare.net	5
url_unknown	4
http://www.thehouseofblogs.com	1
http://www.brijj.com	1
http://wildfire.gigya.com	1
http://www.lmodules.com	1
https://mycourses.purduecal.edu	1

The State of Quality Assurance Tools for PHP

by Sebastian Bergmann on Nov 18, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

9 Embeds 221

Statistics

The State of Quality Assurance Tools for PHP — Presentation Transcript