PHP Tester's Toolbox

The PHP Testers ToolboxSebastian Bergmann | ZendCon | October 19th 2011 sharing experience

SebastianBergmann

sharing experience

Software Testing Dynamic Testing Static Testing sharing experience

Perspective Black Box – Exploratory Testing – Model-Based Testing – Specification-Based Testing – Fuzz Testing White Box – API Testing – Code Coverage – Mutation Testing – Static Testing sharing experience

Scope Unit Testing Integration Testing System Testing sharing experience

Objective Acceptance Tests Regression Tests sharing experience

Non-Functional Aspects Compatibility Performance Security Stability Usability sharing experience

I may be biased.So take everything with a grain of salt! sharing experience

sharing experience

PHPT--TEST--Balance is initially zero--FILE--<?phprequire BankAccount.php;$ba = new BankAccount;var_dump($ba->getBalance());?>--EXPECT--int(0) sharing experience

PHPT➜ ~ php run-tests.php test.phpt=====================================================================PHP : /usr/local/php-5.4/bin/phpPHP_SAPI : cliPHP_VERSION : 5.4.0beta2-devZEND_VERSION: 2.4.0PHP_OS : Linux - Linux thinkpad 2.6.40.4-5.fc15.x86_64 #1 SMPINI actual : /usr/local/php-5.4/lib/php.iniMore .INIs :CWD : /home/sbExtra dirs :VALGRIND : Not used=====================================================================Running selected tests.PASS Balance is initially zero [test.phpt]=====================================================================Number of tests : 1 1Tests skipped : 0 ( 0.0%) --------Tests warned : 0 ( 0.0%) ( 0.0%)Tests failed : 0 ( 0.0%) ( 0.0%)Expected fail : 0 ( 0.0%) ( 0.0%)Tests passed : 1 (100.0%) (100.0%)---------------------------------------------------------------------Time taken : 1 seconds===================================================================== sharing experience

PHPT➜ ~ phpunit test.phptPHPUnit 3.6.0 by Sebastian Bergmann..Time: 0 seconds, Memory: 4.00MbOK (1 test, 1 assertion) sharing experience

SimpleTest<?phprequire_once BankAccount.php;require_once simpletest/autorun.php;class BankAccountTest extends UnitTestCase { public function testBalanceIsInitiallyZero() { $ba = new BankAccount; $this->assertEqual(0, $ba->getBalance()); }} sharing experience

SimpleTest➜ ~ php BankAccountTest.phpBankAccountTest.phpOKTest cases run: 1/1, Passes: 1, Failures: 0, Exceptions: 0 sharing experience

PHPUnit<?phprequire_once BankAccount.php;class BankAccountTest extends PHPUnit_Framework_TestCase { public function testBalanceIsInitiallyZero() { $ba = new BankAccount; $this->assertEquals(0, $ba->getBalance()); }} sharing experience

PHPUnit➜ ~ phpunit BankAccountTestPHPUnit 3.6.0 by Sebastian Bergmann..Time: 0 seconds, Memory: 4.25MbOK (1 test, 1 assertion) sharing experience

Atoum<?phpnamespace vendorprojecttestsunits;require_once BankAccount.php;require_once mageekguy.atoum.phar;use mageekguyatoum;use vendorproject;class BankAccount extends atoumtest { public function testBalanceIsInitiallyZero() { $ba = new projectBankAccount; $this->assert ->integer($ba->getBalance())->isEqualTo(0); }} sharing experience

Atoum➜ ~ php BankAccountTest.php> atoum version nightly-763-201109222317 by Frédéric Hardy> PHP path: /usr/local/php-5.4/bin/php> PHP version:=> PHP 5.4.0beta2-dev (cli) (built: Sep 27 2011 21:35:02) (DEBUG)=> Copyright (c) 1997-2011 The PHP Group=> Zend Engine v2.4.0, Copyright (c) 1998-2011 Zend Technologies=> with Xdebug v2.2.0-dev, Copyright (c) 2002-2011, by Derick Rethans> vendorprojecttestsunitsBankAccount...[S___________________________________________________________][1/1]=> Test duration: 0.00 second.=> Memory usage: 0.25 Mb.> Total test duration: 0.00 second.> Total test memory usage: 0.25 Mb.> Code coverage value: 11.11%=> Class vendorprojectBankAccount: 11.11%==> vendorprojectBankAccount::depositMoney(): 0.00%==> vendorprojectBankAccount::withdrawMoney(): 0.00%==> vendorprojectBankAccount::setBalance(): 0.00%> Running duration: 0.09 second.Success (1 test, 1 method, 2 assertions, 0 error, 0 exception) ! sharing experience

sharing experience

SimpleTest<?phpclass BankAccountControllerTest extends UnitTestCase { public function testReturnsBankAccountViewWhenBankAccountIsSpecified() { Mock::generate(BankAccountMapper); $mapper = new MockBankAccountMapper; $mapper->returns(findById, new BankAccount); }} sharing experience

PHPUnit<?phpclass BankAccountControllerTestextends PHPUnit_Framework_TestCase { public function testReturnsBankAccountViewWhenBankAccountIsSpecified() { $mapper = $this->getMock(BankAccountMapper); $mapper->expects($this->any()) ->method(findById) ->will($this->returnValue(new BankAccount)); // ... }} sharing experience

PHPUnit + Mockery<?phpclass BankAccountControllerTestextends PHPUnit_Framework_TestCase { public function testReturnsBankAccountViewWhenBankAccountIsSpecified() { $mapper = m::mock(BankAccountMapper); $mapper->shouldReceive(findById) ->andReturn(new BankAccount); // ... }} sharing experience

PHPUnit + Phake<?phpclass BankAccountControllerTestextends PHPUnit_Framework_TestCase { public function testReturnsBankAccountViewWhenBankAccountIsSpecified() { $mapper = Phake::mock(BankAccountMapper); Phake::when($mapper)->findById() ->thenReturn(new BankAccount); // ... }} sharing experience

Atoum<?phpnamespace vendorprojecttestsunits;use mageekguyatoum;use vendorproject;class BankAccountController extends atoumtest { public function testReturnsBankAccountViewWhenBankAccountIsSpecified() { $this->mockGenerator->generate(BankAccountMapper); $mapper = new mockBankAccountMapper; $mapper->getMockController() ->findById = new BankAccount; // ...} sharing experience

vfsStream<?phprequire_once vfsStream/vfsStream.php;require_once Example.php;class ExampleTest extends PHPUnit_Framework_TestCase { public function setUp() { vfsStreamWrapper::register(); vfsStreamWrapper::setRoot(new vfsStreamDirectory(exampleDir)); } public function testDirectoryIsCreated() { $example = new Example(id); $this->assertFalse(vfsStreamWrapper::getRoot()->hasChild(id)); $example->setDirectory(vfsStream::url(exampleDir)); $this->assertTrue(vfsStreamWrapper::getRoot()->hasChild(id)); }} sharing experience

vfsStream<?phpclass Example { protected $id; protected $directory; public function __construct($id) { $this->id = $id; } public function setDirectory($directory) { $this->directory = $directory . DIRECTORY_SEPARATOR . $this->id; if (!file_exists($this->directory)) { mkdir($this->directory, 0700, TRUE); } }} sharing experience

sharing experience

PHPUnit➜ ~ phpunit --testdox BankAccountTestPHPUnit 3.6.0 by Sebastian Bergmann.BankAccount [x] Balance is initially zero sharing experience

Behat# features/balance_is_initially_zero.featureFeature: The balance of a bank account is initially zeroScenario: Given I have a fresh bank account Then the balance should be 0 sharing experience

Behat➜ ~ behatFeature: The balance of a bank account is initially zero Scenario: Given I have a fresh bank account Then the balance should be 01 scenario (1 passed)2 steps (2 passed)0m0.034s sharing experience

Behat<?phpuse BehatBehatContextBehatContext;require_once PHPUnit/Autoload.php;require_once PHPUnit/Framework/Assert/Functions.php;require_once BankAccount.php;class FeatureContext extends BehatContext { private $ba; /** @Given /^I have a fresh bank account$/ */ public function freshBankAccount() { $this->ba = new BankAccount; } /** @Then /^the balance should be (d+)$/ */ public function balanceShouldBe($balance) { assertEquals($balance, $this->ba->getBalance()); }} sharing experience

PHPSpec<?phprequire BankAccount.php;class DescribeBankAccount extends PHPSpecContext { public function itsBalanceShouldInitiallyBeZero() { $ba = new BankAccount; $this->spec($ba->getBalance())->should->equal(0); }} sharing experience

PHPSpec➜ ~ phpspec ..Finished in 0.008273 seconds1 example sharing experience

PHPUnit_Story<?phprequire_once PHPUnit/Extensions/Story/TestCase.php;require_once BankAccount.php;class BankAccountSpecextends PHPUnit_Extensions_Story_TestCase { /** * @scenario */ public function balanceIsInitiallyZero() { $this->given(New bank account) ->then(Balance should be, 0); } // ...} sharing experience

PHPUnit_Story➜ ~ phpunit --printer PHPUnit_Extensions_Story_ResultPrinter_Text BankAccountSpecPHPUnit 3.6.0 by Sebastian Bergmann.BankAccount [x] Balance is initially zero Given New bank account Then Balance should be 0Scenarios: 1, Failed: 0, Skipped: 0, Incomplete: 0. sharing experience

PHPUnit_Storyclass BankAccountSpecextends PHPUnit_Extensions_Story_TestCase { // ... public function runGiven(&$world, $action, $arguments) { switch ($action) { case New bank account: { $world[bankaccount] = new BankAccount; } break; default: { return $this->notImplemented($action); } } } // ...} sharing experience

PHPUnit_Storyclass BankAccountSpecextends PHPUnit_Extensions_Story_TestCase { // ... public function runWhen(&$world, $action, $arguments) { return $this->notImplemented($action); } // ...} sharing experience

PHPUnit_Storyclass BankAccountSpecextends PHPUnit_Extensions_Story_TestCase { // ... public function runThen(&$world, $action, $arguments) { switch ($action) { case Balance should be: { $this->assertEquals( $arguments[0], $world[bankaccount]->getBalance() ); } break; default: { return $this->notImplemented($action); } } }} sharing experience

sharing experience

Behat + Mink + { Goutte | Sahi | Zombie.js }# features/search.featureFeature: Search In order to see a word definition As a website user I need to be able to search for a word Scenario: Searching for a page that does exist Given I am on "/wiki/Main_Page" When I fill in "search" with "Behavior Driven Development" And I press "searchButton" Then I should see "agile software development" Scenario: Searching for a page that does NOT exist Given I am on "/wiki/Main_Page" When I fill in "search" with "Glory Driven Development" And I press "searchButton" Then I should see "Search results" sharing experience

sharing experience

Selenium<tr> <td>open</td> <td>/domains/example/</td> <td></td></tr><tr> <td>assertTextPresent</td> <td>Example Domains</td> <td></td></tr> sharing experience

PHPUnit_Selenium<?phpclass WebTest extends PHPUnit_Extensions_SeleniumTestCase { protected function setUp() { $this->setBrowser(*firefox); $this->setBrowserUrl(http://www.iana.org/); } public function testTitle() { $this->open(/domains/example/); $this->assertTextPresent(Example Domains); }} sharing experience

sharing experience

Xdebug➜ bankaccount php -d xdebug.trace_format=1 -d xdebug.auto_trace=1 htdocs/index.phpThe balance of bank account #1 is 1.00. sharing experience

Xdebug➜ bankaccount php tracefile-analyser.php /tmp/trace.1388959365.xt time-own 10parsing...Done.Showing the 10 most costly calls sorted by time-own. Inclusive Ownfunction #calls time memory time memory--------------------------------------------------------------------------------------------------{closure:/usr/local/src/bankaccount/src/autoload.php:3-34} 15 0.0087 158328 0.0067 137800{main} 1 0.0149 186376 0.0014 20368PDO->query 1 0.0011 1320 0.0011 1320require 16 0.0018 21744 0.0007 3704PDO->__construct 1 0.0006 264 0.0006 264strtolower 15 0.0004 1520 0.0004 1520BankAccountMapper->findById 1 0.0023 12912 0.0003 -2760Router->route 1 0.0007 1272 0.0002 -1832FrontController->dispatch 1 0.0082 80496 0.0002 -2608HashMap->set 5 0.0002 3088 0.0002 3088 sharing experience

Xdebug➜ bankaccount php tracefile-analyser.php /tmp/trace.1388959365.xt time-inclusive 10parsing...Done.Showing the 10 most costly calls sorted by time-inclusive. Inclusive Ownfunction #calls time memory time memory--------------------------------------------------------------------------------------------------{main} 1 0.0149 186376 0.0014 20368{closure:/usr/local/src/bankaccount/src/autoload.php:3-34} 15 0.0087 158328 0.0067 137800FrontController->dispatch 1 0.0082 80496 0.0002 -2608ControllerFactory->getController 1 0.0036 56768 0.0001 -448BankAccountController->execute 1 0.0026 14344 0.0002 104MapperFactory->getMapper 1 0.0024 43240 0.0001 -488BankAccountMapper->findById 1 0.0023 12912 0.0003 -2760require 16 0.0018 21744 0.0007 3704PDO->query 1 0.0011 1320 0.0011 1320ViewFactory->getView 1 0.0010 10720 0.0001 -640 sharing experience

Xdebug➜ bankaccount php tracefile-analyser.php /tmp/trace.1388959365.xt memory-own 10parsing...Done.Showing the 10 most costly calls sorted by memory-own. Inclusive Ownfunction #calls time memory time memory--------------------------------------------------------------------------------------------------{closure:/usr/local/src/bankaccount/src/autoload.php:3-34} 15 0.0087 158328 0.0067 137800{main} 1 0.0149 186376 0.0014 20368require 16 0.0018 21744 0.0007 3704HashMap->set 5 0.0002 3088 0.0002 3088explode 1 0.0000 1792 0.0000 1792Request->__construct 1 0.0001 1696 0.0001 1696strtolower 15 0.0004 1520 0.0004 1520PDO->query 1 0.0011 1320 0.0011 1320IdentityMap->set 1 0.0001 1280 0.0001 1280sprintf 2 0.0001 1080 0.0001 1080 sharing experience

Xdebug➜ bankaccount php tracefile-analyser.php /tmp/trace.1388959365.xt memory-inclusive 10parsing...Done.Showing the 10 most costly calls sorted by memory-inclusive. Inclusive Ownfunction #calls time memory time memory--------------------------------------------------------------------------------------------------{main} 1 0.0149 186376 0.0014 20368{closure:/usr/local/src/bankaccount/src/autoload.php:3-34} 15 0.0087 158328 0.0067 137800FrontController->dispatch 1 0.0082 80496 0.0002 -2608ControllerFactory->getController 1 0.0036 56768 0.0001 -448MapperFactory->getMapper 1 0.0024 43240 0.0001 -488require 16 0.0018 21744 0.0007 3704BankAccountMapper->__construct 1 0.0009 15768 0.0001 -344BankAccountController->execute 1 0.0026 14344 0.0002 104BankAccountMapper->findById 1 0.0023 12912 0.0003 -2760ViewFactory->getView 1 0.0010 10720 0.0001 -640 sharing experience

Xdebug➜ bankaccount php tracefile-analyser.php /tmp/trace.1388959365.xt calls 10parsing...Done.Showing the 10 most costly calls sorted by calls. Inclusive Ownfunction #calls time memory time memory--------------------------------------------------------------------------------------------------require 16 0.0018 21744 0.0007 3704{closure:/usr/local/src/bankaccount/src/autoload.php:3-34} 15 0.0087 158328 0.0067 137800strtolower 15 0.0004 1520 0.0004 1520HashMap->set 5 0.0002 3088 0.0002 3088HashMap->get 3 0.0001 256 0.0001 256count 2 0.0001 0 0.0001 0sprintf 2 0.0001 1080 0.0001 1080IdentityMap->hasId 1 0.0000 224 0.0000 224ReflectionProperty->setAccessible 1 0.0000 0 0.0000 0ReflectionProperty->__construct 1 0.0000 632 0.0000 632 sharing experience

Xdebug + KCacheGrindxdebug.profiler_enable=1xdebug.profiler_enable_trigger=0xdebug.profiler_output_dir=/tmpxdebug.profiler_output_name=cachegrind.out.%pxdebug.profiler_append=0 sharing experience

KCacheGrind sharing experience

gprof2dot.py sharing experience

XHProf function.php<?php <?phprequire_once function.php; function foo() {xhprof_enable(); if (TRUE) { $foo = TRUE;foo(); } else { $foo = FALSE;$data = xhprof_disable(); }print_r($data); return NULL;?> }Array( [main()==>foo] => Array ( [ct] => 1 1 call from main() to foo() [wt] => 80 inclusive time in foo() when called from main() ) [main()==>xhprof_disable] => Array ( [ct] => 1 1 call from main() to xhprof_disable() [wt] => 4 inclusive time in xhprof_disable() when called from main() ) [main()] => Array fake symbol for main script entry ( [ct] => 1 [wt] => 140 inclusive time in main() )) sharing experience

XHProf function.php<?php <?phprequire_once function.php; function foo() {xhprof_enable(XHPROF_FLAGS_NO_BUILTINS); if (TRUE) { $foo = TRUE;foo(); } else { $foo = FALSE;$data = xhprof_disable(); }print_r($data); return NULL;?> }Array( [main()==>foo] => Array ( [ct] => 1 1 call from main() to foo() [wt] => 80 inclusive time in foo() when called from main() ) [main()] => Array fake symbol for main script entry ( [ct] => 1 [wt] => 140 inclusive time in main() )) sharing experience

XHProf function.php<?php <?phprequire_once function.php; function foo() {xhprof_enable(XHPROF_FLAGS_CPU + XHPROF_FLAGS_MEMORY); if (TRUE) { $foo = TRUE;foo(); } else { $foo = FALSE;$data = xhprof_disable(); }print_r($data); return NULL;?> }Array( [main()==>foo] => Array ( [ct] => 1 1 call from main() to foo() [wt] => 80 inclusive time in foo() when called from main() [cpu] => 0 cpu time in foo() when called from main() [mu] => 1480 change in PHP memory usage in bar() when called from foo() [pmu] => 0 change in PHP peak memory usage in bar() when called from foo() ) . . .) sharing experience

XHProf WebUI<?phprequire_once function.php;xhprof_enable(XHPROF_FLAGS_CPU + XHPROF_FLAGS_MEMORY);foo();$data = xhprof_disable();require_once xhprof_lib/utils/xhprof_lib.php;require_once xhprof_lib/utils/xhprof_runs.php;$run = new XHProfRuns_Default;$runId = $run->save_run($data, function);print <a href="http://localhost/xhprof_html/index.php?run= . $runId . &source=function">View profile</a>;?> sharing experience

XHProf + PHPUnit<listeners> <listener class="PHPUnit_Util_Log_XHProf" file="PHPUnit/Util/Log/XHProf.php"> <arguments> <array> <element key="xhprofLibFile"> <string>/var/www/xhprof_lib/utils/xhprof_lib.php</string> </element> <element key="xhprofRunsFile"> <string>/var/www/xhprof_lib/utils/xhprof_runs.php</string> </element> <element key="xhprofWeb"> <string>http://localhost/xhprof_html/index.php</string> </element> <element key="appNamespace"> <string>MyTestSuite</string> </element> <element key="xhprofFlags"> <string>XHPROF_FLAGS_CPU,XHPROF_FLAGS_MEMORY</string> </element> </array> </arguments> </listener></listeners> sharing experience

Load Testing, ... Pylot Grinder JMeter ... sharing experience

The End Web: http://thePHP.cc/ http://sebastian-bergmann.de/ Mail: sebastian@thePHP.cc Twitter: @s_bergmann Slides: http://talks.thePHP.cc/ sharing experience

http://paper.li	18
http://www.local.apec.fr	16
http://dionysus.uraganov.net	16
http://www.techgig.com	5
http://us-w1.rockmelt.com	4

PHP Tester's Toolbox

by Sebastian Bergmann on Oct 19, 2011

Accessibility

Categories

Tags

Upload Details

Usage Rights

5 Embeds 59

Statistics

PHP Tester’s Toolbox — Presentation Transcript