Testing with PHPUnit and Selenium

Loading...

Flash Player 9 (or above) is needed to view presentations.
We have detected that you do not have it on your computer. To install it, go here.

1 comments

Comments 1 - 1 of 1 previous next Post a comment

Post a comment
Embed Video
Edit your comment Cancel

Notes on slide 1

Theme created by Sakari Koivunen and Henrik Omma Released under the LGPL license.

10 Favorites

Testing with PHPUnit and Selenium - Presentation Transcript

  1. Welcome! Testing with PHPUnit and Selenium Sebastian Bergmann http://sebastian-bergmann.de/ September 12 th 2007
  2. Who I am
    • Sebastian Bergmann
    • Computer Scientist
    • Involved in the PHP Project since 2000
    • Author, Consultant, Coach, Trainer
    • Developer for eZ Systems AS
  3. Who are you?
    • Your experience with
      • PHP 5?
      • OOP?
      • Testing?
        • PHPUnit?
        • Selenium?
      • Software Metrics?
  4. Why test?
    • Companies develop more and more enterprise-critical applications with PHP.
    • Tests help to make sure that these applications work correctly.
  5. Debugging Sucks, Testing Rocks! Debugging Sucks! Testing Rocks!
  6. What needs testing?
    • Web Application
      • Backend
        • Business Logic
        • Reusable Components
      • Frontend
        • Form Processing, Templates, ...
        • “Rich Interfaces” with AJAX, JSON, ...
        • Feeds, Web Services, ...
  7. And how do we test it?
    • Web Application
      • Backend
        • Functional Testing of the business logic with Unit Tests
        • Reusable Components
          • External components should have their own unit tests.
      • Frontend
        • Acceptance Tests or System Tests that are run “in the browser”
        • Testing of feeds, web services, etc. with unit tests
        • Compatibility Tests for Operating System / Browser combinations
        • Performance Tests and Security Tests
  8. Testing Software
    • Component Tests
      • Executable code fragments, so called Unit Tests , test the correctness of parts, units , of the software ( system under test )
    • System Tests
      • Conducted on a complete, integrated system to evaluate the system's compliance with its specified requirements. (Wikipedia)
    • Non-Functional Tests
      • Performance, Stability, Security, ...
  9. Testing Software
    • Developer Tests
      • Ensure that the code works correctly
    • Acceptance Tests
      • Ensure that the code does what the customer wants
  10. How do you test code? <?php class Calculator { public function add ( $a , $b ) { return $a + $b ; } } ?>
  11. How do you test code? <?php $fixture = new Calculator ; print $fixture -> add ( 0 , 1 ); $fixture = new Calculator ; print $fixture -> add ( 1 , 0 ); ?> 1 1
    • The tests are not specified in the code.
    • The tests are not automatically evaluated.
  12. How do you test code?
    • The tests are specified in the code.
    • The tests are automatically evaluated.
    • The writing of tests is tedious.
    <?php $fixture = new Calculator ; print $fixture -> add ( 0 , 1 ) == 1 ? 'pass' : 'fail' ; $fixture = new Calculator ; print $fixture -> add ( 1 , 0 ) == 1 ? 'pass' : 'fail' ; ?> pass pass
  13. How do you test code?
    • The tests are automatically evaluated.
    • assertTrue() makes the writing of tests easy.
    • The test environment is not reusable.
    <?php $fixture = new Calculator ; assertTrue ( $fixture -> add ( 0 , 1 ) == 1 ); $fixture = new Calculator ; assertTrue ( $fixture -> add ( 1 , 0 ) == 1 ); function assertTrue ( $expression ) { if (! $expression ) { throw new Exception ( 'Assertion failed.' ); } } ?>
  14. Test Tools
    • To make (code) testing viable, good tool support is needed.
    • This is where a testing framework such as PHPUnit comes into play.
    • Requirements
      • Reusable test environment
      • Strict separation of production code and test code
      • Automatic execution of test code
      • Analysis of the result
      • Easy to learn to use and easy to use
  15. Test Tools for PHP / Web
    • Unit Tests
      • PHPUnit
      • SimpleTest
    • System Tests
      • Selenium
        • PHPUnit + Selenium
    • Non-Functional Tests
      • Performance, Load, Stress, Availability, ...
        • ab, httperf, JMeter, Grinder, OpenSTA, ...
      • Security
        • Chorizo
  16. PHPUnit: Webseite http://www.phpunit.de/
  17. PHPUnit: Installation [email_address] ~ % pear channel-discover pear.phpunit.de Adding Channel &quot;pear.phpunit.de&quot; succeeded Discovery of channel &quot;pear.phpunit.de&quot; succeeded [email_address] ~ % pear install phpunit/phpunit Did not download optional dependencies: pear/Log, use --alldeps to download automatically phpunit/PHPUnit can optionally use package &quot;pear/Log&quot; downloading PHPUnit-3.1.8.tgz ... Starting to download PHPUnit-3.1.8.tgz (116,960 bytes) .........................done: 116,960 bytes install ok: channel://pear.phpunit.de/PHPUnit-3.1.8
  18. PHPUnit: TextUI Testrunner [email_address] ~ % phpunit --help PHPUnit 3.2.0 by Sebastian Bergmann. Usage: phpunit [switches] UnitTest [UnitTest.php] . . .
  19. Writing tests for PHPUnit <?php require_once 'PHPUnit/Framework.php' ; require_once 'Calculator.php' ; class CalculatorTest extends PHPUnit_Framework_TestCase { public function testAdd () { $calculator = new Calculator ; $this -> assertEquals ( 0 , $calculator -> add ( 0 , 0 )); $this -> assertEquals ( 1 , $calculator -> add ( 0 , 1 )); $this -> assertEquals ( 1 , $calculator -> add ( 1 , 0 )); $this -> assertEquals ( 2 , $calculator -> add ( 1 , 1 )); $this -> assertNotEquals ( 3 , $calculator -> add ( 1 , 2 )); } } ?>
  20. Running tests with PHPUnit [email_address] ~ % phpunit CalculatorTest PHPUnit 3.2.0 by Sebastian Bergmann. F Time: 0 seconds There was 1 failure: 1) testAdd(CalculatorTest) Failed asserting that <integer:3> is not equal to <integer:3>. /home/sb/CalculatorTest.php:15 FAILURES! Tests: 1, Failures: 1.
  21. Test Fixture
    • One of the most time-consuming parts of writing tests is writing the code to set the world up in a known state and then return it to its original state when the test is complete.
    • This known state is called the fixture of the test.
  22. Test Fixture
    • In our CalculatorTest example the test fixture was a single object.
    • Most of the time, though, the fixture will be more complex than a simple object, and the amount of code needed to set it up will grow accordingly.
    • The actual content of the test gets lost in the noise of setting up the fixture.
      • This problem gets even worse when you write several tests with similar fixtures.
    • PHPUnit supports sharing the setup code through the setUp() and tearDown() template methods.
  23. Writing tests for PHPUnit <?php require_once 'PHPUnit/Framework.php' ; require_once 'Calculator.php' ; class CalculatorTest extends PHPUnit_Framework_TestCase { protected $calculator ; protected function setUp () { $this -> calculator = new Calculator ; } public function testAdd () { $this -> assertEquals ( 0 , $this -> calculator -> add ( 0 , 0 )); } // ... ?>
  24. Running tests with PHPUnit [email_address] ~ % phpunit CalculatorTest PHPUnit 3.2.0 by Sebastian Bergmann. ....F Time: 0 seconds There was 1 failure: 1) testAdd5(CalculatorTest) Failed asserting that <integer:3> is not equal to <integer:3>. /home/sb/CalculatorTest.php:26 FAILURES! Tests: 5, Failures: 1.
  25. Philosophy
    • Without Tests
      • The developer thinks „the feature works“ ⇒ it works
    • With Tests
      • No tests for the feature ⇒ it does not work
  26. PHPUnit API
    • For most use cases, PHPUnit has a very simple API
      • Extend PHPUnit_Framework_TestCase for your test case classes and
      • call assertTrue() or assertEquals() in your test methods
  27. PHPUnit API
  28. PHPUnit API
    • PHPUnit_Framework_Assert
      • Collection of assertion methods
    • PHPUnit_Framework_Test
      • Interface for all test objects
    • PHPUnit_Framework_TestCase
      • Abstract base class for test case classes
      • Child classes declare arbitrary number of tests
      • Template methods setUp() and tearDown()
  29. PHPUnit API
    • PHPUnit_Framework_TestSuite
      • Collection of tests
      • Contains PHPUnit_Framework_Test objects, for instance PHPUnit_Framework_TestCase and PHPUnit_Framework_TestSuite objects
    • PHPUnit_Framework_TestResult
      • Collects test result information
  30. Organizing Test Suites
    • Application/
      • Package/
        • Class (Application/Package/Class.php)
        • ...
      • ...
      • Tests/
        • AllTests.php
        • Package/
          • AllTests.php
          • ClassTest (Application/Tests/Package/ClassTest.php)
  31. Organizing Test Suites <?php require_once 'PHPUnit/Framework.php' ; require_once 'Application/Tests/Package/AllTests.php' ; class AllTests { public static function suite () { $suite = new PHPUnit_Framework_TestSuite ( 'Project' ); $suite -> addTest ( Package_AllTests :: suite ()); return $suite ; } } ?>
  32. Organizing Test Suites <?php require_once 'PHPUnit/Framework.php' ; require_once 'Application/Tests/Package/ClassTest.php' ; class Package_AllTests { public static function suite () { $suite = new PHPUnit_Framework_TestSuite ( 'Package' ); $suite -> addTestSuite ( 'Package_ClassTest' ); return $suite ; } } ?>
  33. Logging XML <?xml version= &quot;1.0&quot; encoding= &quot;UTF-8 &quot; ?> <testsuites> <testsuite name= &quot;CalculatorTest&quot; file= &quot;/home/sb/CalculatorTest.php&quot; tests= &quot;5&quot; failures= &quot;1&quot; errors= &quot;0&quot; time= &quot;0.0021438598632812&quot; > <testcase name= &quot;testAdd&quot; class= &quot;CalculatorTest&quot; file= &quot;/home/sb/CalculatorTest.php&quot; time= &quot;0.00045299530029297&quot; /> <testcase name= &quot;testAdd2&quot; class= &quot;CalculatorTest&quot; file= &quot;/home/sb/CalculatorTest.php&quot; time= &quot;0.0003299713134765&quot; /> <testcase name= &quot;testAdd3&quot; class= &quot;CalculatorTest&quot; file= &quot;/home/sb/CalculatorTest.php&quot; time= &quot;0.0003290176391601&quot; /> <testcase name= &quot;testAdd4&quot; class= &quot;CalculatorTest&quot; file= &quot;/home/sb/CalculatorTest.php&quot; time= &quot;0.0003290176391601&quot; /> <testcase name= &quot;testAdd5&quot; class= &quot;CalculatorTest&quot; file= &quot;/home/sb/CalculatorTest.php&quot; time= &quot;0.00070285797119141&quot; > <failure type= &quot;PHPUnit_Framework_ExpectationFailedException&quot; > Failed asserting that &lt;integer:3&gt; is not equal to &lt;integer:3&gt;. /home/sb/CalculatorTest.php:39 </failure> </testcase> </testsuite> </testsuites>
  34. Logging JavaScript Object Notation (JSON) {&quot;event&quot;:&quot;suiteStart&quot;,&quot;suite&quot;:&quot;CalculatorTest&quot;,&quot;tests&quot;:5} {&quot;event&quot;:&quot;test&quot;,&quot;suite&quot;:&quot;CalculatorTest&quot;,&quot;test&quot;:&quot;testAdd(CalculatorTest)&quot;, &quot;status&quot;:&quot;pass&quot;,&quot;time&quot;:0.0004529953,&quot;trace&quot;:[],&quot;message&quot;:&quot;&quot;} {&quot;event&quot;:&quot;test&quot;,&quot;suite&quot;:&quot;CalculatorTest&quot;,&quot;test&quot;:&quot;testAdd2(CalculatorTest)&quot;, &quot;status&quot;:&quot;pass&quot;,&quot;time&quot;:0.000329971313,&quot;trace&quot;:[],&quot;message&quot;:&quot;&quot;} {&quot;event&quot;:&quot;test&quot;,&quot;suite&quot;:&quot;CalculatorTest&quot;,&quot;test&quot;:&quot;testAdd3(CalculatorTest)&quot;, &quot;status&quot;:&quot;pass&quot;,&quot;time&quot;:0.000329017639,&quot;trace&quot;:[],&quot;message&quot;:&quot;&quot;} {&quot;event&quot;:&quot;test&quot;,&quot;suite&quot;:&quot;CalculatorTest&quot;,&quot;test&quot;:&quot;testAdd4(CalculatorTest)&quot;, &quot;status&quot;:&quot;pass&quot;,&quot;time&quot;:0.000329017639,&quot;trace&quot;:[],&quot;message&quot;:&quot;&quot;} {&quot;event&quot;:&quot;test&quot;,&quot;suite&quot;:&quot;CalculatorTest&quot;,&quot;test&quot;:&quot;testAdd5(CalculatorTest)&quot;, &quot;status&quot;:&quot;fail&quot;,&quot;time&quot;:0.000702857971,&quot;trace&quot;:[],&quot;message&quot;:&quot;Failed asserting that <integer:3> is not equal to <integer:3>.&quot;}
  35. Logging Test Anything Protocol (TAP) # TestSuite &quot;CalculatorTest&quot; started. ok 1 - testAdd(CalculatorTest) ok 2 - testAdd2(CalculatorTest) ok 3 - testAdd3(CalculatorTest) ok 4 - testAdd4(CalculatorTest) not ok 5 - Failure: testAdd5(CalculatorTest) # TestSuite &quot;CalculatorTest&quot; ended. 1..5
  36. Logging GraphViz
  37. Logging Software Metrics and Project Mess Detector
    • A software metric is a measure of some property of a piece of software or its specifications
      • Cyclomatic Complexity
      • NPath Complexity
      • Change Risk Analysis and Predictions (CRAP)
      • ...
    • Violations of rules that are based on software metrics
  38. Software Development Process
    • Unit Tests
      • play an essential role in several software development practices and processes
        • Test-First Programming
        • Test-Driven Development
        • Extreme Programming
      • allow for using the Design-by-Contract approach in programming languages that do not support it through language constructs
        • Contract that is to be followed when invoking a method
          • pre-condition, post-condition
          • semantic safety in addition to type safety
  39. This slide contains material by Brady Kelly. Software Development Process Test-Driven Development
    • Method of designing software, not just a method of testing software
      • Test
        • What do we want X to do?
        • How do we want to tell X to do it?
        • How will we know when X has done it?
      • Code
        • How does X do it?
    • Tests drive the development
      • Tests written before code
      • No code without tests
  40. Test-Driven Development The bank account example
    • Let's implement a class that represents a bank account using TDD
    • API
      • Retrieve the current balance
      • Set the balance
      • Deposit money
      • Withdraw money
    • Rules
      • Balance is initially zero
      • Balance cannot become negative
  41. Test-Driven Development The bank account example <?php require_once 'PHPUnit/Framework.php' ; require_once 'BankAccount.php' ; class BankAccountTest extends PHPUnit_Framework_TestCase { private $ba ; protected function setUp () { $this -> ba = new BankAccount ; } public function testBalanceIsInitiallyZero () { $this -> assertEquals ( 0 , $this -> ba -> getBalance ()); } } ?>
  42. Test-Driven Development The bank account example <?php require_once 'PHPUnit/Framework.php' ; require_once 'BankAccount.php' ; class BankAccountTest extends PHPUnit_Framework_TestCase { // ... public function testBalanceCannotBecomeNegative () { try { $this -> ba -> withdrawMoney ( 1 ); } catch ( BankAccountException $e ) { $this -> assertEquals ( 0 , $this -> ba -> getBalance ()); return; } $this -> fail (); } } ?>
  43. Test-Driven Development The bank account example <?php require_once 'PHPUnit/Framework.php' ; require_once 'BankAccount.php' ; class BankAccountTest extends PHPUnit_Framework_TestCase { // ... public function testBalanceCannotBecomeNegative2 () { try { $this -> ba -> depositMoney (- 1 ); } catch ( BankAccountException $e ) { $this -> assertEquals ( 0 , $this -> ba -> getBalance ()); return; } $this -> fail (); } } ?>
  44. Test-Driven Development The bank account example <?php require_once 'BankAccountException.php' ; class BankAccount { protected $balance = 0 ; public function getBalance () { return $this -> balance ; } protected function setBalance ( $balance ) { if ( $balance >= 0 ) { $this -> balance = $balance ; } else { throw new BankAccountException ; } } } ?>
  45. Test-Driven Development The bank account example <?php require_once 'BankAccountException.php' ; class BankAccount { // ... public function depositMoney ( $amount ) { $this -> setBalance ( $this -> getBalance () + $amount ); return $this -> getBalance (); } public function withdrawMoney ( $amount ) { $this -> setBalance ( $this -> getBalance () - $amount ); return $this -> getBalance (); } } ?>
  46. Test-Driven Development The bank account example [email_address] ~ % phpunit BankAccountTest PHPUnit 3.2.0 by Sebastian Bergmann. ... Time: 0 seconds OK (3 tests) [email_address] ~ % phpunit --testdox BankAccountTest PHPUnit 3.2.0 by Sebastian Bergmann. BankAccount - Balance is initially zero - Balance cannot become negative
  47. Code Coverage
    • Via Xdebug, PHPUnit knows which LOC of the SUT are executed when a test runs
    • This information tells us, among other things, to what degree the SUT is covered by tests
      • 100% code coverage is a required but not a sufficient criteria for test completeness
  48. Code Coverage The bank account example [email_address] ~ % phpunit --report BankAccount BankAccountTest PHPUnit 3.2.0 by Sebastian Bergmann. ... Time: 0 seconds OK (3 tests) Generating report, this may take a moment.
  49. Code Coverage The bank account example
  50. Code Coverage The bank account example
  51. Code Coverage The bank account example <?php require_once 'PHPUnit/Framework.php' ; require_once 'BankAccount.php' ; class BankAccountTest extends PHPUnit_Framework_TestCase { // ... public function testDepositWithdrawMoney () { $this -> assertEquals ( 0 , $this -> ba -> getBalance ()); $this -> ba -> depositMoney ( 1 ); $this -> assertEquals ( 1 , $this -> ba -> getBalance ()); $this -> ba -> withdrawMoney ( 1 ); $this -> assertEquals ( 0 , $this -> ba -> getBalance ()); } } ?>
  52. Code Coverage The bank account example
    • Microsoft Case Study
      • TDD project had twice the code quality
      • Writing the tests had a 15% productivity impact
    • IBM Case Study
      • 40% less bugs
      • No productivity impact
    • John Deere / Ericsson Case Study
      • TDD lead to better code quality
      • Writing the tests had a 16% productivity impact
        • The other teams did not test at all
    Software Development Process Test-Driven Development
  53. Software Development Process Extreme Programming
    • The Test-First approach is part of Extreme Programming
    • Other aspects of Extreme Programming
      • Integration of the Customer
        • Customer sets iteration goals
      • Continous Integration
      • Pair Programming
      • Refactoring
  54. Continous Integration
    • Continuous Integration is a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily, leading to multiple integrations per day.
    • Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.
    • Many teams find that this approach leads to significantly reduced integration problems and allows a team to develop cohesive software more rapidly.
  55. Continous Integration D E M O CruiseControl
  56. PHPUnit
    • Test Framework
      • Member of the xUnit family of test frameworks
      • Mock Objects
    • Integration
      • Selenium RC
      • Bamboo, Bitten, CruiseControl, ...
    • Even More Cool Stuff :-)
      • Code Coverage and Software Metrics
      • Test Database
      • Mutation Testing (GSoC07 project)
  57. Selenium
    • Selenium
      • Test web applications in a web browser
        • Browser Compatibility Testing
        • System Functional Testing
      • Runs in the browser
    • Selenium IDE
      • IDE for Selenium tests
        • Extension for Firefox
        • Record, execute, edit, debug tests in the browser
  58. Selenium
    • Selenium RC
      • Automated execution of Selenium tests
      • Tests can be specified in any language
        • PHP Bindings: PEAR Testing_Selenium
        • PHPUnit natively speaks the Selenium RC protocol
      • One test can be executed on multiple OS / Browser combinations
    Selenium RC
  59. Selenium Selenium RC
  60. Selenium D E M O Selenium RC
  61. Bibliography
    • T. Bhat, N. Nagappan, “Evaluating the Efficacy of Test-Driven Development: Industrial Case Studies”, ISESE'06, September 21–22, 2006, Rio de Janeiro, Brazil.
    • L. Williams, E. M. Maximilien, and M. Vouk, “Test-Driven Development as a Defect-Reduction Practice”, Proceedings of IEEE International Symposium on Software Reliability Engineering, Denver, CO, pp. 34-45, 2003.
    • B. George and L. Williams, “A Structured Experiment of Test-Driven Development”, Information and Software Technology (IST), 46(5), pp. 337-342, 2003.

+ Sebastian BergmannSebastian Bergmann, 3 years ago

custom

5550 views, 10 favs, 10 embeds more stats

Tutorial presented at php|works 2007 in Atlanta, GA more

More info about this document

CC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs LicenseCC Attribution-NonCommercial-NoDerivs License

Go to text version

  • Total Views 5550
    • 4897 on SlideShare
    • 653 from embeds
  • Comments 1
  • Favorites 10
  • Downloads 0
Most viewed embeds
  • 601 views on http://sebastian-bergmann.de
  • 29 views on http://www.planet-php.net
  • 7 views on http://www.planet-php.org
  • 5 views on http://planet-php.org
  • 3 views on http://www.netvibes.com

more

All embeds
  • 601 views on http://sebastian-bergmann.de
  • 29 views on http://www.planet-php.net
  • 7 views on http://www.planet-php.org
  • 5 views on http://planet-php.org
  • 3 views on http://www.netvibes.com
  • 3 views on http://www.phpeye.com
  • 2 views on http://lj-toys.com
  • 1 views on http://planet-php.net
  • 1 views on http://localhost
  • 1 views on http://s3.amazonaws.com

less

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate. If needed, use the feedback form to let us know more details.

Cancel
File a copyright complaint
Having problems? Go to our helpdesk?

Categories