Your SlideShare is downloading. ×
0
So ware Testing on the Web       Martin Schütte
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing        Conclusion . .            ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing        Conclusion . .            ...
So ware Quality     Documentation      Unit Testing        Web Drivers   Behaviour Testing        Conclusion . .          ...
So ware Quality     Documentation      Unit Testing        Web Drivers   Behaviour Testing        Conclusion . .          ...
So ware Quality   Documentation         Unit Testing        Web Drivers   Behaviour Testing        Conclusion . .         ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing        Conclusion . .            ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing        Conclusion . .            ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing        Conclusion . .            ...
So ware Quality   Documentation        Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .         ...
So ware Quality   Documentation       Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .          ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality     Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .         ...
So ware Quality          Documentation           Unit Testing                Web Drivers     Behaviour Testing         Con...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality     Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .         ...
So ware Quality   Documentation       Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .          ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality     Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .         ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality      Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .        ...
So ware Quality      Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .        ...
So ware Quality      Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .        ...
So ware Quality    Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .          ...
So ware Quality      Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .        ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality      Documentation       Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .       ...
So ware Quality       Documentation       Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .      ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality      Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .        ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality     Documentation       Unit Testing        Web Drivers    Behaviour Testing         Conclusion . .       ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality    Documentation      Unit Testing          Web Drivers    Behaviour Testing         Conclusion . .       ...
So ware Quality       Documentation        Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .     ...
So ware Quality   Documentation          Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .       ...
So ware Quality       Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .       ...
So ware Quality   Documentation        Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .         ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality   Documentation       Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .          ...
So ware Quality   Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .           ...
So ware Quality      Documentation      Unit Testing        Web Drivers   Behaviour Testing         Conclusion . .        ...
Upcoming SlideShare
Loading in...5
×

Software Testing on the Web

560

Published on

Overview of some software testing tools with focus on PHP and website tests
Session at WPCamp 2012 Berlin.

Published in: Technology
1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total Views
560
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
9
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Software Testing on the Web"

  1. 1. So ware Testing on the Web Martin Schütte
  2. 2. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Outline So ware Quality Documentation Unit Testing PHPUnit Continuity Web Drivers Behaviour Testing Meta PHP Behat and Ruby Cucumber CodeceptionMartin Schütte So ware Testing on the Web 2012-10-13 2 / 51
  3. 3. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Main Questions How can I know (my) so ware is correct? How does my boss know so ware is correct? How can we discuss what “correct” is, anyway? We always need: • implicit assumptions, • explicit specifications.Martin Schütte So ware Testing on the Web 2012-10-13 3 / 51
  4. 4. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Walking on water and developing so ware from a specification are easy if both are frozen. – Edward V. Berard Let’s just update PHP… Let’s just change the DB schema… Let’s just add feature X… Is the so ware still correct?Martin Schütte So ware Testing on the Web 2012-10-13 4 / 51
  5. 5. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Maintainability requires Documentation Consistency of: 1. Code 2. Comments 3. Documentation When code and comments disagree, both are probably wrong. – Norm SchryerMartin Schütte So ware Testing on the Web 2012-10-13 5 / 51
  6. 6. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Levels of Documentation . abstract UI External API Subsystems Libraries, Functions concreteMartin Schütte So ware Testing on the Web 2012-10-13 6 / 51
  7. 7. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPdoc/JavaDoc/Doxygen etc. /** * Gets single server parameter for specified key. * * @param string $key A key of the parameter to get * @param string $default A default value when key is undefined * * @return string A value of the parameter */ public function getServerParameter($key, $default = ’’) { return (isset($this->server[$key])) ? $this->server[$key] : $default; }Martin Schütte So ware Testing on the Web 2012-10-13 7 / 51
  8. 8. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Low Level DocumentationMartin Schütte So ware Testing on the Web 2012-10-13 8 / 51
  9. 9. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . High Level DocumentationMartin Schütte So ware Testing on the Web 2012-10-13 9 / 51
  10. 10. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Levels of Testing . abstract Acceptance Tests System Tests Unit Tests concreteMartin Schütte So ware Testing on the Web 2012-10-13 10 / 51
  11. 11. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Regression Testing Bug Tracker = DocumentationMartin Schütte So ware Testing on the Web 2012-10-13 11 / 51
  12. 12. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPUnit class DTABaseTest extends PHPUnit_Framework_TestCase { protected function setUp() { $this->fixture = new DTABase(); } public function testGetNumTooShort() { $input = ”12345”; $off = 3; $len = 4; $rc = $this->fixture->getNum($input, $off, $len); $this->assertEquals(”45”, $rc); } // ... }Martin Schütte So ware Testing on the Web 2012-10-13 12 / 51
  13. 13. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPUnit ~/Payment_DTA> phpunit tests/DTABaseTest.php PHPUnit 3.6.11 by Sebastian Bergmann. F.... Time: 0 seconds, Memory: 3.75Mb There was 1 failure: 1) DTABaseTest::testGetNumTooShort Failed asserting that 45.001 matches expected ’45’. /usr/home/mschuett/Payment_DTA/tests/DTABaseTest.php:50 /usr/local/bin/phpunit:46 FAILURES! Tests: 5, Assertions: 5, Failures: 1.Martin Schütte So ware Testing on the Web 2012-10-13 13 / 51
  14. 14. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Tests are Documentation ~/Payment_DTA> phpunit --testdox tests/DTABaseTest.php PHPUnit 3.6.11 by Sebastian Bergmann. DTABase [ ] Get num too short [x] Get num offset too big [x] Get str too short [x] Get str offset too big [x] Get str invalidMartin Schütte So ware Testing on the Web 2012-10-13 14 / 51
  15. 15. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . WordPress Unit Tests . http://unit-tests.trac.wordpress.org/ vagrant@wp:~/trunk$ phpunit Running as single site... To run multisite, use -c multisite.xml Not running ajax tests... To execute these, use --group ajax. PHPUnit 3.6.12 by Sebastian Bergmann. Configuration read from /home/vagrant/trunk/phpunit.xml ................F....................................SS.....S 61 / 1437 ( 4%) S...SSSSSS.............S..SSS..........SS................S... 122 / 1437 ( 8%) .......SS.SSSS...............S.................S............. 183 / 1437 ( 12%) ...............................S............S................ 244 / 1437 ( 16%) .S........................S.................S.......I........ 305 / 1437 ( 21%) .........................SSSS.S..SSS......................... 366 / 1437 ( 25%) ............................................................. 427 / 1437 ( 29%) ............................................................. 488 / 1437 ( 33%) ............................................................. 549 / 1437 ( 38%) ............................................................. 610 / 1437 ( 42%) ..................................................S.......... 671 / 1437 ( 46%) .......SSSSSSSSSSSSSS...........SS........................... 732 / 1437 ( 50%) .............SS...............SSSSSSSSSSSSSSSSSS............. 793 / 1437 ( 55%) ................SSS...S.SSSSSSS................SSSSSSS....... 854 / 1437 ( 59%) .............S..SS....FFF....SSSS.SS......................SSS 915 / 1437 ( 63%) ............................................................. 976 / 1437 ( 67%) ......S..................................................S..S 1037 / 1437 ( 72%) ................SS.S.........................S............... 1098 / 1437 ( 76%) ...............S..................S.......................... 1159 / 1437 ( 80%) ............................................................. 1220 / 1437 ( 84%) .......................F..............................S...... 1281 / 1437 ( 89%) ............................................................. 1342 / 1437 ( 93%) .............................................SS. Time: 08:56, Memory: 122.75Mb [...] FAILURES! Tests: 1390, Assertions: 5640, Failures: 5, Incomplete: 1, Skipped: 117.Martin Schütte So ware Testing on the Web 2012-10-13 15 / 51
  16. 16. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . PHPUnit Code CoverageMartin Schütte So ware Testing on the Web 2012-10-13 16 / 51
  17. 17. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Things to build upon Automated Testing • git bisect run phpunit: find commit that introduces a bug • Continuous Integration: • frequent check-ins to mainline branch • verified by automated builds and tests • Continuous Deployment: • automated deployment to production • useful: automated rollbackMartin Schütte So ware Testing on the Web 2012-10-13 17 / 51
  18. 18. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example: Jenkins CIMartin Schütte So ware Testing on the Web 2012-10-13 18 / 51
  19. 19. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Example: Travis CI for GitHubMartin Schütte So ware Testing on the Web 2012-10-13 19 / 51
  20. 20. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Approaches to Unit Testing • Exploratory testing (e. g. for 3rd party libs) • Regression testing (for found & fixed bugs) • Spike and Stabilise testing (a er code, before refactoring) • Test Driven Development (TDD): test first, then codeMartin Schütte So ware Testing on the Web 2012-10-13 20 / 51
  21. 21. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Program testing can be used to show the presence of bugs, but never to show their absence! – Edsger Dijkstra, Notes on Structured Programming Testing by itself does not improve so ware quality. Test results are an indicator of quality, but in and of themselves, they don’t improve it. […] If you want to improve your so ware, don’t test more; develop better. – Steve McConnell, Code CompleteMartin Schütte So ware Testing on the Web 2012-10-13 21 / 51
  22. 22. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cost of Testing • Testing (like Documentation) has a cost • o en: productivity improvement > cost • but ROI depends on type of project/so wareMartin Schütte So ware Testing on the Web 2012-10-13 22 / 51
  23. 23. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTTP API class TumblrAPITest extends PHPUnit_Framework_TestCase { public function testPostDatetime() { $url = ”http://staff.tumblr.com/api/read”; $ch = curl_init($url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $xmlsrc = curl_exec($ch); $xmlobj = new SimpleXMLElement($xmlsrc); $str_date = date_timestamp_get(date_create( $xmlobj->posts->post[0][’date-gmt’])); $unix_date = intval( $xmlobj->posts->post[0][’unix-timestamp’]); $this->assertEquals($str_date, $unix_date); } }Martin Schütte So ware Testing on the Web 2012-10-13 23 / 51
  24. 24. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HTML-Content class WpCampTest extends PHPUnit_Framework_TestCase { public function testSiteTitle() { $url = ”http://wpcamp.de/”; $ch = curl_init($url); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); $xmlsrc = curl_exec($ch); $dom = @DOMDocument::loadHTML($xmlsrc); $title = $dom->getElementsByTagName(’title’) ->item(0)->textContent; $this->assertContains(’Das WordPress Event’, $title); } } .Martin Schütte So ware Testing on the Web 2012-10-13 24 / 51
  25. 25. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Goutte a simple PHP Web Scraper class WpCampTest extends PHPUnit_Framework_TestCase { public function testSiteTitle() { $client = new Client(); $crawler = $client ->request(’GET’, ’http://wpcamp.de/’); $title = $crawler ->filter(’html head title’) ->first()->text(); $this->assertContains(’Das WordPress Event’, $title); } }Martin Schütte So ware Testing on the Web 2012-10-13 25 / 51
  26. 26. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Selenium a suite of tools to automate web browsers across platforms class WpTest extends PHPUnit_Extensions_SeleniumTestCase { public function setUp() { $this->setHost(”localhost”); $this->setBrowser(”firefox”); $this->setBrowserUrl(”http://wpcamp.de/”); } public function testSiteTitle() { $this->open(”/”); $this->assertText(”//html/head/title”, ”Das WordPress Event *”); }Martin Schütte So ware Testing on the Web 2012-10-13 26 / 51
  27. 27. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Selenium, contd. public function testSearch() { $this->open(”/”); $this->type(”id=s”, ”sponsoren”); $this->click(”id=searchsubmit”); $this->waitForPageToLoad(”30000”); $this->assertText(”css=#content”, ”regexp:.*Premium-Sponsor Chocri.*”); $this->assertText(”css=#content”, ”regexp:.*Platin-Sponsor Adobe.*”); } }Martin Schütte So ware Testing on the Web 2012-10-13 27 / 51
  28. 28. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Mink an acceptance test framework for web applications class WpCampTest extends PHPUnit_Framework_TestCase { public function setUp() { $driver = new BehatMinkDriverGoutteDriver(); $this->session = new BehatMinkSession($driver); } public function testSiteTitle() { $this->session->visit(’http://wpcamp.de/’); $title = $this->session->getPage() ->find(’css’, ’html head title’)->getText(); $this->assertContains(’Das WordPress Event’, $title); }Martin Schütte So ware Testing on the Web 2012-10-13 28 / 51
  29. 29. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Mink, contd. public function testSearch() { $this->session->visit(’http://wpcamp.de/’); $page = $this->session->getPage(); $page->find(’css’, ’#s’)->setValue(’Sponsoren’); $page->find(’css’, ’#searchsubmit’)->press(); $text = $this->session->getPage() ->find(’css’, ’#content’)->getText(); $this->assertContains( ’Premium-Sponsor Chocri’, $text); $this->assertContains( ’Platin-Sponsor Adobe’, $text); } }Martin Schütte So ware Testing on the Web 2012-10-13 29 / 51
  30. 30. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Big Picture PHPUnit . Mink Selenium Sahi Goutte Zombie.js Firefox OperaMartin Schütte So ware Testing on the Web 2012-10-13 30 / 51
  31. 31. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problems with Web Drivers Differences in: • web functionality (HTML, CSS, JS, AJAX) ⇒ performance • selectors (ID, CSS, XPath, DOM) • test functionality and integration with test framework Recommendation: Select two Drivers, 1. simple and fast one for unit testing and 2. Selenium/Sahi for integration and UX testing.Martin Schütte So ware Testing on the Web 2012-10-13 31 / 51
  32. 32. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Testing in Production = Monitoring Example: check webshop availability Possible layers: • ping server • check HTTP status ”200 OK” • check homepage title • login as user • complete order processMartin Schütte So ware Testing on the Web 2012-10-13 32 / 51
  33. 33. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The next level: Behaviour Tests Funktionalität: Selbstpräsentation Um etwas über das WPCamp zu erfahren Als Besucher Möchte ich Infos auf der Website finden Szenario: Sponsoren suchen Angenommen ich bin auf ”/” Und ich suche nach ”sponsoren” Dann ich sollte ”Premium-Sponsor Chocri” sehen Und ich sollte ”Platin-Sponsor Adobe” sehenMartin Schütte So ware Testing on the Web 2012-10-13 33 / 51
  34. 34. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Next Level: Behaviour Driven Development TDD with other syntax? Is BDD the same as TDD? Yes. If you’re a programmer, and your entire team is programmers, and all your stakeholders are programmers. – Dan North We don’t call them “acceptance tests” because you can’t ask a business person “Please help me with my acceptance test”. Try “I’d like to talk to you about the scenario where…” instead. Or, “Can you give me an example?” Either of these are good. – Liz KeoghMartin Schütte So ware Testing on the Web 2012-10-13 34 / 51
  35. 35. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Unit & Behaviour TestingUnit Tests Behaviour Tests • unit testing • acceptance test scenarios • programmers • non-developers • programming language • language of business domain • bottom-up • top-down / outside-in • assertXYZ • X should do Y • tests derived from user stories • execute user stories ⇒ development tool ⇒ communication toolMartin Schütte So ware Testing on the Web 2012-10-13 35 / 51
  36. 36. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . One Notation/“Language”: Gherkin Feature: [one line describing the story] In order to [benefit] As a [role] I want [feature] Scenario: [A scenario specific goal] Given [Something that needs to have happened or be true] When [Some task I must do] And [Another task] Then [Some way I know I’ve achieved my goal]Martin Schütte So ware Testing on the Web 2012-10-13 36 / 51
  37. 37. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I18N # language: de Funktionalität: [one line describing the story] Um zu erreichen [benefit] Als [role] Möchte ich [feature] Szenario: [Title] Angenommen [context] Wenn [event] Dann [outcome] Und [another outcome]Martin Schütte So ware Testing on the Web 2012-10-13 37 / 51
  38. 38. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . TDD with other syntax? Scenario: New lists are empty Given a new list Then the list should be empty. Scenario: Lists with things in them are not empty. Given a new list When we add an object Then the list should not be empty. .Martin Schütte So ware Testing on the Web 2012-10-13 38 / 51
  39. 39. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Implementation / Translation A look behind the curtain: • framework is clever but not magical • some translation needed • statements have to become executable code Mechanism: • plain sentence → method name • quoted words → arguments • matching with annotated RegExp • methods yield success, exception, or pending exceptionMartin Schütte So ware Testing on the Web 2012-10-13 39 / 51
  40. 40. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat/Mink FeatureContext.php use BehatMinkExtensionContextMinkContext; class FeatureContext extends BehatContext { public function __construct(array $parameters) { $this->useContext(’mink’, new MinkContext()); } } MinkContext adds web-related methods and their I18N translations, e. g.: • visit($page) • fillField($field, $value) • pressButton($button)Martin Schütte So ware Testing on the Web 2012-10-13 40 / 51
  41. 41. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Output [mschuett@dagny] ~/wpcamp/code% ./vendor/bin/behat Funktionalität: Selbstpräsentation Um etwas über das WPCamp zu erfahren Als Besucher Möchte ich Infos auf der Website finden Szenario: Sponsoren suchen # features/wpcamp.feature:8 Angenommen ich bin auf ”/” # BehatMinkExtensionContextMinkContext::visit Und ich suche nach ”sponsoren” Dann ich sollte ”Premium-Sponsor Chocri” sehen # BehatMinkExtensionContextMinkContext::asser Und ich sollte ”Platin-Sponsor Adobe” sehen # BehatMinkExtensionContextMinkContext::asser 1 scenario (1 undefined) 4 steps (1 passed, 2 skipped, 1 undefined) 0m1.574s You can implement step definitions for undefined steps with these snippets: /** * @Given /^ich suche nach ”([^”]*)”$/ */ public function ichSucheNach($arg1) { throw new PendingException(); }Martin Schütte So ware Testing on the Web 2012-10-13 41 / 51
  42. 42. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Edit FeatureContext.php add method /** * @Given /^ich suche nach ”([^”]*)”$/ */ public function ichSucheNach($arg) { return array( new When(”I fill in ”s” with ”$arg””), new When(”I press ”searchsubmit””), ); }Martin Schütte So ware Testing on the Web 2012-10-13 42 / 51
  43. 43. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Behat Output [mschuett@dagny] ~/wpcamp/code% ./vendor/bin/behat Funktionalität: Selbstpräsentation Um etwas über das WPCamp zu erfahren Als Besucher Möchte ich Infos auf der Website finden Szenario: Sponsoren suchen # features/wpcamp.feature:8 Angenommen ich bin auf ”/” # BehatMinkExtensionContextMinkContext::visit Und ich suche nach ”sponsoren” # FeatureContext::ichSucheNach() Dann ich sollte ”Premium-Sponsor Chocri” sehen # BehatMinkExtensionContextMinkContext::asser Und ich sollte ”Platin-Sponsor Adobe” sehen # BehatMinkExtensionContextMinkContext::asser 1 scenario (1 passed) 4 steps (4 passed) 0m2.59sMartin Schütte So ware Testing on the Web 2012-10-13 43 / 51
  44. 44. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Big Picture (PHP) Behat PHPUnit . Mink Selenium Sahi Goutte Zombie.js Firefox OperaMartin Schütte So ware Testing on the Web 2012-10-13 44 / 51
  45. 45. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Cucumber original Ruby version, nicer syntax # Example Webrat step definitions When /I search for ”(.*)”/ do |query| @browser.text_field(:name, ’s’).set(query) @browser.button(:name, ’searchsubmit’).click end Then /I should see/ do |text| @browser.text.should =~ /#{text}/m endMartin Schütte So ware Testing on the Web 2012-10-13 45 / 51
  46. 46. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . The Big Picture (Ruby) . Cucumber Selenium Watir Webrat Celerity Firefox OperaMartin Schütte So ware Testing on the Web 2012-10-13 46 / 51
  47. 47. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Codeception unit test a new full-stack testing PHP framework $I = new CodeGuy($scenario); $I->wantTo(’perform actions and see results’); $I->testMethod(’DateTime.getTimestamp’); $date = new DateTime(”2012-10-13T14:45:00+02:00”); $I->executeTestedMethodOn($date); $I->seeResultIs(’int’); $I->seeResultEquals(1350132300);Martin Schütte So ware Testing on the Web 2012-10-13 47 / 51
  48. 48. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Codeception acceptance test $I = new WebGuy($scenario); $I->wantTo(’find infos on the website’); $I->see(’Das WordPress Event’, ’html head title’); $I->fillField(’s’, ’sponsoren’); $I->click(’searchsubmit’); $I->see(’Premium-Sponsor Chocri’, ’#content’); $I->see(’Platin-Sponsor Adobe’, ’#content’);Martin Schütte So ware Testing on the Web 2012-10-13 48 / 51
  49. 49. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Codeception Output [mschuett@dagny] ~/wpcamp/code% php codecept.phar run --steps Codeception PHP Testing Framework v1.1.4 Powered by PHPUnit 3.6.10 by Sebastian Bergmann. Suite unit started Trying to perform actions and see results (exampleCept.php) Scenario: * I test method ”DateTime.getTimestamp” * I execute tested method on ”DateTime” * I see result is ”int” * I see result equals ”1350132300” OK Suite functional started Suite acceptance started Trying to find infos on the website (wpcampCept.php) Scenario: * I am on page ”/” * I see ”Das WordPress Event”,”html head title” * I fill field ”s”,”sponsoren” * I click ”searchsubmit” * I see ”Premium-Sponsor Chocri”,”#content” * I see ”Platin-Sponsor Adobe”,”#content” OK Time: 4 seconds, Memory: 9.50Mb OK (2 tests, 5 assertions)Martin Schütte So ware Testing on the Web 2012-10-13 49 / 51
  50. 50. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Problems • most Web tests require known DB state • behaviour testing is slow • only run locally • use in-memory DB • draw the line • features in documentation (what) → add behaviour test • mere implementation (how) → only unit testsMartin Schütte So ware Testing on the Web 2012-10-13 50 / 51
  51. 51. So ware Quality Documentation Unit Testing Web Drivers Behaviour Testing Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Summary Value individuals and interactions over processes and tools – The Agile Manifesto Links: • PHPUnit • Sebastian Bergmann (PHPUnit) • Selenium • Dan North’s blog (BDD) • Behat & Mink • Liz Keogh’s blog (BDD) • Cucumber • Geek and Poke • Codeception • Not Invented HereMartin Schütte So ware Testing on the Web 2012-10-13 51 / 51
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×