Your SlideShare is downloading. ×
Reliable acceptance testing
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Reliable acceptance testing

1,349
views

Published on

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,349
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Dagfinn Reiersøl, ABC Startsiden 1 Reliable acceptance testing
  • 2. Dagfinn Reiersøl, ABC Startsiden 2 Manual testing is immoral • Not what humans are good at • Time consuming = expensive • High stress • Error-prone • Exception: exploratory testing
  • 3. Dagfinn Reiersøl, ABC Startsiden 3 Types of automated testing • Unit tests: clean code • Acceptance tests: communicate with user / customer • Other functional tests: ensure that everything works
  • 4. Dagfinn Reiersøl, ABC Startsiden 4 High-level challenges of acceptance testing • Getting the Product Owner involved • Test Doubles • Not over-testing the user interface
  • 5. Dagfinn Reiersøl, ABC Startsiden 5 Unit tests • Keep code clean • Avoid “hidden” bugs • Aid design • Test-driven development
  • 6. Dagfinn Reiersøl, ABC Startsiden 6 Scrum Product Owner • Represents customer(s) and other stakeholders • Owns the product backlog • Is not a developer
  • 7. Dagfinn Reiersøl, ABC Startsiden 7 Acceptance tests • For the Product Owner or possibly testers • Goal: Make sure we implement the right features • Ideally editable by a non-programmer • Tests features, not modules • Make sure you're done
  • 8. Dagfinn Reiersøl, ABC Startsiden 8 Other functional tests • Goal: Make sure the implementation works • Could be testing the same thing as acceptance tests, but • need not be business-readable
  • 9. Dagfinn Reiersøl, ABC Startsiden 9 The Test Double pattern
  • 10. Dagfinn Reiersøl, ABC Startsiden 10 Surrounded by “Test Doubles”
  • 11. Dagfinn Reiersøl, ABC Startsiden 11 Web testing: advantages • You can test exactly what the user sees • There are tools to test JavaScript-heavy / Ajax apps • There are tools to record and play back tests • “It seemed like an good idea at the time”
  • 12. Dagfinn Reiersøl, ABC Startsiden 12 Web testing: disadvantages • Fragile: Small UI changes break tests • Fragile: External services break tests • Slow • Harder to replace external services with fakes
  • 13. Dagfinn Reiersøl, ABC Startsiden 13 Strategies for robustness • Test Bus • Test patterns • Test refactoring • Making tests unnecessary
  • 14. Dagfinn Reiersøl, ABC Startsiden 14 Web testing: essentials • Must be in a programming language (conditionals, variables) • Language not necessarily PHP, since HTTP allows communication between languages • “Business-readable” programming language (FitNesse wiki, Gherkin...)
  • 15. Dagfinn Reiersøl, ABC Startsiden 15 Web testing tips • Test the HTML output using regex, XPath, CSS selector • Use element IDs or names to test links, forms and fields • Log HTTP requests in the application
  • 16. Dagfinn Reiersøl, ABC Startsiden 16 How to use the request log Automated test fails but not manual run from browser • Run the automated test, find the failing request in the log. • Run the app manually, find the successful request in the log. • Compare.
  • 17. Dagfinn Reiersøl, ABC Startsiden 17 Using SimpleTest <?php require_once 'simpletest/autorun.php'; require_once 'simpletest/web_tester.php'; error_reporting (E_ALL); class JoomlaSimpleTestTest extends WebTestCase { function testJoomla() { $this->get('http://localhost/joomla'); $this->assertResponse(200); $this->assertTitle('Welcome to the Frontpage'); $this->assertText('Welcome to the Frontpage'); $this->click('Welcome to Joomla!'); $this->assertPattern('/Joomla! is a free.*content publishing system/is'); } }
  • 18. Dagfinn Reiersøl, ABC Startsiden 18 Using PHPUnit with Selenium <?php require_once 'PHPUnit/Extensions/SeleniumTestCase.php'; class JoomlaPhpUnitTest extends PHPUnit_Extensions_SeleniumTestCase { protected function setUp(){ $this->setBrowser('*firefox'); $this->setBrowserUrl('http://localhost/'); } public function testJoomla() { $this->open('http://localhost/joomla/'); $this->assertTitle('Welcome to the Frontpage'); $this->assertTextPresent('Welcome to the Frontpage'); $this ->clickAndWait( '//ul[@class='latestnews']/li[4]/a'); $this ->assertTextPresent('Joomla! is a free open'); } }
  • 19. Dagfinn Reiersøl, ABC Startsiden 19 SimpleTest vs. PHPUnit / Selenium Simpletest: – Faster – Easier to use PHPUnit with Selenium – Heavier but more advanced – Can test JavaScript-heavy / Ajax apps – Can test cross-browser compatibility
  • 20. Dagfinn Reiersøl, ABC Startsiden 20 Don't overdo Selenium “He ate a lot of brazil nuts which is a big deal because they contain selenium, which in high doses causes fatigue, vomiting, skin irritation, discharge from the fingernail beds and hair loss.” - House, M.D.
  • 21. Dagfinn Reiersøl, ABC Startsiden 21 Acceptance test frameworks • FitNesse: has specific support for PHP • Cucumber: has PHP-related documentation • Robot Framework • Concordion
  • 22. Dagfinn Reiersøl, ABC Startsiden 22 Using FitNesse with SimpleTest
  • 23. Dagfinn Reiersøl, ABC Startsiden 23 PhpSlim Fixture for FitNesse test (simplistic!) <?php require_once 'simpletest/browser.php'; class SimpleTestFixture { function __construct() { $this->browser = new SimpleBrowser; } function goToPage($url) { $this->content = $this->browser->get($url); return !empty($this->content); } function pattern($pattern) { return preg_match("/$pattern/is",$this->content) > 0; } }
  • 24. Dagfinn Reiersøl, ABC Startsiden 24 Using Cucumber with Webrat • joomla.feature (Cucumber Gherkin business-readable) • webrat_steps (Ruby code testing via HTTP) Feature: Joomla welcome In order to understand what Joomla is As a potential user I want to be able to read an introduction to Joomla Scenario: Welcome page Given I am on the home page When I follow "Welcome to Joomla!" Then I should see "Joomla! is a"
  • 25. Dagfinn Reiersøl, ABC Startsiden 25 Webrat steps for Cucumber • Ruby code runs Webrat which runs the web app using HTTP Given /^I am on (.+)$/ do |page_name|   visit path_to(page_name) end When /^I follow "([^"]*)"$/ do |link|   click_link (link) end  Then /^I should see "([^"]*)"$/ do |text|   response_body.should contain(text) end
  • 26. Dagfinn Reiersøl, ABC Startsiden 26 Avoiding difficult tests • It's hard to test behaviors that cut across the HTTP request • Improve architecture so these behaviors can't fail • Example: duplication between input control name and HTTP variable name
  • 27. Dagfinn Reiersøl, ABC Startsiden 27 Hard to test: name duplication <input type="text" name="address1" value="<?= $address1 ?>"> // Oops... echo $_POST['address_1'];
  • 28. Dagfinn Reiersøl, ABC Startsiden 28 Getting rid of name duplication • Use a form handling package • Or do it yourself. Simplistic version here: <?php define('ADDRESS1_NAME', 'address1'); $address1_value = $_POST[ADDRESS1_NAME]; <html> ... <input name="<?= ADDRESS1_NAME ?>" value="<?= $address1_value ?>">
  • 29. Dagfinn Reiersøl, ABC Startsiden 29 Refactoring tests • Refactor or be doomed to drown in duplicate code • Especially true of web tests • Most important: eliminate duplication
  • 30. Dagfinn Reiersøl, ABC Startsiden 30 BDD-style test names class JoomlaSimpleTestTest extends WebTestCase {     function setUp() {         $this­>get('http://localhost/joomla');     }     function testFrontPageIsValid() {         $this­>assertResponse(200);         $this­>assertTitle('Welcome to the Frontpage');         $this­>assertText('Welcome to the Frontpage');     }     function testWelcomePageIsValid() {         $this­>click('Welcome to Joomla!');         $this­>assertPattern('/Joomla! is a free.*content  publishing system/is');     } }
  • 31. Dagfinn Reiersøl, ABC Startsiden 31 Using custom assertions instead class JoomlaSimpleTestTest extends WebTestCase {     function testJoomla() {         $this­>get('http://localhost/joomla');         $this­>assertValidHomePage();         $this­>click('Welcome to Joomla!');         $this­>assertValidWelcomePage();     }     function assertValidHomePage() {         $msg = 'Error on home page';         $this­>assertResponse(200,$msg);         $this­>assertTitle('Welcome to the Frontpage',$msg);         $this­>assertText('Welcome to the Fontpage',$msg);     }     function assertValidWelcomePage() {         $this­>assertPattern(            '/Joomla! is a free.*content publishing system/is',            'Error on welcome page');     }
  • 32. Dagfinn Reiersøl, ABC Startsiden 32 ComposedRegex http://martinfowler.com/bliki/ComposedRegex.html • This regex tolerates markup (or anything else) between the content items • Yes, I know the email regex is simplistic $email = '.*w+@w+.w+.*';         $firstname = '.*w.*';         $lastname = '.*w.*';         $date = '.*20dd­dd­dd( |&nbsp;|T)dd:dd.*';         $regex = 'Email address:'. $email .                   'Name: ' . $firstname . $lastname .                   'Date:' . $date;
  • 33. Dagfinn Reiersøl, ABC Startsiden 33 Re-using setup • Refactor gradually as needed: – When test class gets big, split it and... – ...extract setup method into a separate class – When setup is common to many tests... – ...extract setup class into a separate file class JoomlaTestSetup {     function setUp() {         $this­>get('http://localhost/joomla');     } } class JoomlaSimpleTestTest extends JoomlaTestSetup..
  • 34. Dagfinn Reiersøl, ABC Startsiden 34 Useful types of classes in web testing • Scraper • Domain Object • Specification
  • 35. Dagfinn Reiersøl, ABC Startsiden 35 Domain object test version class UserTest extends WebTestCase {     function setUp() {         $this­>user = new TestUser(             'Jane','Doe','jane@example.com');     }     function testCanRegisterUser...         $this­>setFieldByName(             'firstname',            $this­>user­>firstname         );
  • 36. Dagfinn Reiersøl, ABC Startsiden 36 Given-when-then in PHP class JoomlaSimpleTestTest extends WebTestCase {     function setUp() {         $this­>pages = array(             'JoomlaHomePage' => 'http://localhost/joomla');     }     function testWelcomPageIsValid() {         $this­>givenIAmOn('JoomlaHomePage');         $this­>whenIFollow('Welcome to Joomla!');         $this­>thenIShouldSee('Joomla! is a free');     }     function givenIAmOn($name) {         $this­>get($this­>pages[$name]);     }     function whenIFollow($text) {        $this­>clickLink($text);      }     function thenIShouldSee($pattern) {         $this­>assertPattern("/$pattern/is");     }
  • 37. Dagfinn Reiersøl, ABC Startsiden 37 Test bus http://www.martinfowler.com/ieeeSoftware/testBus.pdf
  • 38. Dagfinn Reiersøl, ABC Startsiden 38 Recommendations from Uncle Bob • Test the features through the test bus • Test the user interface separately in isolation • Run a few end-to-end tests to test the “plumbing”
  • 39. Dagfinn Reiersøl, ABC Startsiden 39 Test bus Where is the test bus? Some possbilities: • Test the Model • Test the Controller • Test the server side of an Ajax or “thin server” application • Test a public API
  • 40. Dagfinn Reiersøl, ABC Startsiden 40 Test bus with presentation tests
  • 41. Dagfinn Reiersøl, ABC Startsiden 41 Fake web service • Easier to do when not testing via HTTP • Sensing/verification is very hard to do
  • 42. Dagfinn Reiersøl, ABC Startsiden 42 Self-initializing fake http://martinfowler.com/bliki/SelfInitializingFake.html
  • 43. Dagfinn Reiersøl, ABC Startsiden 43 Self-initializing fake http://martinfowler.com/bliki/SelfInitializingFake.html

×