The document discusses various techniques for automated testing, particularly acceptance testing. It recommends that manual testing is time-consuming and error-prone, so automated testing is preferable. It describes different types of automated tests like unit tests and acceptance tests. It also discusses challenges with acceptance testing, such as getting product owners involved. Finally, it provides examples of implementing automated testing with frameworks like SimpleTest and PHPUnit.

1. Dagfinn Reiersøl, ABC Startsiden 1
Reliable acceptance testing

2. Dagfinn Reiersøl, ABC Startsiden 2
Manual testing is immoral
• Not what humans are good at
• Time consuming = expensive
• High stress
• Error-prone
• Exception: exploratory testing

3. Dagfinn Reiersøl, ABC Startsiden 3
Types of automated testing
• Unit tests: clean code
• Acceptance tests: communicate with user / customer
• Other functional tests: ensure that everything works

4. Dagfinn Reiersøl, ABC Startsiden 4
High-level challenges of
acceptance testing
• Getting the Product Owner involved
• Test Doubles
• Not over-testing the user interface

5. Dagfinn Reiersøl, ABC Startsiden 5
Unit tests
• Keep code clean
• Avoid “hidden” bugs
• Aid design
• Test-driven development

6. Dagfinn Reiersøl, ABC Startsiden 6
Scrum Product Owner
• Represents customer(s) and other stakeholders
• Owns the product backlog
• Is not a developer

7. Dagfinn Reiersøl, ABC Startsiden 7
Acceptance tests
• For the Product Owner or possibly testers
• Goal: Make sure we implement the right features
• Ideally editable by a non-programmer
• Tests features, not modules
• Make sure you're done

8. Dagfinn Reiersøl, ABC Startsiden 8
Other functional tests
• Goal: Make sure the implementation works
• Could be testing the same thing as acceptance tests, but
• need not be business-readable

9. Dagfinn Reiersøl, ABC Startsiden 9
The Test Double pattern

10. Dagfinn Reiersøl, ABC Startsiden 10
Surrounded by “Test Doubles”

11. Dagfinn Reiersøl, ABC Startsiden 11
Web testing: advantages
• You can test exactly what the user sees
• There are tools to test JavaScript-heavy / Ajax apps
• There are tools to record and play back tests
• “It seemed like an good idea at the time”

12. Dagfinn Reiersøl, ABC Startsiden 12
Web testing: disadvantages
• Fragile: Small UI changes break tests
• Fragile: External services break tests
• Slow
• Harder to replace external services with fakes

13. Dagfinn Reiersøl, ABC Startsiden 13
Strategies for robustness
• Test Bus
• Test patterns
• Test refactoring
• Making tests unnecessary

14. Dagfinn Reiersøl, ABC Startsiden 14
Web testing: essentials
• Must be in a programming language (conditionals, variables)
• Language not necessarily PHP, since HTTP allows
communication between languages
• “Business-readable” programming language (FitNesse wiki,
Gherkin...)

15. Dagfinn Reiersøl, ABC Startsiden 15
Web testing tips
• Test the HTML output using regex, XPath, CSS selector
• Use element IDs or names to test links, forms and fields
• Log HTTP requests in the application

16. Dagfinn Reiersøl, ABC Startsiden 16
How to use the request log
Automated test fails but not manual run from browser
• Run the automated test, find the failing request in the log.
• Run the app manually, find the successful request in the log.
• Compare.

17. Dagfinn Reiersøl, ABC Startsiden 17
Using SimpleTest
<?php
require_once 'simpletest/autorun.php';
require_once 'simpletest/web_tester.php';
error_reporting (E_ALL);
class JoomlaSimpleTestTest extends WebTestCase {
function testJoomla() {
$this->get('http://localhost/joomla');
$this->assertResponse(200);
$this->assertTitle('Welcome to the Frontpage');
$this->assertText('Welcome to the Frontpage');
$this->click('Welcome to Joomla!');
$this->assertPattern('/Joomla! is a free.*content
publishing system/is');
}
}

18. Dagfinn Reiersøl, ABC Startsiden 18
Using PHPUnit with Selenium
<?php
require_once 'PHPUnit/Extensions/SeleniumTestCase.php';
class JoomlaPhpUnitTest extends
PHPUnit_Extensions_SeleniumTestCase {
protected function setUp(){
$this->setBrowser('*firefox');
$this->setBrowserUrl('http://localhost/');
}
public function testJoomla() {
$this->open('http://localhost/joomla/');
$this->assertTitle('Welcome to the Frontpage');
$this->assertTextPresent('Welcome to the Frontpage');
$this ->clickAndWait(
'//ul[@class='latestnews']/li[4]/a');
$this ->assertTextPresent('Joomla! is a free open');
}
}

19. Dagfinn Reiersøl, ABC Startsiden 19
SimpleTest vs. PHPUnit /
Selenium
Simpletest:
– Faster
– Easier to use
PHPUnit with Selenium
– Heavier but more advanced
– Can test JavaScript-heavy / Ajax apps
– Can test cross-browser compatibility

20. Dagfinn Reiersøl, ABC Startsiden 20
Don't overdo Selenium
“He ate a lot of brazil nuts which is a big deal because they
contain selenium, which in high doses causes fatigue,
vomiting, skin irritation, discharge from the fingernail beds and
hair loss.”
- House, M.D.

21. Dagfinn Reiersøl, ABC Startsiden 21
Acceptance test frameworks
• FitNesse: has specific support for PHP
• Cucumber: has PHP-related documentation
• Robot Framework
• Concordion

22. Dagfinn Reiersøl, ABC Startsiden 22
Using FitNesse with SimpleTest

23. Dagfinn Reiersøl, ABC Startsiden 23
PhpSlim Fixture for FitNesse test
(simplistic!)
<?php
require_once 'simpletest/browser.php';
class SimpleTestFixture {
function __construct() {
$this->browser = new SimpleBrowser;
}
function goToPage($url) {
$this->content = $this->browser->get($url);
return !empty($this->content);
}
function pattern($pattern) {
return preg_match("/$pattern/is",$this->content) > 0;
}
}

24. Dagfinn Reiersøl, ABC Startsiden 24
Using Cucumber with Webrat
• joomla.feature (Cucumber Gherkin business-readable)
• webrat_steps (Ruby code testing via HTTP)
Feature: Joomla welcome
In order to understand what Joomla is
As a potential user
I want to be able to read an introduction to Joomla
Scenario: Welcome page
Given I am on the home page
When I follow "Welcome to Joomla!"
Then I should see "Joomla! is a"

25. Dagfinn Reiersøl, ABC Startsiden 25
Webrat steps for Cucumber
• Ruby code runs Webrat which runs the web app using HTTP
Given /^I am on (.+)$/ do |page_name|
visit path_to(page_name)
end
When /^I follow "([^"]*)"$/ do |link|
click_link (link)
end
Then /^I should see "([^"]*)"$/ do |text|
response_body.should contain(text)
end

26. Dagfinn Reiersøl, ABC Startsiden 26
Avoiding difficult tests
• It's hard to test behaviors that cut across the HTTP request
• Improve architecture so these behaviors can't fail
• Example: duplication between input control name and HTTP
variable name

27. Dagfinn Reiersøl, ABC Startsiden 27
Hard to test: name duplication
<input type="text" name="address1"
value="<?= $address1 ?>">
// Oops...
echo $_POST['address_1'];

28. Dagfinn Reiersøl, ABC Startsiden 28
Getting rid of name duplication
• Use a form handling package
• Or do it yourself. Simplistic version here:
<?php
define('ADDRESS1_NAME', 'address1');
$address1_value = $_POST[ADDRESS1_NAME];
<html>
...
<input name="<?= ADDRESS1_NAME ?>" value="<?=
$address1_value ?>">

29. Dagfinn Reiersøl, ABC Startsiden 29
Refactoring tests
• Refactor or be doomed to drown in duplicate code
• Especially true of web tests
• Most important: eliminate duplication

30. Dagfinn Reiersøl, ABC Startsiden 30
BDD-style test names
class JoomlaSimpleTestTest extends WebTestCase {
function setUp() {
$this­>get('http://localhost/joomla');
}
function testFrontPageIsValid() {
$this­>assertResponse(200);
$this­>assertTitle('Welcome to the Frontpage');
$this­>assertText('Welcome to the Frontpage');
}
function testWelcomePageIsValid() {
$this­>click('Welcome to Joomla!');
$this­>assertPattern('/Joomla! is a free.*content
publishing system/is');
}
}

31. Dagfinn Reiersøl, ABC Startsiden 31
Using custom assertions instead
class JoomlaSimpleTestTest extends WebTestCase {
function testJoomla() {
$this­>get('http://localhost/joomla');
$this­>assertValidHomePage();
$this­>click('Welcome to Joomla!');
$this­>assertValidWelcomePage();
}
function assertValidHomePage() {
$msg = 'Error on home page';
$this­>assertResponse(200,$msg);
$this­>assertTitle('Welcome to the Frontpage',$msg);
$this­>assertText('Welcome to the Fontpage',$msg);
}
function assertValidWelcomePage() {
$this­>assertPattern(
'/Joomla! is a free.*content publishing system/is',
'Error on welcome page');
}

32. Dagfinn Reiersøl, ABC Startsiden 32
ComposedRegex
http://martinfowler.com/bliki/ComposedRegex.html
• This regex tolerates markup (or anything else) between the
content items
• Yes, I know the email regex is simplistic
$email = '.*w+@w+.w+.*';
$firstname = '.*w.*';
$lastname = '.*w.*';
$date = '.*20dd­dd­dd( |&nbsp;|T)dd:dd.*';
$regex = 'Email address:'. $email .
'Name: ' . $firstname . $lastname .
'Date:' . $date;

33. Dagfinn Reiersøl, ABC Startsiden 33
Re-using setup
• Refactor gradually as needed:
– When test class gets big, split it and...
– ...extract setup method into a separate class
– When setup is common to many tests...
– ...extract setup class into a separate file
class JoomlaTestSetup {
function setUp() {
$this­>get('http://localhost/joomla');
}
}
class JoomlaSimpleTestTest extends JoomlaTestSetup..

34. Dagfinn Reiersøl, ABC Startsiden 34
Useful types of classes in web
testing
• Scraper
• Domain Object
• Specification

35. Dagfinn Reiersøl, ABC Startsiden 35
Domain object test version
class UserTest extends WebTestCase {
function setUp() {
$this­>user = new TestUser(
'Jane','Doe','jane@example.com');
}
function testCanRegisterUser...
$this­>setFieldByName(
'firstname',
$this­>user­>firstname
);

36. Dagfinn Reiersøl, ABC Startsiden 36
Given-when-then in PHP
class JoomlaSimpleTestTest extends WebTestCase {
function setUp() {
$this­>pages = array(
'JoomlaHomePage' => 'http://localhost/joomla');
}
function testWelcomPageIsValid() {
$this­>givenIAmOn('JoomlaHomePage');
$this­>whenIFollow('Welcome to Joomla!');
$this­>thenIShouldSee('Joomla! is a free');
}
function givenIAmOn($name) {
$this­>get($this­>pages[$name]);
}
function whenIFollow($text) {
$this­>clickLink($text);
}
function thenIShouldSee($pattern) {
$this­>assertPattern("/$pattern/is");
}

37. Dagfinn Reiersøl, ABC Startsiden 37
Test bus
http://www.martinfowler.com/ieeeSoftware/testBus.pdf

38. Dagfinn Reiersøl, ABC Startsiden 38
Recommendations from Uncle
Bob
• Test the features through the test bus
• Test the user interface separately in isolation
• Run a few end-to-end tests to test the “plumbing”

39. Dagfinn Reiersøl, ABC Startsiden 39
Test bus
Where is the test bus? Some possbilities:
• Test the Model
• Test the Controller
• Test the server side of an Ajax or “thin server” application
• Test a public API

40. Dagfinn Reiersøl, ABC Startsiden 40
Test bus with presentation tests

41. Dagfinn Reiersøl, ABC Startsiden 41
Fake web service
• Easier to do when not
testing via HTTP
• Sensing/verification is
very hard to do

42. Dagfinn Reiersøl, ABC Startsiden 42
Self-initializing fake
http://martinfowler.com/bliki/SelfInitializingFake.html

43. Dagfinn Reiersøl, ABC Startsiden 43
Self-initializing fake
http://martinfowler.com/bliki/SelfInitializingFake.html