1. $I->wantTo(‘Test our
Software‘);

2. About me
• Susann Sgorzaly, 30
• Statistician and Software developer
• PHP developer for 3,5 years
• currently: Developer at ITEXIA GmbH
(on maternity leave)
@susgo
@susann_sg

3. Motivation
• Situation in the middle of 2017:
• quarterly releases
• before each release: weeks of manually testing and bug
fixing
• Problem:
• slow
• expensive
• late
• and …

4. Motivation

5. Motivation
• How to change it?
• analyse the situation (testing strategy):
• project manager clicks his/her way though the software
• NO test scenarios list/plan
à there was no strategy at this point
à first of all: list of test cases (Excel list)
• search a way to automate this and for the moment only this

6. Motivation
• Searching for know-how:
1. own experiences from previous company (QuoData
GmbH)
• reproduce the test cases by building full navigation
scenarios with CasperJS
• take screenshots of all relevant pages
• product owner can confirm these screenshots as correct
• look at differences of the screenshots in a later run –
confirm if they are correct
• we had a web platform to confirm automatically created
screenshots as correct

7. Motivation
var login = {
name: 'admin',
pass: 'admin',
};
casper.thenOpen('http://localhost');
casper.then(function() {
this.fill('#user-login-form', login);
});
casper.thenClick('#edit-submit');
casper.waitFor(function() {
return !this.getCurrentUrl().match(RegExp(
'/nach-dem-login'));
}, null, function() {
this.echo('[error] [casper] Failed to log
in with '
+ login.name + ':' + login.pass + '!',
'ERROR');
this.exit(1);
});

8. Motivation
Before After changes
Differences

9. Motivation
• Searching for know-how:
1. own experiences from previous company (QuoData
GmbH)
• Pros at first sight:
• result is easy to interpret
• notices design breaks
• confirmation by the product owner
• Cons at first sight:
• language / technology differs from product language
• manually confirmation of screenshots needed

10. Motivation
• Searching for know-how:
2. experiences from other company (portrino GmbH)
• Codeception

11. Motivation
<?php
$I = new AcceptanceTester($scenario);
$I->amOnPage('/site/login');
$I->see('Login', 'h1');
$I->wantTo('try to login as admin with correct credentials');
$I->fillField('input[name="LoginForm[username]"]', 'admin');
$I->fillField('input[name="LoginForm[password]"]', 'admin');
$I->click('login-button');
$I->dontSeeElement('#login-form');
$I->expectTo('see user info');
$I->see('Logout');

12. Motivation

13. Motivation
• Searching for know-how:
2. experiences from other company (portrino GmbH)
• Pros at first sight:
• PHP (uses the software language)
• simple to read
• possible simple to write and easy to maintain
• no manually confirmation needed
• Cons at first sight:
• no screenshots and therefore no design (“sexy
frontend”) confirmation by the product owner

14. Motivation

15. Motivation
…CODECEPTION

16. What Is Codeception?
• powerful testing framework written in PHP
• inspired by BDD
• only requirements are basic knowledge of PHP and the theory
of automated testing
• kept as simple as possible for any kind of users
• powered by PHPUnit
„Describe what you test and how you test it. Use PHP to write
descriptions faster.“

17. What Does Codeception Do?
• multiple approaches:
• acceptance tests
• functional tests
• unit tests

18. What Kind Of Tests? – Acceptance Tests
• browser emulation (Selenium / Webdriver)
• can have no knowledge of the technologies
• can test any website
• testing done from a non-technical person
• readable by humans (managers)
• tests JavaScript / Ajax
• stability against code changes
• SLOW!

19. What Kind Of Tests? – Functional Tests
• web request and submit to application emulation
• assert against response and internal values
• the person testing knows how the software works
• uses different request variables to ensure the functionality of
the software for nearly all cases
• framework-based
• still readable by humans
• can‘t test JavaScript / Ajax
• less slow

20. What Kind Of Tests? – Unit Tests
• test the application core functions
• single isolated tests
• the testing person knows the internals of the application
• only readable by IT professionals
• fastest!

21. Codeception: How to? - Installation
• Install via composer
composer require “codeception/codeception“ --dev
• Execute it as:
./vendor/bin/codecept

22. Codeception: How to? - Setup
• Execute
./vendor/bin/codecept bootstrap
à creates configuration file codeception.yml and tests directory
and default test suites: acceptance, functional, unit
app
| -- tests
| | -- accpeptance
| | -- functional
| | -- unit
| | -- accpeptance.suite.yml
| | -- functional.suite.yml
| | -- unit.suite.yml
| -- codeception.yml
..

23. Codeception: How to? - Configuration
• example: configure acceptance test suite
• edit configuration file: tests/acceptance.suite.yml
actor: AcceptanceTester
modules:
enabled:
- PhpBrowser:
url: {YOUR APP'S URL}
- HelperAcceptance

24. Side Note: Actors and Modules?
• test classes use Actors to perform actions à act as a dummy user
• actor classes are generated from the suite configuration
• methods of actor classes are taken from Codeception Modules
• each module provides predefined actions for different testing
purposes
• modules can be combined to fit the testing environment
actor: AcceptanceTester
modules:
enabled:
…

25. Side Note: PHP Browser?
• doesn’t require running an actual browser
• runs in any environment: only PHP and cURL are required
• uses a PHP web scraper, which acts like a browser: sends a
request, receives and parses the response
• can’t work with JS (modals, datepickers, …)
• can’t test actual visibility of elements
• …
• fastest way to run acceptance tests
• But: not the situation a manager or customer is in

26. Side Note: Selenium WebDriver
• requires running an actual browser (Firefox, Chrome, …)
• can work with JS (modals, datepickers, …)
• can test actual visibility of elements
• …
• slower
• but: a manager or customer uses it too

27. Side Note: PhpBrowser vs WebDriver?
• doesn’t matter what you choose at the beginning
• most tests can be easily ported between the testing backends
• PhpBrowser tests can be executed inside a real browser with
Selenium WebDriver
• you only have to change the acceptance test suite
configuration file module and rebuild the AcceptanceTester
class

28. Codeception: How to? - Configuration
• Example: configure acceptance test suite
• Edit configuration file: tests/acceptance.suite.yml
• minimum: add your application url
actor: AcceptanceTester
modules:
enabled:
- PhpBrowser:
url: {YOUR APP'S URL}
- HelperAcceptance

29. Codeception: How to? - Generate Test
• Execute
./vendor/bin/codecept generate:cest acceptance Login
àgenerates new php class file
LoginCest.php for class LoginCest in the folder
‚tests/acceptance‘
Note: Cest is the class based format. Codeception also supports Cept which is a
scenario based format (see example from the Motivation slides)

30. Codeception: How to? - Write Test
<?php
class LoginCest
{
public function tryToLoginAsAdmin(AcceptanceTester $I)
{
$I->amOnPage('/site/login');
$I->see('Login', 'h1');
$I->wantTo('try to login as admin with correct credentials');
$I->fillField('input[name="LoginForm[username]"]', 'admin');
$I->fillField('input[name="LoginForm[password]"]', 'admin');
$I->click('login-button');
$I->dontSeeElement('#login-form');
$I->expectTo('see user info');
$I->see('Logout');
}
}

31. Codeception: How to? - Write Test
• Actions
$I->fillField('input[name="LoginForm[username]"]', 'admin');
$I->click('login-button');
• Assertions
$I->see('Login', 'h1');
$I->dontSeeElement('#login-form');
$I->see('Logout');

32. Codeception: How to? - Run Test!
./vendor/bin/codecept run --steps
./vendor/bin/codecept run --debug
./vendor/bin/codecept run acceptance
./vendor/bin/codecept run acceptance
LoginCest:ensureThatLoginWorks
./vendor/bin/codecept run
tests/acceptance/LoginCest.php::ensureThatLoginWorks
./vendor/bin/codecept run --xml

33. Codeception: How to? - Run Test!

34. Codeception: Basic Features
• multiple backends, easily changed in configuration
• Selenium, PhpBrowser, PhantomJS
• elements matched by name, CSS, XPath
• data clean-up after each run
• integrates with different frameworks (e.g. Symfony2, Yii2,
Laravel)
• Dependency Injection

35. Codeception: Basic Features
• executes PHPUnit tests
• BDD-style
• WebService testing via REST and SOAP is possible
• generates reports: HTML, XML, JSON
• Fixtures (known test data)
• Database helpers
• Code Coverage

36. Codeception - solves all my problems!
YEAH!!!!
Seems to be easy. Let‘s go home and write some tests…

37. Codeception - solves all my problems?

38. Codeception - solves all my problems?
NO!
My tests often broke!
My data are unstable!
So many tests,
so less structure!
NO!
NO!
NO!NO!
NO!
NO!
NO!
NO!
NO!
NO!
NO!

39. Best practice. My tests often broke
• use ids
• use the right / a good selector (CSS, name, XPath, label)
• use constants: PageObjects in Codeception

40. Best practice. My tests often broke
• PageObjects:
• represents a web page as a class
• the DOM elements on that page are its properties
• some basic interactions are its methods
• example:
./vendor/bin/codecept generate:pageobject Login

41. Best practice. My tests often broke
class LoginCest
{
public function tryToLoginAsAdmin(AcceptanceTester $I)
{
$I->amOnPage('/site/login');
$I->see('Login', 'h1');
$I->wantTo('try to login as admin with correct credentials');
$I->fillField('input[name="LoginForm[username]"]', 'admin');
$I->fillField('input[name="LoginForm[password]"]', 'admin');
$I->click('login-button');
$I->dontSeeElement('#login-form');
$I->expectTo('see user info');
$I->see('Logout');
}
}

42. Best practice. My tests often broke
namespace Page;
class Login
{
// include url of current page
public static $URL = '/site/login';
/**
* Declare UI map for this page here. CSS or XPath allowed.
*/
public static $form = '#login-form';
public static $usernameField = 'input[name="LoginForm[username]"]';
public static $passwordField = 'input[name="LoginForm[password]"]';
public static $formSubmitButton = 'login-button';
}

43. Best practice. My tests often broke
class LoginCest
{
public function tryToLoginAsAdmin(AcceptanceTester $I, PageLogin $loginPage)
{
$I->amOnPage($loginPage::$URL);
$I->seeElement($loginPage::$form);
$I->wantTo('try to login as admin with correct credentials');
$I->fillField($loginPage::$usernameField, 'admin');
$I->fillField($loginPage::$passwordField, 'admin');
$I->click($loginPage::$formSubmitButton);
$I->dontSeeElement($loginPage::$form);
$I->expectTo('see user info');
$I->see('Logout');
}
}

44. Best practice. My data are unstable
• try to search for stable elements on the website
• use fixtures instead of database dumps
• fixtures:
• sample data for tests
• data can be either generated, loaded from an array or taken
from a sample database
• usage with Db module:
$I->haveInDatabase('posts', array('title' => My title', 'body' => The body.'));

45. Best practice. My data are unstable
• use DataFactory module to generate the test data dynamically
• uses an ORM of your application to define, save and
cleanup data
• should be used with ORM or Framework modules
• requires package: "league/factory-muffin“

46. Best practice. My tests often broke
<?php
use LeagueFactoryMuffinFakerFacade as Faker;
$fm->define(User::class)->setDefinitions([
'name' => Faker::name(),
// generate email
'email' => Faker::email(),
'body' => Faker::text(),
// generate a profile and return its Id
'profile_id' => 'factory|Profile'
]);
• Generation rules can be defined in a factories file

47. Best practice. My tests often broke
modules:
enabled:
- Yii2:
configFile: path/to/config.php
- DataFactory:
factories: tests/_support/factories
depends: Yii2
• load factory definitions from a directory

48. Best practice. My data are unstable
• DataFactory module actions
• generate and save record:
$I->have('User');
$I->have('User', ['is_active' => true]);
• generate multiple records and save them:
$I->haveMultiple('User', 10);
• generate records (without saving):
$I->make('User');

49. Best practice. So many tests, so less
structure
• test code is source code and therefore should follow the same
rules
• reuse code
• extend AcceptanceTester class located inside the
tests/_support directory
• use StepObjects
• use PageObjects

50. Best practice. So many tests, so less
structure – Extend AcceptanceTester
class AcceptanceTester extends CodeceptionActor
{
// do not ever remove this line!
use _generatedAcceptanceTesterActions;
public function login($name, $password)
{
$I = $this;
$I->amOnPage('/site/login');
$I->submitForm('#login-form', [
'input[name="LoginForm[username]"]' => $name,
'input[name="LoginForm[password]"]' => $password
]);
$I->see($name, 'Logout');
}
}

51. Best practice. So many tests, so less
structure - StepObjects
• groups some common functionality for a group of tests
(for testing a part of a software: admin area, …)
• extends AcceptanceTester class
• example:
./vendor/bin/codecept generate:stepobject Admin
à generate a class in tests/_support/Step/Acceptance/Admin.php

52. Best practice. So many tests, so less
structure - StepObjects
namespace StepAcceptance;
class Admin extends AcceptanceTester
{
public function loginAsAdmin()
{
$I = $this;
// do login
}
}

53. Best practice. So many tests, so less
structure - StepObjects
class TestCest
{
function tryToTest(StepAcceptanceAdmin $I)
{
$I->loginAsAdmin();
// test something
}
}

54. Codeception: Nice to have
• session snapshots (for faster execution)
• share cookies between tests
• e.g. test user can stay logged in for other tests:
• $I->saveSessionSnapshot('login‘);
• $I->loadSessionSnapshot('login');
• group tests with @group annotation for test methods:
• e.g. @group important
• ./vendor/bin/codecept run acceptance -g important

55. Codeception: Nice to have
• before/after annotations
• example annotations
• dataProvider annotations
• environments
• dependencies
• multi session testing
• …

56. Codeception: Summary

57. Codeception: Summary
• easy for developers but difficult for product owner
• layout tests missing
• should we use another tool?

58. Codeception: Summary
• Codeception is extendable
• solution: use module Visualception
• see https://github.com/Codeception/VisualCeption for more
information