Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Testing mit Codeception: Full-stack testing PHP framework

62 views

Published on

Codeception is a PHP testing framework for Behavior Driven Development, which covers all kinds of tests: unit tests, functional tests and acceptance tests. It is fast and simple in both usage and execution. This talk will give you a introduction to the software testing basics using codeception. It will also cover some stumbling blocks when writing tests, like:
- Test code stability against small changes
- Data stability
- Test structure
Last but not least I will give you a short outlook how to make your tests also understandable for product owners.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Testing mit Codeception: Full-stack testing PHP framework

  1. 1. $I->wantTo(‘Test our Software‘);
  2. 2. About me • Susann Sgorzaly, 30 • Statistician and Software developer • PHP developer for 3,5 years • currently: Developer at ITEXIA GmbH (on maternity leave) @susgo @susann_sg
  3. 3. Motivation • Situation in the middle of 2017: • quarterly releases • before each release: weeks of manually testing and bug fixing • Problem: • slow • expensive • late • and …
  4. 4. Motivation
  5. 5. Motivation • How to change it? • analyse the situation (testing strategy): • project manager clicks his/her way though the software • NO test scenarios list/plan à there was no strategy at this point à first of all: list of test cases (Excel list) • search a way to automate this and for the moment only this
  6. 6. Motivation • Searching for know-how: 1. own experiences from previous company (QuoData GmbH) • reproduce the test cases by building full navigation scenarios with CasperJS • take screenshots of all relevant pages • product owner can confirm these screenshots as correct • look at differences of the screenshots in a later run – confirm if they are correct • we had a web platform to confirm automatically created screenshots as correct
  7. 7. Motivation var login = { name: 'admin', pass: 'admin', }; casper.thenOpen('http://localhost'); casper.then(function() { this.fill('#user-login-form', login); }); casper.thenClick('#edit-submit'); casper.waitFor(function() { return !this.getCurrentUrl().match(RegExp( '/nach-dem-login')); }, null, function() { this.echo('[error] [casper] Failed to log in with ' + login.name + ':' + login.pass + '!', 'ERROR'); this.exit(1); });
  8. 8. Motivation Before After changes Differences
  9. 9. Motivation • Searching for know-how: 1. own experiences from previous company (QuoData GmbH) • Pros at first sight: • result is easy to interpret • notices design breaks • confirmation by the product owner • Cons at first sight: • language / technology differs from product language • manually confirmation of screenshots needed
  10. 10. Motivation • Searching for know-how: 2. experiences from other company (portrino GmbH) • Codeception
  11. 11. Motivation <?php $I = new AcceptanceTester($scenario); $I->amOnPage('/site/login'); $I->see('Login', 'h1'); $I->wantTo('try to login as admin with correct credentials'); $I->fillField('input[name="LoginForm[username]"]', 'admin'); $I->fillField('input[name="LoginForm[password]"]', 'admin'); $I->click('login-button'); $I->dontSeeElement('#login-form'); $I->expectTo('see user info'); $I->see('Logout');
  12. 12. Motivation
  13. 13. Motivation • Searching for know-how: 2. experiences from other company (portrino GmbH) • Pros at first sight: • PHP (uses the software language) • simple to read • possible simple to write and easy to maintain • no manually confirmation needed • Cons at first sight: • no screenshots and therefore no design (“sexy frontend”) confirmation by the product owner
  14. 14. Motivation
  15. 15. Motivation …CODECEPTION
  16. 16. What Is Codeception? • powerful testing framework written in PHP • inspired by BDD • only requirements are basic knowledge of PHP and the theory of automated testing • kept as simple as possible for any kind of users • powered by PHPUnit „Describe what you test and how you test it. Use PHP to write descriptions faster.“
  17. 17. What Does Codeception Do? • multiple approaches: • acceptance tests • functional tests • unit tests
  18. 18. What Kind Of Tests? – Acceptance Tests • browser emulation (Selenium / Webdriver) • can have no knowledge of the technologies • can test any website • testing done from a non-technical person • readable by humans (managers) • tests JavaScript / Ajax • stability against code changes • SLOW!
  19. 19. What Kind Of Tests? – Functional Tests • web request and submit to application emulation • assert against response and internal values • the person testing knows how the software works • uses different request variables to ensure the functionality of the software for nearly all cases • framework-based • still readable by humans • can‘t test JavaScript / Ajax • less slow
  20. 20. What Kind Of Tests? – Unit Tests • test the application core functions • single isolated tests • the testing person knows the internals of the application • only readable by IT professionals • fastest!
  21. 21. Codeception: How to? - Installation • Install via composer composer require “codeception/codeception“ --dev • Execute it as: ./vendor/bin/codecept
  22. 22. Codeception: How to? - Setup • Execute ./vendor/bin/codecept bootstrap à creates configuration file codeception.yml and tests directory and default test suites: acceptance, functional, unit app | -- tests | | -- accpeptance | | -- functional | | -- unit | | -- accpeptance.suite.yml | | -- functional.suite.yml | | -- unit.suite.yml | -- codeception.yml ..
  23. 23. Codeception: How to? - Configuration • example: configure acceptance test suite • edit configuration file: tests/acceptance.suite.yml actor: AcceptanceTester modules: enabled: - PhpBrowser: url: {YOUR APP'S URL} - HelperAcceptance
  24. 24. Side Note: Actors and Modules? • test classes use Actors to perform actions à act as a dummy user • actor classes are generated from the suite configuration • methods of actor classes are taken from Codeception Modules • each module provides predefined actions for different testing purposes • modules can be combined to fit the testing environment actor: AcceptanceTester modules: enabled: …
  25. 25. Side Note: PHP Browser? • doesn’t require running an actual browser • runs in any environment: only PHP and cURL are required • uses a PHP web scraper, which acts like a browser: sends a request, receives and parses the response • can’t work with JS (modals, datepickers, …) • can’t test actual visibility of elements • … • fastest way to run acceptance tests • But: not the situation a manager or customer is in
  26. 26. Side Note: Selenium WebDriver • requires running an actual browser (Firefox, Chrome, …) • can work with JS (modals, datepickers, …) • can test actual visibility of elements • … • slower • but: a manager or customer uses it too
  27. 27. Side Note: PhpBrowser vs WebDriver? • doesn’t matter what you choose at the beginning • most tests can be easily ported between the testing backends • PhpBrowser tests can be executed inside a real browser with Selenium WebDriver • you only have to change the acceptance test suite configuration file module and rebuild the AcceptanceTester class
  28. 28. Codeception: How to? - Configuration • Example: configure acceptance test suite • Edit configuration file: tests/acceptance.suite.yml • minimum: add your application url actor: AcceptanceTester modules: enabled: - PhpBrowser: url: {YOUR APP'S URL} - HelperAcceptance
  29. 29. Codeception: How to? - Generate Test • Execute ./vendor/bin/codecept generate:cest acceptance Login àgenerates new php class file LoginCest.php for class LoginCest in the folder ‚tests/acceptance‘ Note: Cest is the class based format. Codeception also supports Cept which is a scenario based format (see example from the Motivation slides)
  30. 30. Codeception: How to? - Write Test <?php class LoginCest { public function tryToLoginAsAdmin(AcceptanceTester $I) { $I->amOnPage('/site/login'); $I->see('Login', 'h1'); $I->wantTo('try to login as admin with correct credentials'); $I->fillField('input[name="LoginForm[username]"]', 'admin'); $I->fillField('input[name="LoginForm[password]"]', 'admin'); $I->click('login-button'); $I->dontSeeElement('#login-form'); $I->expectTo('see user info'); $I->see('Logout'); } }
  31. 31. Codeception: How to? - Write Test • Actions $I->fillField('input[name="LoginForm[username]"]', 'admin'); $I->click('login-button'); • Assertions $I->see('Login', 'h1'); $I->dontSeeElement('#login-form'); $I->see('Logout');
  32. 32. Codeception: How to? - Run Test! ./vendor/bin/codecept run --steps ./vendor/bin/codecept run --debug ./vendor/bin/codecept run acceptance ./vendor/bin/codecept run acceptance LoginCest:ensureThatLoginWorks ./vendor/bin/codecept run tests/acceptance/LoginCest.php::ensureThatLoginWorks ./vendor/bin/codecept run --xml
  33. 33. Codeception: How to? - Run Test!
  34. 34. Codeception: Basic Features • multiple backends, easily changed in configuration • Selenium, PhpBrowser, PhantomJS • elements matched by name, CSS, XPath • data clean-up after each run • integrates with different frameworks (e.g. Symfony2, Yii2, Laravel) • Dependency Injection
  35. 35. Codeception: Basic Features • executes PHPUnit tests • BDD-style • WebService testing via REST and SOAP is possible • generates reports: HTML, XML, JSON • Fixtures (known test data) • Database helpers • Code Coverage
  36. 36. Codeception - solves all my problems! YEAH!!!! Seems to be easy. Let‘s go home and write some tests…
  37. 37. Codeception - solves all my problems?
  38. 38. Codeception - solves all my problems? NO! My tests often broke! My data are unstable! So many tests, so less structure! NO! NO! NO!NO! NO! NO! NO! NO! NO! NO! NO!
  39. 39. Best practice. My tests often broke • use ids • use the right / a good selector (CSS, name, XPath, label) • use constants: PageObjects in Codeception
  40. 40. Best practice. My tests often broke • PageObjects: • represents a web page as a class • the DOM elements on that page are its properties • some basic interactions are its methods • example: ./vendor/bin/codecept generate:pageobject Login
  41. 41. Best practice. My tests often broke class LoginCest { public function tryToLoginAsAdmin(AcceptanceTester $I) { $I->amOnPage('/site/login'); $I->see('Login', 'h1'); $I->wantTo('try to login as admin with correct credentials'); $I->fillField('input[name="LoginForm[username]"]', 'admin'); $I->fillField('input[name="LoginForm[password]"]', 'admin'); $I->click('login-button'); $I->dontSeeElement('#login-form'); $I->expectTo('see user info'); $I->see('Logout'); } }
  42. 42. Best practice. My tests often broke namespace Page; class Login { // include url of current page public static $URL = '/site/login'; /** * Declare UI map for this page here. CSS or XPath allowed. */ public static $form = '#login-form'; public static $usernameField = 'input[name="LoginForm[username]"]'; public static $passwordField = 'input[name="LoginForm[password]"]'; public static $formSubmitButton = 'login-button'; }
  43. 43. Best practice. My tests often broke class LoginCest { public function tryToLoginAsAdmin(AcceptanceTester $I, PageLogin $loginPage) { $I->amOnPage($loginPage::$URL); $I->seeElement($loginPage::$form); $I->wantTo('try to login as admin with correct credentials'); $I->fillField($loginPage::$usernameField, 'admin'); $I->fillField($loginPage::$passwordField, 'admin'); $I->click($loginPage::$formSubmitButton); $I->dontSeeElement($loginPage::$form); $I->expectTo('see user info'); $I->see('Logout'); } }
  44. 44. Best practice. My data are unstable • try to search for stable elements on the website • use fixtures instead of database dumps • fixtures: • sample data for tests • data can be either generated, loaded from an array or taken from a sample database • usage with Db module: $I->haveInDatabase('posts', array('title' => My title', 'body' => The body.'));
  45. 45. Best practice. My data are unstable • use DataFactory module to generate the test data dynamically • uses an ORM of your application to define, save and cleanup data • should be used with ORM or Framework modules • requires package: "league/factory-muffin“
  46. 46. Best practice. My tests often broke <?php use LeagueFactoryMuffinFakerFacade as Faker; $fm->define(User::class)->setDefinitions([ 'name' => Faker::name(), // generate email 'email' => Faker::email(), 'body' => Faker::text(), // generate a profile and return its Id 'profile_id' => 'factory|Profile' ]); • Generation rules can be defined in a factories file
  47. 47. Best practice. My tests often broke modules: enabled: - Yii2: configFile: path/to/config.php - DataFactory: factories: tests/_support/factories depends: Yii2 • load factory definitions from a directory
  48. 48. Best practice. My data are unstable • DataFactory module actions • generate and save record: $I->have('User'); $I->have('User', ['is_active' => true]); • generate multiple records and save them: $I->haveMultiple('User', 10); • generate records (without saving): $I->make('User');
  49. 49. Best practice. So many tests, so less structure • test code is source code and therefore should follow the same rules • reuse code • extend AcceptanceTester class located inside the tests/_support directory • use StepObjects • use PageObjects
  50. 50. Best practice. So many tests, so less structure – Extend AcceptanceTester class AcceptanceTester extends CodeceptionActor { // do not ever remove this line! use _generatedAcceptanceTesterActions; public function login($name, $password) { $I = $this; $I->amOnPage('/site/login'); $I->submitForm('#login-form', [ 'input[name="LoginForm[username]"]' => $name, 'input[name="LoginForm[password]"]' => $password ]); $I->see($name, 'Logout'); } }
  51. 51. Best practice. So many tests, so less structure - StepObjects • groups some common functionality for a group of tests (for testing a part of a software: admin area, …) • extends AcceptanceTester class • example: ./vendor/bin/codecept generate:stepobject Admin à generate a class in tests/_support/Step/Acceptance/Admin.php
  52. 52. Best practice. So many tests, so less structure - StepObjects namespace StepAcceptance; class Admin extends AcceptanceTester { public function loginAsAdmin() { $I = $this; // do login } }
  53. 53. Best practice. So many tests, so less structure - StepObjects class TestCest { function tryToTest(StepAcceptanceAdmin $I) { $I->loginAsAdmin(); // test something } }
  54. 54. Codeception: Nice to have • session snapshots (for faster execution) • share cookies between tests • e.g. test user can stay logged in for other tests: • $I->saveSessionSnapshot('login‘); • $I->loadSessionSnapshot('login'); • group tests with @group annotation for test methods: • e.g. @group important • ./vendor/bin/codecept run acceptance -g important
  55. 55. Codeception: Nice to have • before/after annotations • example annotations • dataProvider annotations • environments • dependencies • multi session testing • …
  56. 56. Codeception: Summary
  57. 57. Codeception: Summary • easy for developers but difficult for product owner • layout tests missing • should we use another tool?
  58. 58. Codeception: Summary • Codeception is extendable • solution: use module Visualception • see https://github.com/Codeception/VisualCeption for more information

×