Presentation goes through each type of automated tests. Provides a definition, particular qualities and examples of implementation by Magento team.

1. AUTOMATED TESTS
SERGII IVASHCHENKO
ON EXAMPLE OF MAGENTO 2

2. AUTOMATED TESTS BY @SERGEIVASCHENKO
WHOAMI
▸ PHP Developer from Ukraine
▸ Ex Magento Core Developer
▸ Magento Testing Framework
▸ Created a lot of tests
▸ sivaschenko.com
▸ @sergeivaschenko

3. AUTOMATED TESTS BY @SERGEIVASCHENKO
WHY MAGENTO 2 IS A GOOD EXAMPLE
▸ Big and versatile project
▸ More a solution then a framework
▸ Many tests of all types
▸ Open Source https://github.com/magento/
magento2

4. AUTOMATED TESTS BY @SERGEIVASCHENKO
THE PLAN
▸ Go through each type of test
▸ Definition
▸ Compare
▸ Magento 2
▸ Alternatives
▸ Your current project

5. AUTOMATED TESTS BY @SERGEIVASCHENKO
FACTORS FOR COMPARISON
▸ Code coverage accuracy
▸ Functionality coverage accuracy
▸ Creation efforts (complexity x time)
▸ Maintenance efforts (sensitivity + stability)

6. AUTOMATED TESTS BY @SERGEIVASCHENKO
UNIT
▸ Testing one class in isolation
▸ Precise code coverage
▸ Lots of routine work to create
▸ Very sensitive to code changes
▸ Magento 2: pure PHPUnit + easy dependency
mocking

7. AUTOMATED TESTS BY @SERGEIVASCHENKO
EXAMPLES
▸ Test case: https://github.com/magento/magento2/blob/
develop/app/code/Magento/Catalog/Test/Unit/Block/
NavigationTest.php
▸ Configuration: https://github.com/magento/magento2/
blob/develop/dev/tests/unit/phpunit.xml.dist
▸ Test Framework Object Manager: https://github.com/
magento/magento2/blob/develop/lib/internal/Magento/
Framework/TestFramework/Unit/Helper/
ObjectManager.php

8. AUTOMATED TESTS BY @SERGEIVASCHENKO
INTEGRATION
▸ Testing method call without any code isolation
▸ Between unit and functional
▸ Often complicated preconditions
▸ Not sensitive to low level code changes (but to app
state)
▸ Magento 2: fixtures, db and app isolation, abstract
layer for controllers

9. AUTOMATED TESTS BY @SERGEIVASCHENKO
EXAMPLES
▸ Test case: https://github.com/magento/magento2/blob/
develop/dev/tests/integration/testsuite/Magento/Catalog/
Controller/ProductTest.php
▸ Annotations: https://github.com/magento/magento2/tree/
develop/dev/tests/integration/framework/Magento/
TestFramework/Annotation
▸ Abstract Layer: https://github.com/magento/magento2/
tree/develop/dev/tests/integration/framework/Magento/
TestFramework/TestCase

10. AUTOMATED TESTS BY @SERGEIVASCHENKO
WEB API
▸ Web API request/response validation
▸ Between integration and functional
▸ Requires preconditions
▸ Not sensitive to low level code changes (but to
app state)
▸ Magento 2: abstract layer

11. AUTOMATED TESTS BY @SERGEIVASCHENKO
EXAMPLES
▸ Test case: https://github.com/magento/
magento2/blob/develop/dev/tests/api-
functional/testsuite/Magento/Catalog/Api/
AttributeSetRepositoryTest.php
▸ Abstract: https://github.com/magento/
magento2/blob/develop/dev/tests/api-
functional/framework/Magento/TestFramework/
TestCase/WebapiAbstract.php

12. AUTOMATED TESTS BY @SERGEIVASCHENKO
FUNCTIONAL
▸ Browser scenarios
▸ Precise functionality coverage
▸ Efforts = amount of JS
▸ Sensitive to interface changes
▸ Magento 2: MTF https://github.com/magento/
mtf

13. AUTOMATED TESTS BY @SERGEIVASCHENKO
EXAMPLES
▸ Test case: https://github.com/magento/
magento2/blob/develop/dev/tests/functional/
tests/app/Magento/Customer/Test/TestCase/
CreateCustomerGroupEntityTest.php
▸ Test case config: https://github.com/magento/
magento2/blob/develop/dev/tests/functional/
tests/app/Magento/Customer/Test/TestCase/
CreateCustomerGroupEntityTest.xml

14. Codecoverageaccuracy
0
2
4
6
8
Functionality coverage accuracy
0 2 4 6 8
Unit Integration Web API
Functional

15. AUTOMATED TESTS BY @SERGEIVASCHENKO
STATIC
▸ Do not require functional code execution
▸ Can be reused between projects/companies/etc
▸ Code standards, typos, potential errors, complexity,
dependency issues
▸ Use those right now: phpmd, phpcs, phpcpd, php-
cs-fixer, pdepend, phpcbf, etc (and even PHPSorm
code inspection)

16. AUTOMATED TESTS BY @SERGEIVASCHENKO
EXAMPLES
▸ Copy-paste, mess, standards test: https://
github.com/magento/magento2/blob/develop/
dev/tests/static/testsuite/Magento/Test/Php/
LiveCodeTest.php
▸ Obsolete references: https://github.com/
magento/magento2/blob/develop/dev/tests/
static/testsuite/Magento/Test/Legacy/_files/
obsolete_config_nodes.php

17. AUTOMATED TESTS BY @SERGEIVASCHENKO
PERFORMANCE
▸ Tests result depends on execution time, requests
throughput
▸ Tests on system loaded with data
▸ Magento 2:
▸ Performance fixtures (data generation)
▸ New Relic reporting

18. AUTOMATED TESTS BY @SERGEIVASCHENKO
EXAMPLES
▸ Profiles for data generation: https://
github.com/magento/magento2/tree/
develop/setup/performance-toolkit/profiles/
ce
▸ JMeter test suite: https://github.com/
magento/magento2/blob/develop/setup/
performance-toolkit/benchmark.jmx

19. AUTOMATED TESTS BY @SERGEIVASCHENKO
INTERESTING READ
▸ PHP Native Mocking https://medium.com/
@IvanChepurnyi/native-mocking-and-
stubbing-in-php-ad11a32596e4
▸ Magento automated tests by Magento VP of
Architecture: https://alankent.me/
2014/06/28/magento-2-test-automation/

20. THANK YOU!
@SERGEIVASCHENK