This document introduces the Magento Functional Testing Framework (MFTF). MFTF allows developers to write automated functional tests for Magento using XML. It is based on the Codeception framework and allows testing Magento features in a reliable, maintainable and performant way. The document provides instructions on setting up MFTF and running a sample test to create a product, check it is found in search results, and then delete the product.

1. Hi!
I’m Stan
https://twitter.com/st4nsm

2. MFTF
Introduction to
Magento Functional Testing Framework

3. Why perform
functional testing?

4. MFTF
• Reliable
• Maintainable
• Easy to use
• Performant.

5. Why write tests in XML?
• Fabian Schmangler’s thoughts on this topic:
https://www.schmengler-se.de/en/2017/06/my-opinion-on-xml-based-testing-in-magento/

6. Why MFTF?
• Based on a well-established framework: Codeception
• Built for Magento
• Scalability
• Automatic XML processing by custom tools
• Widely supported (browserstack.com, etc)

7. Flow
XML PHP
Codeception tests
Codeception/
Robo
Selenium
Driver

8. Environment Setup
Follow instructions from the official documentation:
http://devdocs.magento.com/guides/v2.2/magento-functional-testing-framework/release-1/getting-started.html
Install Selenium, Chrome driver and Allure CLI
Don’t forget to modify dev/tests/acceptance/.env:
MAGENTO_BASE_URL=http://msi.hom/
MAGENTO_BACKEND_NAME=admin
MAGENTO_ADMIN_USERNAME=admin
MAGENTO_ADMIN_PASSWORD=admin123

9. Basic Concepts
• Cest
• Page
• Section
• Data

10. Magento Search test
• Create a simple product
• Check that we can find this product
• Delete the product

12. Creating a product
<before>
<createData entity="SearchableProduct" stepKey="searchableProduct"/>
<amOnPage url="{{HomePage.url}}" stepKey="amOnPage"/>
</before>

13. Annotate the test
<annotations>
<title value="Simple Search Test"/>
<description value="Creates a simple product, checks that it displays
in the search results"/>
<group value="search"/>
</annotations>

14. Actual test
<fillField stepKey="enterSearchQuery"
selector="{{SearchFormSection.query}}" userInput="$$searchableProduct.name$
$"/>
<click stepKey="clickSearchButton"
selector="{{SearchFormSection.searchButton}}"/>
<waitForPageLoad stepKey="waitPageLoad"/>
<see stepKey="checkFirstProductTitle"
selector="{{SearchResultsSection.firstProductTitle}}" userInput="$
$searchableProduct.name$$"/>
<see stepKey="checkPageTitle" selector="{{SearchResultsSection.pageTitle}}"
userInput=“Search results for: '$$searchableProduct.name$$'"/>

15. Remove the product
<after>
<deleteData createDataKey="searchableProduct" stepKey="deleteProduct"/>
</after>

16. <section name="SearchFormSection">
<element name="query" type="input" selector="#search"/>
<element name="searchButton" type="button" selector="#search_mini_form button[type=submit]"/>
</section>
<section name="SearchResultsSection">
<element name="pageTitle" type="text" selector="h1.page-title"/>
<element name="firstProductTitle" type="text" selector=".product-items>.product-item:first-child a.product-item-link"/>
</section>
Sections

17. <entity name="SearchableProduct" type="product">
<data key="sku" unique="suffix">SearchableProduct</data>
<data key="type_id">simple</data>
<data key="attribute_set_id">4</data>
<data key="name" unique="suffix">SearchableProduct</data>
<data key="price">123.00</data>
<data key="visibility">4</data>
<data key="status">1</data>
<data key="qty">123</data>
<required-entity type="product_extension_attribute">EavStockItem</required-entity>
</entity>
Example type definitions:
CatalogMetadata
Data

18. Generate the tests
$ robo generate:tests

19. Let’s run it!

20. Ways to run tests
• Using Robo:
robo group search
• Using Codeception:
./vendor/bin/codecept run functional --verbose --steps --skip-group skip --group
search

21. Test Reports
• vendor/bin/robo allure2:generate
• vendor/bin/robo allure2:open

22. Generated Allure Report

23. Try it yourself
• Link to GitHub repo:
https://github.com/stansm/mftf-introduction

24. References
• http://devdocs.magento.com/guides/v2.2/magento-functional-testing-framework/release-1/introduction.html
• http://codeception.com/
• https://github.com/allure-framework/allure2
• https://www.schmengler-se.de/en/2017/06/my-opinion-on-xml-based-testing-in-magento/
• http://codeception.com/docs/reference/Locator