Behavior driven development with OPA5 automated tests
1. András Kónya, Lóránt Guth SAP Labs Hungary
June 30, 2017
Behavior Driven
Development with
OPA5 Automated Tests
2. 2
„I remember thinking “If only someone had told me that!” far more often
than I thought “Wow, a door has opened.””
Dan North
Benefits of Using BDD
Reduces the communication gap between business and tech people
Improves product quality
Less customer complaints
More progress & less stress for the team
7. 9
Business readable-writeable language
Cucumber understands and executes it
Based on the Given-When-Then pattern
Gherkin
Feature: Application test
Background: Open the application
Scenario: Create a new entity
Given Write MyAttribute to name field
When Click on Create button
Then Check entity with name MyAttribute
Cucumber
Framework
Product
Owner
OPA5
QUnit
Gherkin
Developer
JavaScript
8. 10
Cucumber
OPA5
Gherkin
Specification tool
–Uses Gherkin to describe the behaviour
Not a testing framework!
–JUnit, TestNG, QUnit are testing frameworks
Uses Step Definitions to translate Gherkin steps
Cucumber Cucumber
Framework
Product
Owner
OPA5
QUnit
Gherkin
Developer
JavaScript
this.register(
/^Click on the (.*) button$/i,
function(sButtonName, oPageObjects) {
oPageObjects.onAttributePage.iClickOnButton(sButtonName);
}
);
9. 11
QUnit based API for easy access and manipulation of UI5
controls
Helpful for writing automated tests for UI5 applications
OPA5
iClickOnButton: function(sButtonName) {
return this.waitFor({
controlType: "sap.m.Button",
viewName: viewName,
matchers: function($node) {
if ($node.getText() === sButtonName) { return true; }
return false;
},
success: function(oButtons) { this.pressOnButton(oButtons[0]); },
errorMessage: "Failed to click on the " + sButtonName + " button."
});
},
Cucumber
Framework
Product
Owner
OPA5
QUnit
Gherkin
Developer
JavaScript
11. 13
Keep it Business Readable!
Lessons Learned
Click on a button
–”Click on the New button”
–”Click on createButtonId button”
iClickOnButton: function(sButtonName) {
return this.waitFor({
controlType: "sap.m.Button",
viewName: viewName,
matchers: function($node) {
if ($node.getText() === sButtonName) { return true; }
return false;
},
success: function(oButtons) { this.pressOnButton(oButtons[0]); },
errorMessage: "Failed to click on the " + sButtonName + " button."
});},
matchers: function($node) {
if ($node.getText() === sButtonName)
{
return true;
}
return false;
},
id: sButtonId
Do not use element ID in the behaviour specification!
12. 14
Example
Acceptance criteria is an essential part of the backlog items
For complex behaviours (such as error handling), acceptance criteria definition is a huge
overhead for the Product Owner (PO). In this case, the following process is used:
PO adds all required information to the backlog item
Developer Team creates acceptance tests
PO and Quality Engineer validates the tests and the test results
Result
Deep understanding of the desired functionality while optimal workload balancing
Tailor Your Process!
Lessons Learned
13. 15
Example
Granularity of acceptance criteria must be chosen based on the nature of the functionality.
Generally, a high-level formulation is recommended
Use a more specific formulation critical scenarios with lots of possible outcomes (typically data
validation and error handling)
Result
Acceptance tests can be used for regression
Choose the Perfect Granularity!
Lessons Learned
14. Thank you.
Contact information:
András Kónya
andras.konya@sap.com
Lóránt Guth
guth.lorant@sap.com
You are welcome to give feedback for this session
in the UI5con Event App
Editor's Notes
Quote is from the article Dan North: Introducing BDD (https://dannorth.net/introducing-bdd/)
BDD affects your product and the daily life of your team in several ways. Some important aspects:
Improves communication
BDD is meant to ensure that all stakeholders have the same understanding of the functionality to be delivered. In case your team suffers from misunderstandings between the business and the tech people, BDD can improve this situation. It relies on simple, understandable, executable and verificable specification methods.
Less customer complaints
With BDD, the developers can better understand the customer’s goal. The difference between the delivered functionality and what the customer wanted is smaller.
Automated testing
BDD tools usually support automated verification of specification. Although it is not the primary goal of BDD, these tools can be used for regression tests.
Team velocity and stress level
BDD requires the deep understanding of the functionality from all stakeholders at the beginning of the implementation. Clarifying questions happen at the beginning of the development, not in a later stage, when the it cost more (time and money). This relieves stress for the team and helps keeping timelines/scope of delivery.
BDD is an agile practice that aims to specify functionality in an executable, yet business-readable manner.
In practice, BDD involves the following stages:
Specify the desired behavior using examples. (BDD is also commonly referred to as „Specification by examples”.) By using BDD tools, the outcome of this stage is a human-readable specification that is understandable by all stakeholders and can be used to verify the behavior later on.
Implement the behavior. During implementation, usage of other agile tools, like Test-Driven Development (TDD) are encouraged.
Ensure that the behavior/functionality is working as specified. BDD tools support automated verification by the specified examples.
Refactor, if needed. BDD tools can be used to check if the behavior did not change due to the modifications.
BDD steps are very much similar to TDD. However, BDD puts an emphasis on (the „cornerstones of BDD”):
Uses a human readable, domain-specific language common for all stakeholders. It helps avoid communication gaps and misunderstandings in the specification.
Uses examples to specify requirements. Examples are a very specific and executable way of describing functionality.
Cucumber is a tool to formulate and automate the validation of BDD-style acceptance tests.
As it integrates well with various testing tools, like OPA5 and Selenium, it provides ways to run acceptance tests on various levels of your test pyramid.
Using acceptance tests for regression testing (e.g. to apply zero defects principle in the definition of done criteria), this gives you a great freedom on the usability of acceptance tests.
The testing pyramid shows the testing levels of a typical SAP UI5 application.
Going up in the pyramid, higher level business cases can be addressed, but the tests becoming slow and their maintenance cost is higher (less reliable tests).
DEMO shown in the hidden slide
I set the OPA speed to extermly slow just for the demo.
To simplify to understand, first of look at this high level architecture figure.
In our case the Product Owner only see the human-readable interface of Cucumber and able to create tests on Gherkin language.
After that Cucumber interpret these test cases and translate Gherkin’s sentences to the native code written by the developer – in our case in OPA5 based on JavaScript - and run it on a testing framework, like QUnit
Gherkin, as mentioned before, is a business readable and writeable language that cucumber understands. It has a simple syntax – line oriented, use indentation, line endings and some speicific words like feature, background and scenario - the parser divides the code into three part based of these.
These three part is
Feature: one feature is the collection of test case for one page.
Background: it is the setUp method of the test cases, so it is running before every test scenario
Scenario: this is the test case
Also its use the well-known given-when-then style of representing tests, specifying the system’s behavior:
Given: put he system in a known state before start interacting with the system
when: describe the key action
Then: observe outcomes
Cucumber is not a testing framework, testing framework helps to execute and report the test scripts like the well-known JUnit, TestNG or in our case the QUnit.
Cucumber is a specifications tool what uses Gherkin to describe the software behaviour.
Cucumber uses Step Definitions to translate the software behaviour, writen on Gherkin, to the native programming language – and I have to mentioned that Cucumber support a variety of different programming languages.
You can see one step definiton on the example. When Cucumber find the mathching step definition to the gherkin line, in this case „Click on the WILDCARD button” and execute the code part of it – in our case the OPA5 code.
OPA5 is Qunit based API for UI5 controls written on JavaScript. With OPA5 we can easily find and access the desired UI5 element and because of this property OPA5 is a helpful tool for write automated tests for UI5.
To get a not-so-deep-dive, we can check this little piece of code. OPA5 has two part
first give some information to OPA5, and depends on it OPA5 try to find our application
the second part where you can throw an error message if OPA5 did not find the control or define actions if it found.
As you see there is two way to describe a step „click on a button” – one with the name of the field and one with the ID of the field.
Of course, if we want to keep our steps business readable we have to chose the first one, because a customer or even the PO do not want to search for the ID of the elements.
But because of it we have to write a more complex code, we cant give the ID to OPA or at least we have to use a HashMap to get the ID from the Name.
Acceptance criteria is part of the backlog item
Sometimes it requires several test cases to cover all possible outcomes of a scenario (e.g. typically error handling). In these cases, the backlog contains all required information (but not the acc. criteria) and the scrum team members will create these steps on demand. PO and QA later validates it.
Experience:
the deepest level of understanding of the functionality comes from writing these scenarios with BDD.
Helps team progress while reducing stress level