Expo qa from user stories to automated acceptance tests with bdd

2. From user stories to Automated
Acceptance Tests with BDD
Eduardo Riol

3. Who am I?
I’m Eduardo Riol, QA & Testing Community Technical Leader in
atSistemas, where we are enthusiasts of Software Quality Assurance,
Tests Automation and Testing best practices.
I started as developer in Java and .NET environments more than a decade
ago, but I’ve been involved the last eight years in Software Quality, both
products and development processes oriented.
Currently my interests are focused in the integration of QA activities in
Agile and DevOps environments, technical debt control and BDD.
Lately I research about other topics like Security and Blockchain.
twitter.com/eduriol
github.com/eduriol
linkedin.com/in/eduriol

4. We are a dependable partner, with a long trajectory and a solid future
in the industry
of revenue in 2017
54,5 M€ estimated for
2018
Present in every
industry
Spanish capital
Presence in Madrid,
Barcelona, Cádiz,
Zaragoza, A Coruña,
Mallorca and Huelva
International vocation
And currently IT referent in Innovation
Expert Centers and cutting-edge
Technological Communities,
leaders in Digital Transformation
IT professionals
of proved Technical Solvency
growth in international
interannual billing at 2017
closure
atSistemas
45,6 M€
100%
1994Founded in
R&D
1000
16,5%

5. Houston, we have a problem!
“This is not what we asked for” Does that ring a bell?

6. Houston, we have a problem!
Chinese
whispers
The Gossips, Norman Rockwell, 1948
“This is not what we asked for” Does that ring a bell?

7. We focused on developing software
CORRECTLY…
Functional testing
Unit testing
Performance testing
Integration testing
Static analysis Ethical hacking
Process audits
UAT

8. … but do we develop the CORRECT software?
Features Used
The Standish Group estimate of features used in custom application development, 2014
Hardly Ever
50%
Often
20%
Infrequently
30%

9. Features Used
The Standish Group estimate of features used in custom application development, 2014
Hardly Ever
50%
Often
20%
Infrequently
30%
… but do we develop the CORRECT software?

10. We don’t trust releases

11. Most of the time we don’t have an acceptance tests suite
robust and complete enough to be trusted
We don’t trust releases

12. Lack of documentation
Individuals and interactions OVER processes and tools
Working software OVER comprehensive documentation
Customer collaboration OVER contract negotiation
Responding to change OVER following a plan
We are uncovering better
ways of developing
software by doing it and
helping others do it.
Through this work we have
come to value:
That is, while there is
value in the items on
the right, we value
the items on the left
more.
The Agile manifestoThe Agile manifesto

13. How often have you finished a project in which technical documentation is up-to-
date with what software really does?
Individuals and interactions OVER processes and tools
Working software OVER comprehensive documentation
Customer collaboration OVER contract negotiation
Responding to change OVER following a plan
We are uncovering better
ways of developing
software by doing it and
helping others do it.
Through this work we have
come to value:
That is, while there is
value in the items on
the right, we value
the items on the left
more.
The Agile manifestoThe Agile manifesto
Lack of documentation

14. And collaboration within the team?
We have Quality
issues
Ain’t my problem, Software
Quality is what testers
should care about, isn’t it?

15. Devs? I think those are the guys in
the 4th floor or something
Dev people keep returning
unresolved bugs
And collaboration within the team?
We have Quality
issues
Ain’t my problem, Software
Quality is what testers
should care about, isn’t it?

16. This problem starts from here
• Analysis
• Development
• Testing
• Operations
As separated phases, estimated
and planned from the beginning of
the project

17. This problem starts from here
As separated phases, estimated
and planned from the beginning of
the project
• Analysis
• Development
• Testing
• Operations

18. BDD: How Devs and Business collaborate
BDD is a collaboration model between Business users and
Development team…
… consisting of establishing conversations based in concrete
examples of the application use, with the objective of reducing
misconceptions and assumptions…
… discovering during the process the features that truly provide
value

19. • Examples describing a new feature are
translated to a simple and common language,
without ambiguities (for example Gherkin)
• Development team transforms these examples
to a series of executable specifications as
automated tests
• A software feature will be completed
whenever every specification executes
correctly
BDD
TDD
Write a failing test
N cycles
BDD: How Devs and Business collaborate

20. Then, what is (and what not) BDD?
• Writing requirements in Gherkin
language
• Automating tests with
Cucumber
• Documenting functionalities
after programming them
• A collaboration model
• A discovering process
• Understanding Business needs
• Describing software with
examples
What it is What it’s not

21. Three amigos
Business QA
Dev

22. Three amigos
1. PO talks with Business
about its needs
2. The PO, a Dev and a Tester
meet to elaborate scenarios
jointly
4. The scenarios guide the dev
and act as automated tests
3. The tester implements the
scenarios as acceptance tests
5. Automated tests provide
feedback about progress and
document the application

23. A collaboration scenario
We want our application to require a password
with at least 8 characters, a number and a
capital letter
Don’t you mean you want
the application to require a
strong password?

24. Password strenght, xkcd, Randall Munroe
Password Security Acceptable?
secret Weak No
password Weak No
password1 Weak No
aBcdEfg1 Weak No
qwertY12 Weak No
dJeZDip1 Medium Yes
SeagullHedgehog Strong Yes
SeagullHedgehogCatapult Very strong Yes
A collaboration scenario
We want our application to require a password
with at least 8 characters, a number and a
capital letter
Don’t you mean you want
the application to require a
strong password?

25. Discovering our Business
“In software development, ignorance is the constraint. You know a lot more about the
best way to build a particular solution after you’ve finished building it, but by then it’s
too late to take advantage of your knowledge.”
John Ferguson Smart, BDD in action, 2015

26. Implementing BDD: Describing scenarios
SCENARIO
Sending Contact Form
Given John Doe enters contact form
And he fills the fields with his data and the message
And accept legal clause
When he sends the query
Then the confirmation message is received
Given: Defines the context in which the
scenario is executed. In this step the starting
point for the example is established.
When: It’s the action that triggers the
example. It consists of an interaction with the
application, usually by a user, whose behavior
we want to validate.
Then: In this step the result expected by the
Business is defined, meaning the condition
that must be fulfilled for the scenario to be
considered correctly executed.

27. Implementing BDD: Automating scenarios
Scenario: Transfer money to a Savings account
Given my Checking account has a balance of 1000.00
And my Savings account has a balance of 2000.00
When I transfer 500.00 from my Checking account to my Savings account
Then I should have 500.00 in my Checking account
And I should have 2500.00 in my Savings account

28. Scenario: Transfer money to a Savings account
Given my Checking account has a balance of 1000.00
And my Savings account has a balance of 2000.00
When I transfer 500.00 from my Checking account to my Savings account
Then I should have 500.00 in my Checking account
And I should have 2500.00 in my Savings account
@Given(“^my (.*) account has a balance of (d+)$")
public void setupInitialAccount(AccountType accountType,
double amount) {
// Setup account
}
@When(“^I transfer (d+) from my (.*) account to my (.*) account$")
public void transferAmountBetweenAccounts(double amount, AccountType source, AccountType destination) {
// Perform action
}
@Then(“^I should have (d+) in my (.*) account$")
public void checkAccountAmount(double amount, AccountType accountType) {
// Assert amount
}
Implementing BDD: Automating scenarios

29. Exploratory Testing
BDD levels: The Testing pyramid
UI
API / Services
Unit
Prevent errors / Provide value

30. UI (manual & automatic)
API / Services
Unit
Detect errors / Certify releases
Exploratory Testing
BDD levels: The Testing pyramid
UI
API / Services
Unit
Prevent errors / Provide value

31. BDD in the software building process
Scenarios execution should be part of the software integration,
building and deployment process.
Each scenario should be able to be executed separately, without any
concrete order. There should be no dependency between scenarios.
Scenarios are a code asset to be maintained under version control
systems.
The software building and deployment process additionally becomes a
process for building and deploying documentation and reports.

32. Reports: feature readiness
A feature is considered ready to
be deployed in production
whenever its scenarios execute
correctly.
Feature readiness
Your business does not care whether your tests pass or not. What they want to know is
if the functionality is ready to be deployed in production.

33. Reports: feature coverage
Feature coverage shows the % of
features with every defined
scenario executing correctly.
Feature coverage

34. Warning!, BDD is not a silver bullet
Business implication: We need the stakeholders involved
since the beginning of the project.
BDD is thought for Agile: it is a collaboration model to
fostering an iterative discovering of the requirements.
BDD doesn’t like silos: if the organization works in
isolated silos and collaboration doesn’t flow, the
progressive clarification of goals disappears.
Risk of high cost in test maintenance: experience and
knowledge are required in order to design maintainable
functional specifications and implement them correctly.

35. Technologies for BDD

36. Thanks for your attention!
Any question?