Nowadays, test automation has become an essential part of testers' lives. However, there are not many resources available for beginners that do not know how to start. This is why we would like to share their experience and explain the steps followed to develop our test automation framework, including why we've chosen their current testing tools. The emphasis will be on all the challenges we had to face; the problems we found, our possible solutions and our reasoning for implementing one or the other. We will share their insight on how to write test scenarios from a functional point of view and how to speed up automated tests through browser initialization. You will learn which are the steps you have to follow to choose the correct tool and tips & tricks to success in your project.

1. CHEESECAKE
The story of our automatic test suite

2. BIO
The story of our automated test suite 1

3. INTRODUCTION
• We had to create an automated test suite for a web
application
• Evolution of our automated test suite
The story of our automated test suite 2

4. OUR TOOLS
• Open Source
• Easy to learn
• Continuous Integration
• Document app behavior
• Flexibility
The story of our automated test suite 3

5. #1 WRITE SCENARIOS
Initial State: We need to agree on the wording
Solution 1
The story of our automated test suite 4
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6
Scenario: Log in with valid credentials
Given I am on the login page www.myapp.com/login
When I fill the user name with jo@mail.es
And I fill the password with iLoveBdd
And I click the login button
Then I should be redirected to my dashboard

6. #1 WRITE SCENARIOS
Initial State: We need to agree on the wording
Solution 2
The story of our automated test suite 5
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6
Scenario: Log in with valid credentials
Given I am on www.myapp.com/login
When I log in with the following data:
| username | password |
| jo@mail.es | iLoveBdd |
Then I should be redirected to my dashboard

7. #1 WRITE SCENARIOS
Initial State: We need to agree on the wording
Solution 3
The story of our automated test suite 6
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6
Scenario: Log in with valid credentials
Given I am on the login page
When I log in with valid credentials
Then I should be redirected to my dashboard

8. #1 WRITE SCENARIOS
Initial State: We need to agree on the wording
Solution 3
The story of our automated test suite 7
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6
Scenario: Log in with valid credentials
Given I am on the login page
When I log in with valid credentials [email:jo@mail.es]
Then I should be redirected to my dashboard

9. #2 ENVIRONMENT VARIABLES
Initial State: All test run against same environment
Problem: No flexibility
The story of our automated test suite 8
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

10. #2 ENVIRONMENT VARIABLES
Solution:
• Pass environment variables on the command line to
Cucumber: ENVIRONMENT
The story of our automated test suite 9
$ cucumber ENVIRONMENT=local
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

11. #3 BROWSER INITIALIZATION
Initial State: Browser started for each scenario
Problem: Slow
The story of our automated test suite 10
Before do
@browser = Watir::Browser.new :firefox
end
After do
@browser.quit
end
hooks.rb
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

12. #3 BROWSER INITIALIZATION
Problem: Slow
Solution:
• Open/close the browser when starting/ending the suite
• Clean cookies between scenarios
The story of our automated test suite 11
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

13. #3 BROWSER INITIALIZATION
The story of our automated test suite 12
AfterConfiguration do
$browser = Watir::Browser.new :firefox
end
Before do
@browser = $browser
@browser.cookies.clear
end
at_exit do
$browser.quit
end
hooks.rb
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

14. #4 THE NIGHTLY BUILD
Initial State: Tests were executed on demand
Problem: Slow feedback loop
Solution: Nightly build + Rerun report
The story of our automated test suite 13
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

15. #4 THE NIGHTLY BUILD
The story of our automated test suite 14
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

16. #5 COVERAGE &
REGRESSION
Initial State: Before a deploy all tests are run
Problem: Slow feedback loop
Solution: Select a subset of tests
• Coverage: Tag scenarios
The story of our automated test suite 15
INTRODUCTION REQUIREMENTS OUR TOOLS CHALLENGES
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6
smoke
regression
nightly$ cucumber --tags @smoke

17. $ cucumber features/acceptance/fields
-r features/
#5 COVERAGE &
REGRESSION
Initial State: Before a deploy all tests are run
Problem: Slow feedback loop
Solution: Select a subset of tests
• Regression: Change files structure
The story of our automated test suite 16
INTRODUCTION REQUIREMENTS OUR TOOLS CHALLENGES
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

18. #6 BACKEND TESTING
Initial State: We were only doing UI testing
Problem: Slow feedback and flickering scenarios
Solution: Backend testing
Option 1: New project
The story of our automated test suite 17
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

19. #6 BACKEND TESTING
Initial State: We were only doing UI testing
Problem: Slow feedback and flickering scenarios
Solution: Backend testing
Option 2: Use tags and conditionals inside the
step definitions
The story of our automated test suite 18
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

20. #6 BACKEND TESTING
Initial State: We were only doing UI testing
Problem: Slow feedback and flickering
scenarios
Solution: Backend testing
Option 3: Load different contexts for backend
and frontend testing.
The story of our automated test suite 19
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

21. #6 BACKEND TESTING
Proof of concept
1. Tag scenarios with @front/@back
2. Re-write scenarios
3. Make step definitions generic
4. Load only one context
The story of our automated test suite 20
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6
World(RegistrationStepsFront) if ENV[‘LEVEL’] == ‘front’

22. #6 BACKEND TESTING
• Reliable
• Fast – Quick feedback
• Easy to maintain
The story of our automated test suite 21
CHALLENGE
#1
CHALLENGE
#2
CHALLENGE
#3
CHALLENGE
#4
CHALLENGE
#5
CHALLENGE
#6

23. TAKE AWAYS
• Do not be afraid of changes
• Do not try to fix the big problem at once
• Do not think small improvements do not count
• Do not stop improving
The story of our automated test suite 22

24. THANK YOU
The story of our automated test suite 23
@bugbustersbcn
#bugbustersbcn
linkedin.com/in/gloriahornero
linkedin.com/in/aidamanna