Test Automation is Good - provided you do it right way! These slides give example of how right automation strategy made regression faster and easier.

1. Achieving the Right Testing Pyramid

2. Background
5%
95%No AutomationAutomation Coverage

3. Background
1.5 Months
4.5Months
0
1
2
3
4
5
6
7
MonthsTypical Release Cycle
Development
Regression

4. Background

5. Critical Issues reported from Production
0
2
4
6
8
10
12
Critical Data Issues
Cosmetic
Localization
Issues

6. Background

7. After Automation Introduced
Selenium Tests 45%
Unit & Integration Tests 20%
No Automation 35% Automation Coverage

8. After Automation Introduced
2.5 Months
3 Weeks
0
0.5
1
1.5
2
2.5
3
3.5
MonthsTypical Release Cycle
Development
Regression

9. Critical Issues reported from Production
0
2
4
6
8
10
12
Critical Data Issues
Cosmetic
LocalizationAfter automationBefore automation

10. State of the Product
Tightlycoupledwith UI
Browser DependentData tests
Falsefails
Difficult to pinpoint real culprit
Difficult to maintainand time consuming as test are fragile
Laggingwith current release development
Image Courtesy: Alex Yakyma

11. State of the Product
Manual Checking
End-to End Selenium Tests 70%
Integration Tests
20%
Unit Tests
10%

12. Analysis of Bugs Report
10%
90%Data FailuresBug Report Analysis
UI Navigation Failures

13. Our transition story…
•We decided to "Test things right , at right place on right time" which would result in "right testing pyramid“

14. Our transition story…
Application UI
Service Layer
DAO Layer
Automated UI Tests
Tightly Coupled
Fragile
Lag with current development
Automate One Level Below UI
UI Independent
Robust & Stable
With current devcycle
Fast Execution

15. Introduced BDD with Cucumber

16. Current Status
Selenium
Tests 10%
Workflow Tests: 40%
No Automation 20%
Unit & Integration Tests 30% Test Automation Status

17. After Right Testing Pyramid
1 Week
2.75 Months
0
0.05
0.1
0.15
0.2
0.25
0.3
0
0.5
1
1.5
2
2.5
3
3.5
MonthsTypical Release Cycle
Regression Time
Development

18. Pain points
•Learning curve for the team
•Difficult to implement in case where no API is exposed or tight coupling with UI
•Heavy dependency on pre-populated database

19. Looking ahead
•Expose RESTful APIs for all upcoming features
•Move BDD specs to lower layers
•Moving away from baseline database