Adopting Test Pyramid to maximise effectiveness of manual and automation testing: With evolving technology trends (including voice-enabled interactions, NLP, ML, etc.,) and continuous delivery becoming a norm, both functional and automation testing is becoming more challenging and at times even puzzling. However, "Test Pyramid", one of the core testing techniques accepted and advocated by global QA experts, can enable us answer these challenges more efficiently. Test Pyramid basically says "Understand the different layers of logic in the application and test the right logic in the right layer." Given the significance of this technique, for the benefit of our global QA community, we at ThoughtWorks have invested time to build an open-source GitHub repository to demonstrate this technique.

1. Understanding Layers of Testing
By Christopher Rex,
ThoughtWorks
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

2. Software Evolution
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

3. Challenge
What can be a sensible test approach?
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

4. Evolution of Automated Testing
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

5. Challenge
How to build an efficient automation suite?
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

6. Problems
What can be a sensible test approach? How to build an efficient automation suite?
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

7. Anti Patterns
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

8. Solution
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

9. How to proceed?
1. Build a deeper understanding of
AUT
๏ Layered approach to testing
2. Test for the right logic in the right
layer
๏ Sensible functional testing
๏ Efficient automation testing
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/
Demo code - https://github.com/christopher-rex/sample-test-pyramid

10. QuickTour of AUT
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

11. ActivityTime: List test scenarios
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

12. Login Functionality
Acceptance Criteria:
AC #1:
Given I am a registered user
And my email account has been verified
When I login to the application
Then login succeeds
AC #2:
Given I am not a registered user
When I login to the application
Then login fails
AC #3:
Given I am a registered user
And my email has not been verified yet
When I login to the application
Then login succeeds with a prompt to verify email
AC #4:
Given I am a registered user
And my email account has been verified
And my account has been disabled
When I login to the application
Then login fails
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

13. Login Test Scenarios
1. Verify successful login
2. Verify on providing an email-id of user not registered, login fails with an appropriate
error message
3. Verify not providing any input and submitting the form, results in appropriate error
message
4. Verify on providing partial input say email-id or password and submitting the form,
results in appropriate error message
5. Verify on providing incomplete email-id, login fails with an appropriate error message
6. Verify on providing an email-id of a disabled user and submitting the form, login fails
with an appropriate error message
7. Verify on providing an email-id of user whose account is not verified, login succeeds
with an appropriate warning message
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

14. Testing Approach
1. How can we approach testing this user-authentication feature?
2. Do we have enough information to define a testing approach?
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

15. Understanding Application Architecture
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

16. Testing E2E layer
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

17. Testing Service layer
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

18. Testing UI layer with a stubbed server
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

19. Summary of Testing approach
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

20. ActivityTime: Layer the scenarios
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

21. Login Test Scenarios -Layered
UI Tests
E2E Tests
1. Verify successful login
2. Verify on providing an email-id of user not
registered, login fails with an appropriate error
message
Stubbed Tests
3. Verify not providing any input and submitting the
form, results in appropriate error message
4. Verify on providing partial input say email-id or
password and submitting the form, results in
appropriate error message
Service Tests
Integration Tests
1. Verify valid login results in success response
2. Verify login request with email-id of an
unregistered user, results in error response
3. Verify login request with email-id of a disabled
user, results in error response
4. Verify login request with an unverified email-id,
results in error response with appropriate
response body
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

22. Applying Layered Testing Approach
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

23. Benefits of the Layered Testing Approach
1. Deeper understanding of responsibility/functionality of each layer
๏ Sensible testing
๏ Increased value of QA feedback
๏ Opens-out opportunities to do more NFR testing
2. Independent testing at each layer
๏ Uninterrupted development
๏ Testing edge cases with ease
3. Defects can be reported with analysis information
๏ Reduced time-to-fix
4. Reused logic can be tested only once
๏ Reduced testing time
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

24. Application Architecture: Unit layer
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

25. Automation Walkthrough
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

26. Test Automation -Layered
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

27. Test layer Number of tests Execution time
Last mile (Extension of E2E to
include integration with legacy
systems)
3 ~ 10 mins
E2E tests ~30 ~ 15 mins
Stubbed webapp tests ~280 ~ 20 mins (4 parallel threads with 70
tests per thread)
Integration tests at each service 100 to 200 ~ 2 mins
Unit tests at each service 100 to 200 ~ 1 min
Test Pyramid from an Actual Project
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

28. Benefits of the Layered Automation Approach
1. Dependable builds
๏ Increased automation effectiveness
2. Reduced maintenance effort
๏ Reduced cost
3. Faster feedback cycle due to reduced execution time
๏ Increased efficiency of automation suite
4. Continuous integration powered by Layered Automation can instantly highlight
defective layer
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

29. ActivityTime: Coupon Testing with Layered Approach
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

30. Coupon Validation Functionality
Acceptance Criteria:
AC #1:
Given I have a valid coupon
When I validate the coupon
Then coupon validation succeeds
AC #2:
Given I have an invalid coupon
When I validate the coupon
Then coupon validation fails
AC #3:
Given I have an expired coupon
When I validate the coupon
Then coupon validation fails
AC #4:
Given I have an already used coupon
When I validate the coupon
Then coupon validation fails
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

31. Coupon Validation Test Scenarios -Layered
UI Tests
E2E Tests
1. Verify successful coupon validation
2. Verify invalid coupon results in validation error
Stubbed Tests
3. Verify not providing any input and submitting the
form, results in appropriate error message
4. Verify on providing incomplete input and
submitting the form, results in appropriate error
message
Service Tests
Integration Tests
1. Validate-coupon request for an available coupon,
results in success response
2. Validate-coupon request for invalid coupon,
results in error response
3. Validate-coupon request for an expired coupon,
results in error response
4. Validate-coupon request for an already used
coupon, results in error response
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

32. Q & A
ThoughtWorks, http://twchennai.github.io/VODQA-Chennai/

33. Thanks for joining
- Christopher Rex,
ThoughtWorks