Online QADay 2021 MESUT DURUKAL «4 Pillars of Successful Agile Test Automation» telegram: wwww.t.me/goqameetup fb: www.fb.com/goqaevent fb: www.fb.com/qaday.org linkedin: https://www.linkedin.com/company/goqa/ Сайт: www.qaday.org

1. 4 Pillars of Successful
Agile Test
Automation
Rapyuta Robotics
mesutdurukal
@DurukalMesut
mesutdurukal.com
Mesut Durukal

2. Mindset Monitoring Robustness Team Spirit
AGENDA

3. Why should we care?
Technical Debt
Motivation

4. Mindset >> Monitoring >> Robustness >> Team Spirit

5. Awareness
Motivation on the Correct Goal
•Know what you do?
•Know what you should do?
•Have a tendency to complete the execution?

6. Customer Perspective

7. Mindset >> Monitoring>> Robustness >> Team Spirit

8. Why Monitoring?
Current evaluation
Future projection

9. What to track
• Passed test case percentage
• Failed test case percentage
• Blocked test case percentage
• Test Case Productivity
• Total number of test cases
• Number of tests run per time
• Test Design Efficiency
• Test Review Efficiency
• Average time to test a fix/feature
• Average time to test the full set
• Rework Ratio
• False Alarm Ratio
• Automation Rate
• Requirement coverage percentage
• Test Cases by Requirement
• Requirements without Test Coverage
• Bugs found after release ratio
• Bug find rate
• Number of bugs per test
• Defects per Requirement
• Mean Time Between Failures (MTBF)
• Defect Removal Efficiency
• Fixed defects percentage
• Accepted defects percentage
• Distribution by age
• Critical Defects Percentage
• Defect Severity Index (DSI)
• Defect Resolution Time
• Total allocated costs for testing
• Actual cost of testing
• Budget variance
• Schedule Variance
• Bugs found by tests ratio

10. Customization
Time Elapsed for each step in Test Design

11. Automated Metrics Collection

12. Visualization

13. A sample application

14. Mindset >> Monitoring >> Robustness>> Team Spirit

15. Problems: From Test Smells

16. Problem Smell Root Cause Solution
Analysis /
Re-run Effort
Flaky Tests
Asynchronous waits Polling Mechanisms
Load and Conflicts Clean Up
Suite Dependency Test Groups
No Track Test History
Escaped Bugs Scope Smells Execution Variances Improved Coverage
Refactoring
Effort
Structural
Smells
Code Duplication Helper Classes
Weaknesses in Code Static Code Analysis
All:
• Automate Jira Queries,
• Use S3 to store input/outputs
• Use CloudWatch Dashboards to track progress

18. Refactoring Effort: Quality Gates

19. Mindset >> Monitoring >> Robustness >> Team Spirit

20. Efficiency: Collaboration

21. Last but not least..

22. Mindset Monitoring Robustness Team Spirit
SUMMARY
Customer
Perspective
Progress
Graphs
Remove Smells &
Improve Code
Quality
Holistic
Approach