Agile Testing - Challenges


Published on

Agile Testing - Challenges

Published in: Software, Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Agile Testing - Challenges

  1. 1. Challenges in Agile Testing
  2. 2. List of Challenges  Volume and Speed of requirement change  Over-commitment and Sprint durations  Accidental Broken Code due to frequent builds  Inconsistent and Inadequate Unit Testing  Inadequate Test Coverage  Inadequate Testing for published API  Poor, Changing and Missing Test Oracles  Detecting defects early, When they’re easier and cheaper to fix Challenges in Agile Testing……
  3. 3.  Volume and Speed of Requirement Challenges in Agile Testing…… Caused by: Changing requirements, even late in development Changes in definition of product and its correct behavior Resolved by: Risk-based Testing Automated Testing Lightweight reactive Testing Keep test team informed of such changes Account for differential impact of changes on testing
  4. 4.  Over-commitment and Sprint durations Challenges in Agile Testing…… Caused by: Some projects ritualize the rules, particularly sprint time deadlines Ex: if a project follows four week sprints and continually over-commits, the test team gets squeezed on the last weekend of every sprint Resolved by: Requires team and management maturity Risk-based testing helps the test team deal with over-commitment  Reduce the scope of testing based on risk  Slip low-risk tests into next sprint
  5. 5.  Inconsistent and Inadequate Unit Testing Challenges in Agile Testing…… Caused by: Unit testing has limited bug-finding effectiveness, averaging 25 to 30%, while good system testing averages around 85% Not all programmers do unit testing Resolved by: Good Unit Testing is the only solution
  6. 6.  Broken code due to frequent builds Caused by: Daily build introduces the rick of breaking code/ existing functionality Resolved by: Perform continuous integration – automated process for creating software build Employ an automation tool that performs regression testing whenever you launch a new build Employ Test-Driven development [TDD] Ensure that Automated Test team is working optimally  Review Automated Test metrics  Use Source Control for reviewing changes to automated tests and protection  Continue to build on your Automated Tests Track metrics are most important for successful automated testing: Metrics for successful automated testing: Feature Coverage Requirement/Feature blocked Daily Test Run trending Daily Test Runs by Host Challenges in Agile Testing……
  7. 7.  Inadequate Test Coverage Caused by: Few test cases written for User story Lack of visibility of code check-ins for regular corrections to code Resolved by: Creating test cases at the level of User-story Design sufficient number of tests:  Positive Tests/ Negative Tests/ Performance Tests/ Ancillary Tests Branch code using SCM; any code outside the target always remains untouched Adding triggers to SCM to inform team of code changes Integration with source check-in to find changed code that was not anticipated or planned for Analyzing specific metrics to identify traceability and missing test coverage Metrics for traceability and test coverage: Traceability Coverage Blocked Tests Test Runs by Requirement [User Story] & Configuration Daily Test Run Trending Defects by Requirement/ Daily Defect Trending/ Defect Duration Challenges in Agile Testing……
  8. 8.  Inadequate Testing for published API Caused by: API testing often is omitted because of the misperception that it takes programming skills to call the properties and methods of your API. Resolved by: Dedicated resource High Use functions Usability Tests Security Tests Stopwatch-level performance Tests API metrics to be watched: Function Coverage Blocked Tests Coverage within Function Daily API Test Run Trending Ensure API testing team is working optimally: Review API Run Metrics Continue to build on your API Tests Challenges in Agile Testing……
  9. 9.  Poor, changing and missing Test Oracles Challenges in Agile Testing…… Caused by: Agile de-values documentation Change the definition of correct behavior after test execution has started Change the definition of correct behavior without involving the test team Resolved by: Embracing change Conveying information [in] face-to-face conversation Change Management needs to be in-place
  10. 10.  Detecting defects early Challenges in Agile Testing…… Problems from late-detection of defects:  More time to fix  More money to fix. Resolved by: Performing peer reviews of source code and test artifacts to find early stage defects  Peer Code Review Metrics:  Lines of Code Inspected  Requirement Review Coverage  Manual Test Review Coverage  Automated Test Review Coverage  Code Review Coverage  Static Analysis Issues found  Defects discovered by Peer review Using static analysis tools to identify early stage defects Analyzing peer review statistics and defect aging to address defects early when they’re least costly to fix  Perform Peer reviews  Automatically run static analysis  Review Defect metrics
  11. 11. Challenges in Agile Testing…… Testers and Test managers must help their organizations understand and manage the challenges to manage expectations