Adaptive software testing


Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Adaptive software testing

  1. 1. Method change based on observation
  2. 2. Introduction <ul><li>Automated tests, scripted manual tests, exploratory testing – everyone has it’s place in an effective test methodology </li></ul><ul><li>Test with low execution cost can give indications to what high execution cost tests should be executed for best return of investment </li></ul><ul><li>Your test method adapts as more and more information about the system is revealed </li></ul><ul><li>Everything is risk-based, but the risk becomes clearer after observing the system during test execution </li></ul>
  3. 3. Overview
  4. 4. Scope Size Low Level Automated Tests High Level Automated Tests Manual User Scenarios Manual Scripted Tests Exploratory Testing
  5. 5. Low Level Automated Tests <ul><li>Start by executing the fastest test cases available </li></ul><ul><li>Low level automated tests that run below the UI are often the fastest tests </li></ul><ul><li>The actual scope is based on all known risk </li></ul><ul><li>It is better to err on the side of caution when selecting the test scope because there is no information available with regards to the operational system </li></ul><ul><li>Since the tests have short execution time, it is possible to add more than necessary without a major cost increase </li></ul>
  6. 6. High Level Automated Tests <ul><li>Based on all known risks and the execution of the low level automated tests, the high level automated test scope is set </li></ul><ul><li>These test cases are slightly more expensive to run since they often involve more complex and time consuming operations, and thus a too large scope can be more costly </li></ul><ul><li>Based on the low level automated test execution it should be easier to set a correct high level automated test scope </li></ul>
  7. 7. Manual User Scenarios <ul><li>Once the automated tests have been executed a larger part of the risk space has now been revealed </li></ul><ul><li>User Scenarios are selected where the automated tests, or other known risks, indicate that there could be problems </li></ul><ul><li>These are costly to execute, so the scope should be selected with care to avoid any unnecessary tests </li></ul><ul><li>Once the user scenarios have shown where the major problems are located, finding the root cause of the failing user scenarios can begin </li></ul>
  8. 8. Manual Scripted Tests <ul><li>Manual scripted tests are selected based on where the User Scenario execution indicated that problems exist </li></ul><ul><li>A set of basic test cases are executed for the problem area to reveal what parts of the user scenario contains the defect </li></ul><ul><li>Example: If a user scenario indicated that there was a problem with a saving operation in a program, then a set of test cases exercising the save operation is executed </li></ul><ul><li>These manual tests are then input to the exploratory session charters </li></ul><ul><li>These are also costly and should be selected with care </li></ul>
  9. 9. Exploratory Testing <ul><li>Based on the results of the scripted tests and all known risks, exploratory session charters are created </li></ul><ul><li>It is with these sessions that all the major defects are detected </li></ul><ul><li>The results of the session is then compared to the results of the previous automated and manual test executions to secure that all failed test cases are accounted for and root causes to the defects have been found </li></ul>
  10. 10. Conclusion <ul><li>By using this Adaptive Software Testing method resources can be saved, at the same time as the risk space becomes increasingly clearer during test execution </li></ul><ul><li>This is a way to make exploratory testing more effective and synergies are achieved with automated tests and scripted manual tests </li></ul><ul><li>Automated tests are great for quickly and at low costs exploring the risk space, giving the tester significant insight into where the major problems are located </li></ul><ul><li>Adding scripted user scenarios and test cases to cover all basic scenarios within a risk area gives the tester a systematic way of covering the risk space </li></ul><ul><li>Using exploratory testing is key to actually finding the root causes to the major problems, and also to smoke out any problems that have not been revealed by the previous tests </li></ul>