13 14-test planning and management and rbt

603 views

Published on

Published in: Software, Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
603
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
18
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

13 14-test planning and management and rbt

  1. 1. Introduction to Test Planning, Process and Project Management: Risk Based Testing Prof Gopalaswamy Ramesh Gopalaswamy_Ramesh@Yahoo.Com
  2. 2. 5/3/2014 Test Planning and Management Slide : 2 Test Planning and Management: Agenda of the Session  Testing phase as a project  Planning a testing project  Test Management  Test Process  Test Reporting
  3. 3. 5/3/2014 Test Planning and Management Slide : 3 Essential Reading  http://www.testingeducation.org/BBST/BBSTRisk -BasedTesting.html  Google search on “Risk Based testing” and read a few articles  Rest of the outline from Software Testing book…
  4. 4. 5/3/2014 Test Planning and Management Slide : 4 Perspectives of a “Project” Projects Products Services Engineering Activities “In-stream” Activities “Umbrella” Activities Technology Process People
  5. 5. 5/3/2014 Test Planning and Management Slide : 5 Characteristics of a Project (in Software context)  Originates from organizational vision  Focused in achieving a very specific requirement (to create a unique product or service)  Targeted, controlled execution (planned, executed and controlled)  Has a definite beginning and an end (temporary)  Usually in a stand alone group (performed by people)  Normally has to obey some constraints (budget, time …) and dependencies  Operates in the context of various stakeholders  Life period ties back to product if applicable All these apply to Testing also
  6. 6. 5/3/2014 Test Planning and Management Slide : 6 Test Planning and Management: Agenda of the Session  Testing phase as a project  Planning a testing project  Test Management  Test Process  Test Reporting
  7. 7. 5/3/2014 Test Planning and Management Slide : 7 Parts of Test Planning Comm’n Mgmt Risk Mgmt Test WBS And Scheduling Identifying Test Deliverables Identifying Env needs Identifying Skill sets / Trng Setting Entry / Exit Criteria Deciding Test Strategy Scope Mgmt Preparing A Test Plan Test Planning Start Here
  8. 8. 5/3/2014 Test Planning and Management Slide : 8 Scope Management  Understanding what constitutes a release / engagement  Breaking down the release / engagement into features (may be given to you already)  Prioritizing the features for testing (may be given to you already)  Deciding which features will be tested and which will not be (may be given to you already)  Gathering details to prepare for estimation
  9. 9. 5/3/2014 Test Planning and Management Slide : 9 Estimation  Challenges in Estimation for testing services  Risk Based Testing as a method of scope determination and resource allocation  Briefly mentioning other scope determination / resource allocation methods
  10. 10. 5/3/2014 Test Planning and Management Slide : 10 Challenges in Testing Projects for Testing Services  Testing is done with incomplete information (compared to a product organization)  Testers are more insulated from developers  Testing is a crunch time job  Testing has no definite end and hence you need to decide when to stop, but…  Before you stop, you better have tested the right things because…  Testing is highly visible when incomplete or ineffective!
  11. 11. 5/3/2014 Test Planning and Management Slide : 11 What are the “right” things to test?  Are you testing an entire product or some components?  Which functions are critical for a product?  Which functions are highly visible when they fail?  Which functions have been highly error-prone?  Which functions are most often used?  Which functions are changed most often / recently?  Which functions suffered turnover of employees?  Where is high geographic distribution of work?  Which functions require new skills / technology?
  12. 12. 5/3/2014 Test Planning and Management Slide : 12 Risk Based Testing – Basic Principles  Is a method to objectively decide the scope of testing to maximize effectiveness  Basic Premise # 1: Testing is never exhaustive – Dijkstra’s doctrine: “Program Testing can never prove the absence of bugs – only their presence”  Basic Premise #2: There is never enough time for testing  Basic Premise #3: The purpose of testing is to find serious defects before customers find them, and find them early  Basic Premise #4: The purpose of testing is to minimize the impact of the risk of bugs
  13. 13. 5/3/2014 Test Planning and Management Slide : 13 The Essence of Risk Based Testing How Could a program Fail? Can you Design Tests to make it Fail this way Evaluate the Likelihood & Consequences Prioritize & Develop Test cases Risk Based Test Design Risk Based Test Management
  14. 14. 5/3/2014 Test Planning and Management Slide : 14 Visualizing How a Program Could Fail…
  15. 15. 5/3/2014 Test Planning and Management Slide : 15 So the steps are…  #1: Build a Risk Taxonomy  #2: Prioritize the risks by likelihood and impact  #3: Decide which are testable  #4: Design and test those test cases which are high priority and can be testable  #5: Continually re-assess the risks and re-order the list
  16. 16. 5/3/2014 Test Planning and Management Slide : 16  Start with an organization-wide list of factors – An Initial “Risk Brainstorming” to go into your Process Database  Add project-specific failure types – Do the “Risk Brainstorming” as a part of the Project Initiation  Update as needed from industry-standard taxonomy lists available Building a “Risk Taxonomy” or “Risk Catalog” How Could a program Fail?
  17. 17. 5/3/2014 Test Planning and Management Slide : 17 Risk/Bug Taxonomy: Organization-wide Risks  Infrastructure  Attrition / People Factors  Distribution of team in multiple locations  …
  18. 18. 5/3/2014 Test Planning and Management Slide : 18 Risk/Bug Taxonomy: Project Specific Risk Factors  New technology / environment / version  New people  Customer-specific risks  Third party code
  19. 19. 5/3/2014 Test Planning and Management Slide : 19 Risk/Bug Taxonomy: Identifying specific ways how products can break  “Installer” (or Installation product) – Wrong files installed – Temp files left behind – Required files missing – Extra files installed – Installation in wrong location – Older files replace newer files – User data loss – Affects other applications
  20. 20. 5/3/2014 Test Planning and Management Slide : 20 Risk / Bug Taxonomy: Categorize Risks  Categorization groups “logical” risks together  Categorization can be by – Function (Validation, Reporting, etc.) – Functional Area (Order Processing, AP, etc.) – Quality Characteristic (Visibility, Reliability, etc.)
  21. 21. 5/3/2014 Test Planning and Management Slide : 21 Risk/Bug Taxonomy: Check with Industry Standard Lists
  22. 22. 5/3/2014 Test Planning and Management Slide : 22 Prioritize the List of Possible Defects  Ways to Prioritize – By Usage Frequency – By Visibility of Failure – By Impact on Business – By complexity of function – …  Any dimension(s) you use should eventually result in a scale of 1 to n (typically 1-5) for likelihood and a similar scale for impact  The risk value is likelihood * impact
  23. 23. 5/3/2014 Test Planning and Management Slide : 23 Another Method of Identifying the Risk Prone Areas to be Tested  Identify the characteristics to be tested (Functionality, Reliability, Usability, Maintainability and Portability)  Alternatively, identify the functional areas / functions to be tested  Divide a fixed booty of 100 points across these characteristics
  24. 24. 5/3/2014 Test Planning and Management Slide : 24 What do you do when you don’t know anything about the product?  Take a breadth-first view  A little of everything  Use the information gained to fine tune
  25. 25. 5/3/2014 Test Planning and Management Slide : 25 Some Other Methods of Scope Determination / Estimation  Estimating with Use Case Points – http://www.methodsandtools.com/archive/archive.php? id=25  Requirements Based Testing – http://www.stsc.hill.af.mil/CrossTalk/2003/03/mogyorod i.pdf  Test Driven Development – http://www.agiledata.org/essays/tdd.html
  26. 26. 5/3/2014 Test Planning and Management Slide : 26 Having Got The Scope of Testing…
  27. 27. 5/3/2014 Test Planning and Management Slide : 27 Deciding the test approach / strategy for the chosen features  What type of testing would you use for testing the functionality of each feature?  What are the configurations or scenarios for testing the features?  What integration testing would you do to ensure these features work together?  What localization validations would be needed?  What “non functional” tests would you need to do?
  28. 28. 5/3/2014 Test Planning and Management Slide : 28 Setting up criteria for testing  Criteria can be at phase level – See example in page 143, 163, 164  Entry Criteria – Parallelism Vs Ready for use  Exit Criteria – Completeness Vs Risk of release  Suspension Criteria – Show stopper bugs – Crossing a threshold number of bugs – Developers producing a new version making the old one redundant  Resumption Criteria – Above hurdles being cleared
  29. 29. 5/3/2014 Test Planning and Management Slide : 29 Identifying Test Deliverables  The test plan (master test plan, and various other test plans for the project)  Test case design specifications  Test cases, including any automation that is specified in the plan  Test logs produced by running the tests  Test Summary Reports
  30. 30. 5/3/2014 Test Planning and Management Slide : 30 Activity Breakdown and Scheduling
  31. 31. 5/3/2014 Test Planning and Management Slide : 31 Scheduling of Activities  Identifying external and internal dependencies among the activities  Sequencing the activities, based on the expected duration as well as on the dependencies  Identifying the time required for each of the WBS activities, taking into account the above two factors.  Monitoring the progress in terms of time and effort  Rebalancing the schedules and resources as necessary
  32. 32. 5/3/2014 Test Planning and Management Slide : 32 External Dependencies  Availability of the product from developers  Hiring  Training  Acquisition of hardware / software required for training  Availability of translated message files for testing
  33. 33. 5/3/2014 Test Planning and Management Slide : 33 Internal Dependencies for Testing  Completing the test specification  Coding / scripting the tests  Executing the tests
  34. 34. 5/3/2014 Test Planning and Management Slide : 34 Parallelism Constraints for Testing  Machines  People  Software dependency  Test serialization
  35. 35. 5/3/2014 Test Planning and Management Slide : 35 Test Planning and Management: Agenda of the Session  Testing phase as a project  Planning a testing project  Test Management  Test Process  Test Reporting
  36. 36. 5/3/2014 Test Planning and Management Slide : 36 Test Management Integrating With Product Release People Mgmt Test Infrastructure Mgmt Technology Mgmt Test Mgmt
  37. 37. 5/3/2014 Test Planning and Management Slide : 37 Test Technology Management  Naming and storage conventions for test artifacts – Product functionality --- Test suites mapping – Documentation standards – Test coding standards – Test reporting standards (covered separately)
  38. 38. 5/3/2014 Test Planning and Management Slide : 38 Test Infrastructure Management
  39. 39. 5/3/2014 Test Planning and Management Slide : 39
  40. 40. 5/3/2014 Test Planning and Management Slide : 40 People Management (Covered in more detail next session)
  41. 41. 5/3/2014 Test Planning and Management Slide : 41 Integrating with Product Release  Sync points between development and testing as to when the different types of testing can commence. For example, when integration testing could start, when system testing start, etc could.  Service Level Agreements between development and testing as to how long kit would take for the testing team to complete the testing  Consistent definitions of the various priorities and severities of the defects  Communication mechanisms to the documentation group to ensure that the documentation is kept in sync with the product in terms of known defects, workarounds, etc.
  42. 42. 5/3/2014 Test Planning and Management Slide : 42 Test Planning and Management: Agenda of the Session  Testing phase as a project  Planning a testing project  Test Management  Test Process  Test Reporting
  43. 43. 5/3/2014 Test Planning and Management Slide : 43 Test Process  Putting together and baselining a Test Plan (discussion of template)  Getting detailed test specifications – Purpose of test – Items being tested, with their version numbers – Environmental setup needed – Input data required – Steps to be followed – Expected results – Any relationship to other tests  Update of RTM  Identifying possible automation candidates  Developing and baselining test cases  Executing tests and keeping RTM and DR current  Collecting and analyzing metrics (next session)  Preparation of Test Summary Report (refer IEEE template and two slides later)  Recommending product release criteria
  44. 44. 5/3/2014 Test Planning and Management Slide : 44 Test Planning and Management: Agenda of the Session  Testing phase as a project  Planning a testing project  Test Management  Test Process  Test Reporting
  45. 45. 5/3/2014 Test Planning and Management Slide : 45 Test Reporting  Test Incidence Report – Update to DR – Id, product info, component info, defect description, fix info, etc  Test Cycle Report – a summary of the activities carried out during that cycle – defects that were uncovered during that cycle, based on their severity and impact – progress from the previous cycle to the current cycle, in terms of defects fixed – outstanding defects that are yet to be fixed in this cycle – any variations observed in effort or schedule (that can be used for future  Test Summary Report – phase-wise summary and final test summary
  46. 46. 5/3/2014 Test Planning and Management Slide : 46 Test Summary Report  A summary of the activities carried out during the test cycle or phase.  Variance of the activities carried out from the activities planned. This includes – the tests that were planned to be run but could not be run (with reasons), – modifications to tests from what was in the original test specifications (in this case, the TCDB should be updated) – additional tests that were run (that were not in the original test plan), – differences in effort and time taken between what was planned and what was executed – any other deviations from plan  Summary of results – tests that failed, with any root cause descriptions – severity of impact of the defects uncovered by the tests  Comprehensive assessment and recommendation for release – “Fit for release” assessment  Recommendation of release
  47. 47. 5/3/2014 Test Planning and Management Slide : 47 Recommendation for Release  Dijkstra’s Doctrine Again!  Testing team gives management (and customer) an idea of what defects exist and what risks to be expected  Management makes a final call whether to release as is or expend extra resources to fix the reported defects
  48. 48. 5/3/2014 Test Planning and Management Slide : 48 Test Planning and Management: Agenda of the Session  Testing phase as a project  Planning a testing project  Test Management  Test Process  Test Reporting
  49. 49. 5/3/2014 Test Planning and Management Slide : 49 Test Planning and Management: Agenda of the Session - Recap  Testing phase as a project  Planning a testing project  Test Management  Test Process  Test Reporting

×