Testing Framework


Published on

Published in: Technology
1 Comment
1 Like
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Testing Framework

  1. 1. Welcome IPMA and SolutionsIQ Professional Event Testing, Testing, 1…2…3… Improving software quality -- one bug at a time
  2. 2. Agenda <ul><li>Building the Test Framework </li></ul><ul><ul><li>Jan McCollum, SolutionsIQ </li></ul></ul><ul><ul><ul><li>Break </li></ul></ul></ul><ul><li>Practical Panel Discussion </li></ul><ul><ul><li>Cheryl Hainje – AFRS Product Manager, OFM </li></ul></ul><ul><ul><li>Dotti Lane – QA Project Manager, OFM </li></ul></ul><ul><ul><li>Tim Vessey – POS Project Manager, LCB </li></ul></ul><ul><ul><li>Stein Wang – Quality Assurance Lead, SolutionsIQ </li></ul></ul><ul><ul><ul><li>Break </li></ul></ul></ul><ul><li>Testing Templates & Checklists </li></ul>
  3. 3. SolutionsIQ Overview <ul><li>SolutionsIQ is a full-spectrum IT services company </li></ul><ul><li>25 years of technology services and solutions </li></ul><ul><li>400+ consultants </li></ul><ul><li>Corporate headquarters (Bellevue, WA) </li></ul><ul><li>Professional Services (Bellevue, WA) </li></ul><ul><li>Oregon Branch Office (Lake Oswego, OR) </li></ul><ul><li>8+ years of serving the State of WA </li></ul><ul><ul><li>DOC, AOC, LCB, DNR, DOL, LNI, & DSHS </li></ul></ul>
  4. 4. SolutionsIQ Expertise <ul><li>Professional Services Division </li></ul><ul><ul><li>Consulting and Analytical Solutions </li></ul></ul><ul><ul><ul><li>Project management </li></ul></ul></ul><ul><ul><ul><li>Assessments and feasibility studies </li></ul></ul></ul><ul><ul><ul><li>Design and architecture roadmaps </li></ul></ul></ul><ul><ul><li>Development and Test Solutions </li></ul></ul><ul><ul><ul><li>Full life cycle development projects </li></ul></ul></ul><ul><ul><ul><li>Custom application development </li></ul></ul></ul><ul><ul><ul><li>EAI, portals, and business intelligence </li></ul></ul></ul><ul><ul><ul><li>Quality assurance and testing solutions </li></ul></ul></ul>
  5. 5. Building the Testing Framework Jan McCollum Manager, Quality Assurance and Testing Solutions
  6. 6. Setting Goals <ul><li>Knowing WHAT you want is as important as knowing how to get it </li></ul><ul><ul><li>Defining the vision </li></ul></ul><ul><ul><li>Defining the timeline </li></ul></ul><ul><ul><li>Gaining acceptance and buy in </li></ul></ul>
  7. 7. Defining the Vision <ul><li>To define the vision look at what came before </li></ul><ul><ul><li>What went well </li></ul></ul><ul><ul><li>What went badly </li></ul></ul><ul><ul><li>What now </li></ul></ul><ul><ul><li>Where do you want to go </li></ul></ul>
  8. 8. Testing vs. Quality Assurance <ul><li>Testing is about finding bugs </li></ul><ul><li>Quality Assurance is about preventing them! </li></ul>
  9. 9. Quality Assurance <ul><li>Takes time </li></ul><ul><li>Is about the overall effort – including development </li></ul><ul><li>Methodologies can be very formal </li></ul>
  10. 10. QA Applied to Testing <ul><li>Quality assurance principals applied to the testing effort will produce higher quality work </li></ul>
  11. 11. Establishing a Timeline <ul><li>The 6 month / 1 year / 3 year plan </li></ul><ul><ul><li>Implement processes and strategies that give the best return on investment </li></ul></ul>
  12. 12. Quality Testing Roadmap <ul><li>After the goals and objectives are complete, make them real by publishing the quality testing roadmap </li></ul>
  13. 13. Quality Testing Roadmap <ul><li>Roadmap should include… </li></ul><ul><ul><li>Test team structure </li></ul></ul><ul><ul><li>Communications plans </li></ul></ul><ul><ul><li>Test processes </li></ul></ul><ul><ul><li>Test procedures </li></ul></ul>
  14. 14. Quality Testing Roadmap <ul><li>Test scope </li></ul><ul><li>Test dependencies and impacts </li></ul><ul><li>Automation transition plan </li></ul><ul><li>Test deliverables </li></ul>
  15. 15. Gaining Acceptance and Buy In <ul><li>Development </li></ul><ul><li>Business management </li></ul><ul><li>Project management </li></ul><ul><li>IT management </li></ul><ul><li>Customer/product support </li></ul>
  16. 16. Making it Happen! <ul><li>Organizational structure </li></ul><ul><li>Qualified candidates </li></ul><ul><li>Roles and responsibilities </li></ul>
  17. 17. Test Planning <ul><li>The master test plan: a one-stop shopping guide for your project </li></ul><ul><ul><li>Contents </li></ul></ul><ul><ul><li>Contributing documents </li></ul></ul><ul><ul><li>Sign-off procedures </li></ul></ul>
  18. 18. Test Planning <ul><li>Test matrix and test suites </li></ul><ul><ul><li>Detailed test steps </li></ul></ul><ul><ul><li>Pass/Fail results </li></ul></ul><ul><ul><li>Tester who performed tests </li></ul></ul>
  19. 19. Test Planning <ul><li>Test case design – what is a good test case? </li></ul><ul><ul><li>Accurate – tests what it’s designed to test </li></ul></ul><ul><ul><li>Repeatable, reusable – has a life after this release </li></ul></ul><ul><ul><li>Economical – no unnecessary steps </li></ul></ul>
  20. 20. Test Planning <ul><li>Test case design </li></ul><ul><ul><li>Traceable to a requirement </li></ul></ul><ul><ul><li>Appropriate for test environment, testers </li></ul></ul><ul><ul><li>Self-standing has enough information for anyone to run </li></ul></ul>
  21. 21. Test Planning <ul><li>Test case design: How to make good test cases better </li></ul><ul><ul><li>Setup, environment, data </li></ul></ul><ul><ul><li>Steps, actions and expected results </li></ul></ul><ul><ul><li>Use active voice in expected results </li></ul></ul><ul><ul><li>System displays this, does that </li></ul></ul><ul><ul><li>Simple, conversational language </li></ul></ul>
  22. 22. Test Planning <ul><li>Test case design: Why work to improve test cases? </li></ul><ul><ul><li>Productivity – less time to write and maintain cases </li></ul></ul><ul><ul><li>Testability – less time to execute them </li></ul></ul><ul><ul><li>Scheduling – better reliability in estimates </li></ul></ul>
  23. 23. Defect (Bug) Management <ul><li>Deciding upon a tool </li></ul><ul><ul><li>Easy of configuration </li></ul></ul><ul><ul><li>Ability to add/change fields </li></ul></ul><ul><ul><li>Reporting capabilities </li></ul></ul><ul><li>Integrated solution </li></ul>
  24. 24. Defect (Bug) Management <ul><li>The bug lifecycle </li></ul><ul><ul><li>Who can create bugs </li></ul></ul><ul><ul><li>Who can assign bugs </li></ul></ul><ul><ul><li>Who can close bugs </li></ul></ul>
  25. 25. Defect (Bug) Management <ul><li>The bug triage meeting </li></ul><ul><ul><li>Purpose and who should go </li></ul></ul><ul><li>Reporting </li></ul><ul><ul><li>Determining a trend </li></ul></ul><ul><li>Bug metrics </li></ul><ul><ul><li>Number of bugs found </li></ul></ul><ul><ul><li>Bugs found in production vs. test cycle </li></ul></ul>
  26. 26. Moving On <ul><li>Improving the process: Requirements traceability </li></ul><ul><ul><li>Test cases for each requirement </li></ul></ul><ul><ul><li>Requirements matrix </li></ul></ul><ul><ul><li>Tracing requirements to defects </li></ul></ul>
  27. 27. Moving On <ul><li>Improving the process: Risk-based testing </li></ul><ul><ul><li>You can’t test everything so test what is important </li></ul></ul><ul><ul><li>The risk list and how to use it to drive test strategy </li></ul></ul>
  28. 28. Broadening Your Scope <ul><li>Build verification testing </li></ul><ul><ul><li>Also called smoke or acceptance tests </li></ul></ul><ul><ul><li>Is a subset of the major functional areas </li></ul></ul><ul><li>Integration testing </li></ul><ul><ul><li>Testing the entire system </li></ul></ul>
  29. 29. Broadening Your Scope <ul><li>Compatibility testing </li></ul><ul><ul><li>How application works with other apps </li></ul></ul><ul><li>Configuration testing </li></ul><ul><ul><li>Testing on different configurations </li></ul></ul><ul><li>Setup testing </li></ul><ul><ul><li>Testing the installation </li></ul></ul><ul><li>Regression testing </li></ul><ul><ul><li>Verify if bug fixes are successful </li></ul></ul>
  30. 30. Broadening Your Scope <ul><li>Black box testing </li></ul><ul><li>White box testing </li></ul><ul><li>Grey box testing </li></ul>
  31. 31. Improving Quality <ul><li>Testing metrics – measure your success </li></ul><ul><li>Bug tracking metrics </li></ul><ul><ul><li>Number found </li></ul></ul><ul><ul><li>Number found per component </li></ul></ul><ul><ul><li>Daily bug find rate </li></ul></ul>
  32. 32. Improving Quality <ul><li>Test case effectiveness </li></ul><ul><ul><li>Metric: Test case effectiveness; test case effectiveness = bugs found in test/total found * 100 </li></ul></ul><ul><li>Test coverage </li></ul><ul><ul><li>Metric: Test coverage (absolute) = tests conducted/total tests * 100 </li></ul></ul>
  33. 33. Improving Quality <ul><li>Test team performance </li></ul><ul><ul><li>Metric: Test process effectiveness: test process effectiveness = bugs fixed/bugs found * 100 </li></ul></ul><ul><ul><li>Metric: Planned days vs. actual days in test </li></ul></ul>
  34. 34. Improving Quality <ul><li>QA and test involvement early! </li></ul><ul><li>Design reviews </li></ul><ul><ul><li>Why testers should attend </li></ul></ul><ul><li>Develop and use checklists </li></ul><ul><li>Project closeout meetings </li></ul><ul><ul><li>You should have them </li></ul></ul>
  35. 35. Questions? <ul><li>For additonal information, email </li></ul><ul><ul><li>[email_address] </li></ul></ul>