Testing process

7,471 views
7,279 views

Published on

Based on V-model (Extension of waterfall model)
describes testing process,strategy
Unit test,Integration test,System Test,User acceptance test.

Published in: Technology, Business
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
7,471
On SlideShare
0
From Embeds
0
Number of Embeds
2,326
Actions
Shares
0
Downloads
415
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Testing process

  1. 1. SOFTWARE TESTING PROCESS Byungwook Cho C. (bwcho75@gmail.com) Sr Consultant 2008.08.08
  2. 2. ICE BREAK
  3. 3. AGENDA <ul><li>Test model </li></ul><ul><li>Test process </li></ul><ul><li>Static testing </li></ul>
  4. 4. Test model
  5. 5. Test oriented development process model <ul><li>V-Model </li></ul><ul><ul><li>Enhanced traditional waterfall model </li></ul></ul><ul><li>Verification & Validation </li></ul>< V-Model > < Verification & Validation > Dynamic testing Static testing Testing with system Review & Inspection with artifacts from each development step Valid & estimate the system To verify the artifacts that has been produced in each development cycle. Validation Verification
  6. 6. Test Level ※ Integration strategies Lead by QA Team or Customer Lead by QA Team (Specialized for system test) Lead by Dev Team Lead by Dev Team Responsibility Alpha Test Beta Test Use Mock Object (Test Stub ,Driver) Use Code coverage Black box test Black box test White box test White box test Testing Type Verify customer’s requirement (End User) Verify customer’s Legal issue (Legal) Verify customer’s maintanance issue (SM) Acceptance Test Test system over production environment to verify system itself and production environment Include Functional & Non Functional Test (Availibility, Stablity, Extendability, Performance etc) System Test Verify integration between component. Verify software flow ,interface & interaction Integration Test Verify software component Unit Test Description Level Small project SOA svc BPM Ex. Short time Can test implementation logic easily find architectural defect Adv Most important & high risk component first Integrate all in time Integrate from bottom (use test driver) Integrate from top (use test stub) Desc Backbone Big-bang Bottom up Top down
  7. 7. Test Cycle <ul><li>Each test is consists of test cycle </li></ul><ul><li>Test cycle </li></ul><ul><ul><li>Pre test - check up testability </li></ul></ul><ul><ul><li>Main test </li></ul></ul><ul><ul><li>Conformation test - check up the defect found in previous “Main test” </li></ul></ul><ul><ul><li>Regression test - check up the impact from change </li></ul></ul>TEST CYCLE Pre Test Main Test Conformation Test Regression Test
  8. 8. Test Process
  9. 9. Test Process Master Test Plan Organization Policy Strategy Mgmt sub tests Acceptance Test Plan System Test Plan Integration Test Plan Unit Test Plan <ul><li>Test plan </li></ul><ul><ul><li>Define objective of Test and scope </li></ul></ul><ul><ul><li>Define Test strategy </li></ul></ul><ul><ul><li>Test estimation </li></ul></ul><ul><ul><li>Set up test organization </li></ul></ul><ul><li>Test analysis and design </li></ul><ul><ul><li>Evaluation test object & basis </li></ul></ul><ul><ul><li>Designing and prioritizing test case </li></ul></ul><ul><ul><li>Identify test </li></ul></ul><ul><ul><li>Identify & prepare test data </li></ul></ul><ul><ul><li>Designing test env & identify required infra & tools </li></ul></ul><ul><ul><li>Make test procedure </li></ul></ul><ul><li>Test Implementation & Execution </li></ul><ul><ul><li>Implement TC </li></ul></ul><ul><ul><li>Implement test script </li></ul></ul><ul><ul><li>Set up test environment </li></ul></ul><ul><ul><li>Pre Test </li></ul></ul><ul><ul><li>Run Test </li></ul></ul><ul><ul><li>Logging Test Result </li></ul></ul><ul><li>Test Closure activity </li></ul><ul><ul><li>Store test-ware into repository </li></ul></ul><ul><ul><li>Analyzing lesson learned for future test maturity </li></ul></ul><ul><li>Evaluating exit criteria and reporting </li></ul><ul><ul><li>Write a test summary report for stakeholders. </li></ul></ul><ul><ul><li>Assessing if more test are needed </li></ul></ul>Test control Test mgmt & control Test monitoring & reporting <ul><li>Test plan </li></ul><ul><ul><li>Define objective of Test and scope </li></ul></ul><ul><ul><li>Define Test strategy </li></ul></ul><ul><ul><li>Test estimation </li></ul></ul><ul><ul><li>Set up test organization </li></ul></ul><ul><li>Test analysis and design </li></ul><ul><ul><li>Evaluation test object & basis </li></ul></ul><ul><ul><li>Designing and prioritizing test case </li></ul></ul><ul><ul><li>Identify test </li></ul></ul><ul><ul><li>Identify & prepare test data </li></ul></ul><ul><ul><li>Designing test env & identify required infra & tools </li></ul></ul><ul><ul><li>Make test procedure </li></ul></ul><ul><li>Test Implementation & Execution </li></ul><ul><ul><li>Implement TC </li></ul></ul><ul><ul><li>Implement test script </li></ul></ul><ul><ul><li>Set up test environment </li></ul></ul><ul><ul><li>Pre Test </li></ul></ul><ul><ul><li>Run Test </li></ul></ul><ul><ul><li>Logging Test Result </li></ul></ul><ul><li>Test Closure activity </li></ul><ul><ul><li>Store test-ware into repository </li></ul></ul><ul><ul><li>Analyzing lesson learned for future test maturity </li></ul></ul><ul><li>Evaluating exit criteria and reporting </li></ul><ul><ul><li>Write a test summary report for stakeholders. </li></ul></ul><ul><ul><li>Assessing if more test are needed </li></ul></ul>Test control Test mgmt & control Test monitoring & reporting <ul><li>Test plan </li></ul><ul><ul><li>Define objective of Test and scope </li></ul></ul><ul><ul><li>Define Test strategy </li></ul></ul><ul><ul><li>Test estimation </li></ul></ul><ul><ul><li>Set up test organization </li></ul></ul><ul><li>Test analysis and design </li></ul><ul><ul><li>Evaluation test object & basis </li></ul></ul><ul><ul><li>Designing and prioritizing test case </li></ul></ul><ul><ul><li>Identify test </li></ul></ul><ul><ul><li>Identify & prepare test data </li></ul></ul><ul><ul><li>Designing test env & identify required infra & tools </li></ul></ul><ul><ul><li>Make test procedure </li></ul></ul><ul><li>Test Implementation & Execution </li></ul><ul><ul><li>Implement TC </li></ul></ul><ul><ul><li>Implement test script </li></ul></ul><ul><ul><li>Set up test environment </li></ul></ul><ul><ul><li>Pre Test </li></ul></ul><ul><ul><li>Run Test </li></ul></ul><ul><ul><li>Logging Test Result </li></ul></ul><ul><li>Test Closure activity </li></ul><ul><ul><li>Store test-ware into repository </li></ul></ul><ul><ul><li>Analyzing lesson learned for future test maturity </li></ul></ul><ul><li>Evaluating exit criteria and reporting </li></ul><ul><ul><li>Write a test summary report for stakeholders. </li></ul></ul><ul><ul><li>Assessing if more test are needed </li></ul></ul>Test control Test mgmt & control Test monitoring & reporting <ul><li>Test plan </li></ul><ul><ul><li>Define objective of Test and scope </li></ul></ul><ul><ul><li>Define Test strategy </li></ul></ul><ul><ul><li>Test estimation </li></ul></ul><ul><ul><li>Set up test organization </li></ul></ul><ul><li>Test analysis and design </li></ul><ul><ul><li>Evaluation test object & basis </li></ul></ul><ul><ul><li>Designing and prioritizing test case </li></ul></ul><ul><ul><li>Identify test </li></ul></ul><ul><ul><li>Identify & prepare test data </li></ul></ul><ul><ul><li>Designing test env & identify required infra & tools </li></ul></ul><ul><ul><li>Make test procedure </li></ul></ul><ul><li>Test Implementation & Execution </li></ul><ul><ul><li>Implement TC </li></ul></ul><ul><ul><li>Implement test script </li></ul></ul><ul><ul><li>Set up test environment </li></ul></ul><ul><ul><li>Pre Test </li></ul></ul><ul><ul><li>Run Test </li></ul></ul><ul><ul><li>Logging Test Result </li></ul></ul><ul><li>Test Closure activity </li></ul><ul><ul><li>Store test-ware into repository </li></ul></ul><ul><ul><li>Analyzing lesson learned for future test maturity </li></ul></ul><ul><li>Evaluating exit criteria and reporting </li></ul><ul><ul><li>Write a test summary report for stakeholders. </li></ul></ul><ul><ul><li>Assessing if more test are needed </li></ul></ul>Test control Test mgmt & control Test monitoring & reporting
  10. 10. Test Plan <ul><li>Test planning & control </li></ul><ul><ul><li>Define activity of each testing level </li></ul></ul><ul><ul><li>Define milestone ,resource ,schedule. </li></ul></ul><ul><ul><li>Make a plan based on analyzed risk factor </li></ul></ul><ul><ul><li>Define test strategy based on risk </li></ul></ul><ul><ul><li>Define approach & techniques for testing (testing techniques, coverage, test item, test ware) </li></ul></ul><ul><ul><li>Must include time & resource for preparing testware </li></ul></ul><ul><ul><li>Define completion condition </li></ul></ul>Define test scope objective Risk anal Set up strategy Estimate resource & time Planning PRE TEST TEST Monitoring Report
  11. 11. Test Plan <ul><li>Risk analysis </li></ul><ul><ul><li>Risk = Likelihood * impact </li></ul></ul><ul><ul><li>Risk identification </li></ul></ul><ul><ul><ul><li>Recommend < 36 identification . </li></ul></ul></ul><ul><ul><ul><li>Likelihood : Complexity, Implementation technical Level, Size,Developer skill </li></ul></ul></ul><ul><ul><ul><li>Impact : Biz impact </li></ul></ul></ul>Erik van neneendaal, Risk Based Testing, STAREAST,2006 factor Risk item Risk item factor factor factor Impact Likelihood Likelihood Impact STA (Severe Test Area) SSTA (Strong Test Area) FTA (Fundamental Test Area) ITA (Intensive Test Area)
  12. 12. Test Plan <ul><li>Risk based test strategy </li></ul><ul><li>Other strategies </li></ul><ul><ul><li>Analytical approaches </li></ul></ul><ul><ul><li>Model based approaches </li></ul></ul><ul><ul><li>Methodical approaches </li></ul></ul><ul><ul><li>Dynamic and heuristic approaches </li></ul></ul>( Low Level Test) ( High Level Test) - Unit Test - Integration Test - System Test - Acceptance Test Likelihood Impact STA (Severe Test Area) SSTA (Strong Test Area) FTA (Fundamental Test Area) ITA (Intensive Test Area) Likelihood Impact STA (Severe Test Area) SSTA (Strong Test Area) FTA (Fundamental Test Area) ITA (Intensive Test Area)
  13. 13. Test Analysis and Design <ul><li>Test analysis </li></ul><ul><ul><li>Review - test basis, testability </li></ul></ul><ul><ul><li>Identify test requirement & test data </li></ul></ul><ul><ul><li>Identify test infra & tools </li></ul></ul><ul><li>Test design </li></ul><ul><ul><li>Identify test condition </li></ul></ul><ul><ul><li>Test case specification </li></ul></ul><ul><ul><li>Define test procedure </li></ul></ul>
  14. 14. Test Analysis and Design <ul><li>Design technique </li></ul><ul><ul><li>Specification based </li></ul></ul><ul><ul><ul><li>Equivalence partitioning, Boundary value analysis </li></ul></ul></ul><ul><ul><ul><li>Pairwise, Decision table testing </li></ul></ul></ul><ul><ul><ul><li>State transition testing, Usecase testing </li></ul></ul></ul><ul><ul><li>Structure based </li></ul></ul><ul><ul><ul><li>Control Flow Test </li></ul></ul></ul><ul><ul><ul><li>Basic Path Test – Based on Cyclomatic complexity (Edge – Node + 2P) P = number of connected component </li></ul></ul></ul><ul><ul><ul><li>Elementary comparision test – Based on MC/DC Coverage </li></ul></ul></ul><ul><ul><ul><li>* Coverage : Statement, Condition, Decision, Condition Decision Coverage, Modified Condition/Descision Coverage(MC/DC) , Multiple Condition Coverage </li></ul></ul></ul><ul><ul><li>Experience based </li></ul></ul><ul><ul><ul><li>Exploratory Testing – Adhoc testing, based on test charter, define timezone </li></ul></ul></ul><ul><ul><ul><li>Classification Tree method </li></ul></ul></ul><ul><ul><ul><li>CheckList </li></ul></ul></ul>
  15. 15. Test Execution <ul><li>Execution </li></ul><ul><ul><li>Define serverity & priority </li></ul></ul><ul><ul><li>Develop TC,Test data,Test stub & driver,Test script,Test suite </li></ul></ul><ul><li>Reporting </li></ul><ul><ul><li>Logging expected result, real result </li></ul></ul><ul><ul><li>Current status </li></ul></ul><ul><ul><li>Time & resource usage </li></ul></ul><ul><li>Defect tracking </li></ul><ul><ul><li>Use Bug(Issue) tracking system ( Mantis,Bugzilla,JIRA,Trac) </li></ul></ul><ul><ul><li>Define tracking workflow </li></ul></ul>Test Execution Test Report Defects Issue Tracking System Progress Report Test logs TEST DRIVER TEST STUB TARGET SYSTEM TEST CASE TESTING TOOL TEST DATA MODULE MODULE MODULE MODULE TEST SCRIPT TEST SUITE1 TEST SUITE2 TEST SIMULATOR
  16. 16. Evaluating exit criteria and reporting <ul><li>Test exit criteria </li></ul><ul><ul><li>Coverage, Duration, number of unresolved defects </li></ul></ul><ul><li>Test result report </li></ul><ul><ul><li>Progress report </li></ul></ul><ul><ul><li>Release advice </li></ul></ul><ul><ul><li>Final report per Level </li></ul></ul><ul><li>Matrix </li></ul><ul><ul><li>Defects per KLOC, Days test effort per requirement, Unresolved defect impact analysis, Defects / Hour ,Cumulative defects / Day ,Test coverage / Day , Risk vs number of TC </li></ul></ul>
  17. 17. Test closure activities <ul><li>Test closure activities </li></ul><ul><ul><li>Evaluating </li></ul></ul><ul><ul><ul><li>Evaluate test target </li></ul></ul></ul><ul><ul><ul><li>Evaluate test process for maturing next text </li></ul></ul></ul><ul><ul><li>Final reporting </li></ul></ul><ul><ul><li>Store test-ware </li></ul></ul>
  18. 18. Static Testing
  19. 19. Static testing techniques <ul><li>Performed in Verification phase </li></ul><ul><ul><li>Test with artifacts not a runtime software </li></ul></ul><ul><ul><li>Test by human </li></ul></ul>
  20. 20. Manual Review <ul><li>Inspection </li></ul><ul><ul><li>Objective : To find defects </li></ul></ul><ul><ul><li>Inspect artifacts </li></ul></ul><ul><ul><li>Lead by moderator </li></ul></ul><ul><li>Walkthrough </li></ul><ul><ul><li>Objective : Knowledge sharing </li></ul></ul><ul><ul><li>Knowledge transfer by author </li></ul></ul><ul><li>Code review </li></ul><ul><ul><li>Objective : Find defects and share knowledge </li></ul></ul><ul><ul><li>Lead by Programming Leader </li></ul></ul><ul><li>Pair programming </li></ul><ul><ul><li>XP </li></ul></ul><ul><ul><li>Lead by mentor or coworker </li></ul></ul><ul><ul><li>Share just “ONE” keyboard </li></ul></ul>Formal action Informal action
  21. 21. Static Analysis <ul><li>Static Analysis </li></ul><ul><ul><li>Test code or diagram with “static analysis tool” </li></ul></ul><ul><ul><li>Analysis </li></ul></ul><ul><ul><ul><li>Can find syntax error or well known bugs </li></ul></ul></ul><ul><ul><ul><li>Validate naming rules </li></ul></ul></ul><ul><ul><ul><li>Can find dead code or missing parameter </li></ul></ul></ul><ul><ul><ul><li>Can analysis code complexity & dependency </li></ul></ul></ul><ul><ul><li>Quality? </li></ul></ul><ul><ul><ul><li>Not so smart </li></ul></ul></ul><ul><ul><ul><li>It need a rule and need a maturing </li></ul></ul></ul><ul><ul><li>Easy to appeal your manager </li></ul></ul><ul><ul><li>Tools </li></ul></ul><ul><ul><ul><li>Open source – PMD, Find Bugs </li></ul></ul></ul><ul><ul><ul><li>Commercial – Klockworks, Jtest etc. </li></ul></ul></ul>

×