Testing in the lifecycle


Published on

Testing in the lifecycle

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Testing in the lifecycle

  1. 1. TESTING IN THE LIFECYCLE Hoang V. Nguyen startnewday85@gmail.com startnewday85.blogspot.com Dept. of Computer Science – FIT - HUATester Training Course
  2. 2. Tester Training Course 2011
  3. 3. Tester Training Course 2011
  4. 4. User Acceptancerequirements Testing System System Testing requirements Integration Global Design Testing Detailed Design Unit Testing implementationTester Training Course 2011
  5. 5.  Also know as component, module, program testing  Each “part” is tested in isolation  lowest level and most thorough look at detail  Goals: • to ensure that the code written for the unit meets its specification, prior to its integration with other units • both functional and non-functional characteristics  Test bases: • component requirements • detailed design • the codeTester Training Course 2011
  6. 6.  Who do it? • usually done by programmer  Reports • Defects are typically fixed as soon as they are found, with out formally recording the incidents found  Tools: • Programmer often writes pieces of code for unit test • Mock frameworks • Dependency Injection and IoC Containers • Unit test frameworksTester Training Course 2011
  7. 7.  Programmers: • have to create pieces of code are separately testable  Test Driven Development • An approach to software development using Unit Test • Test –first approach • is an iterative approach to unit test • is good in practice A A Driver • Stubs and Drivers B Stubs BTester Training Course 2011
  8. 8.  more than two components/systems  Goals: • discover defects in interface and interaction between components/systems • what the set can perform that is not possible individually • both functional and non-functional characteristics  Test bases: • software and system design • software architecture • workflows and use-casesTester Training Course 2011
  9. 9.  There may be more than one level of integration testing • component integration testing • system integration testing  Who do it? • Developer • Designer • Independent testerTester Training Course 2011
  10. 10.  Integration strategy • big-bang • incrementalTester Training Course 2011
  11. 11.  In theory • if we have already tested components why not just combine them all at once? Wouldn’t this save time? • based on false assumption of no faults  In practice: difficult to isolate any errors found • takes longer to locate and fix faults • re-testing after fixes more extensive ==> don’t save timeTester Training Course 2011
  12. 12.  In theory • start with a tested component/system • increment: add a tested component/system and test it • baselines  Advantages • easier fault location and fix • easier recovery from disaster/problems  Some types • top-down • bottom-up • functionalTester Training Course 2011
  13. 13.  Baselines • baseline 0: a tested component a a • baseline 1: a+b b c • baseline 2: a+b+c • baseline 3: a+b+c+d d e f g • … Need to call to lower level h i j k l mcomponents not yet integrated n o Stubs Stubs: simulate missingcomponentsTester Training Course 2011
  14. 14.  Advantages: • critical control structure tested first and most often • can demonstrate system early (show working menus) Disadvantages: • needs stubs • detail left until last • may be difficult to "see" detailed output (but should have been tested in component test) • may look more finished than it isTester Training Course 2011
  15. 15.  Baselines Drivers • baseline 0: component n a • baseline 1: n+i b c • baseline 2: n+i+o • baseline 3: n+i+o+d d e f g • …  Needs driver to call the h i j k l m base line configuration  Also needs stubs for some n o baselines  Drivers: test harness/scaffolding StubsTester Training Course 2011
  16. 16.  Advantages: • lowest levels tested first and most thoroughly (but should have been tested in unit testing) • good for testing interfaces to external environment (hardware, network) • visibility of detail  Disadvantages • no working system until last baseline • needs both drivers and stubs • major control problems found lastTester Training Course 2011
  17. 17. Tester Training Course 2011
  18. 18. Tester Training Course 2011
  19. 19.  Baselines • baseline 0: component a a • baseline 1: a+b b c • baseline 2: a+b+d • baseline 3: a+b+d+h d e f g • … Need Stubs h i j k l m Shouldn’t need drivers(if topdown) n oTester Training Course 2011
  20. 20.  Advantages: • control level tested first and most often • visibility of detail • real working partial system earliest  Disadvantages • needs stubsTester Training Course 2011
  21. 21. Thread Integration order of processing some event a determines integration order interrupt, user transaction b c minimum capability in time d e f g advantages: • critical processing first h i j k l m • early warning of performance problems n o disadvantages: • may need complex drivers and stubsTester Training Course 2011
  22. 22. Integration Guidelines  minimise support software needed  integrate each component only once  each baseline should produce an easily verifiable result  integrate small numbers of components at once • one at a time for critical or fault-prone components • combine simple related components Tester Training Course 2011
  23. 23.  integration should be planned in the architectural design phase  the integration order then determines the build order • components completed in time for their baseline • component development and integration testing can be done in parallel - saves timeTester Training Course 2011
  24. 24. Question Which of the following statements about component testing is not true?Component testing should be performed bydeveloperComponent testing is alse know as isolation ormodule testingComponent testing should have completion criteriaplannedComponent testing does not involve regressiontesting Tester Training Course 2011
  25. 25. Question Which of the following combinations correctly describes a valid aproach to component testing i. Functional testing of the component in isolation ii. Structure-based testing of the code with out recording incidents iii. Automated tests that rare run until the component passes iv. Functional testing of the interfaces between modules i and ii i, ii and iii iii ii and ivTester Training Course 2011
  26. 26. Question Testing activity which is performed to expose defects in the interfaces and in the interaction between integrated components is:System TestingComponent TestingIntegration TestingUnit Testing Tester Training Course 2011
  27. 27. Question A common test technique during component test is Statement and branch testing Usability testing Security testing Performance testingTester Training Course 2011
  28. 28. Question The process starting with the terminal modules is calledTop-down integrationModule integrationBottom-up integrationNone of the above Tester Training Course 2011
  29. 29. Question Why is incremental integration preferred over big bang integration? Because incremental integration has better early defects screening and isolation ability Because big bang integration is suitable only for real time applications Incremental integration is preferred over big bang integration only for bottom-up development model Because incremental integration can compensate for weak and inadequate component testingTester Training Course 2011
  30. 30.  Last integration step  end-to-end perspective  Goals: • discover failure in behaviors of whole system in environment • both functional and non-functional characteristics  Test bases • System and software requirement specifications • Use cases • system, user and operation manuals  Who do it • often and should be independent test teamTester Training Course 2011
  31. 31. • Usability • configuration / installation • Security • reliability / qualities • Documentation • back-up / recovery • Storage • performance, load, stress • volumeTester Training Course 2011
  32. 32. Performance Tests  Timing Tests • response and service times • database back-up times  Capacity & Volume Tests • maximum amount or processing rate • number of records on the system • graceful degradation  Endurance Tests (24-hr operation?) • robustness of the system • memory allocationTester Training Course 2011
  33. 33. Multi-User Tests  Concurrency Tests • small numbers, large benefits • detect record locking problems  Load Tests • the measurement of system behaviour under realistic multi-user load  Stress Tests • go beyond limits for the system - know what will happen • particular relevance for e-commerceTester Training Course 2011
  34. 34. Usability Tests messages tailored and meaningful to (real) users? coherent and consistent interface? sufficient redundancy of critical information? within the "human envelope"? (7±2 choices) feedback (wait messages)? clear mappings (how to escape)? Who should design / perform these tests?Tester Training Course 2011
  35. 35. Security Tests  passwords  encryption  hardware permission devices  levels of access to information  authorisation  covert channels  physical securityTester Training Course 2011
  36. 36. Configuration and Installation Configuration Tests • different hardware or software environment • configuration of the system itself • upgrade paths - may conflict Installation Tests • distribution (CD, network, etc.) and timings • physical aspects: electromagnetic fields, heat, humidity, motion, chemicals, power supplies • uninstall (removing installation)Tester Training Course 2011
  37. 37. Reliability / Qualities  Reliability • "system will be reliable" - how to test this? • "2 failures per year over ten years" • Mean Time Between Failures (MTBF) • reliability growth models  Other Qualities • maintainability, portability, adaptability, etc.Tester Training Course 2011
  38. 38. Back-up and Recovery Back-ups • computer functions • manual procedures (where are tapes stored) Recovery • real test of back-up • manual procedures unfamiliar • should be regularly rehearsed • documentation should be detailed, clear and thoroughTester Training Course 2011
  39. 39. Documentation Testing  Documentation review • check for accuracy against other documents • gain consensus about content • documentation exists, in right format  Documentation tests • is it usable? does it work? • user manual • maintenance documentation Tester Training Course 2011
  40. 40.  Final stage of validation  Goals • To provide the end users with confidence that the system will function according to their expectations  Test bases • Requirement s specification  Who do it • Customers/users • Also can include testersTester Training Course 2011
  41. 41. Tester Training Course 2011
  42. 42.  Done by users  To check that the system meets their business needs  Can include • Factory acceptance testing • Site acceptance testingTester Training Course 2011
  43. 43.  Also called operational readiness testing  To check that the processes and procedures are in place to allow the system to be used and maintained  Include checking: • Back-up facilities • Procedures for disaster recovery • Training for end users • Maintenance procedures • Data load and migration tasks • Security proceduresTester Training Course 2011
  44. 44. Tester Training Course 2011
  45. 45.  The criteria for accepting a system can documented in a contact  To check that these criteria have been met • Not what the user wish they had asked for! • This system, not wish systemTester Training Course 2011
  46. 46. Check that governmental, legal or safety standard have been metTester Training Course 2011
  47. 47.  Similarities: • When software is stable • To collect comments  fault found  how the product meets their expectations  improvement suggestions  Differences : • Alpha testing is done on developer site • Beta testing is done on customer siteTester Training Course 2011
  48. 48. Question System Integration testing should be done afterIntegration testingSystem testingUnit testingComponent integration testing Tester Training Course 2011
  49. 49. Question When is beta testing? Testing performed by potential customers at the developers location. Testing performed by portential customer at their own locations. Testing performed by product developers at the customer ’s location Testing performed by product developers at their own locations.Tester Training Course 2011
  50. 50. Question Which testing is used to verify that the system can perform properly when internal program or system limitations have been exceeedStress testingLoad testingPerformance testingVolume testing Tester Training Course 2011
  51. 51.  Additional features being required  The system being migrated to a new operating platform  The system being retired  Planned upgrade to COTS-based system  New faults being found requiring fixing (these can be ‘hot fixes’)Tester Training Course 2011
  52. 52. Maintenance testing is important!Tester Training Course 2011
  53. 53.  To ensure that software system still works correctly (both new and old characteristics)  We have to: • Breath test to establish overall confidence • Depth test to investigate changes and critical areas  Predominantly regression testingTester Training Course 2011
  54. 54. What to test in maintenance testing  Test any new or changed code  Impact analysis • what could this change have an impact on? • how important is a fault in the impacted area? • test what has been affected, but how much? most important affected areas? areas most likely to be affected? whole system?  The answer: “It depends” Tester Training Course 2011
  55. 55. Question Which of the following is correct?Impact Analysis assesses the effect on the system ofa defect found in regression testingImpact Analysis assesses the effect of a new personjoining the regression test teamImpact Analysis assesses whether or not a defectfound in regression testing has been fixed correctlyImpact Analysis assesses the effect of a change tothe system to determine how much regression to do Tester Training Course 2011
  56. 56. Question Testing where in we subject the target of the test, to varying workloads to measure and evaluate the performance behaviors and ability of the target and of the test to continue to function properly under these different workloads. Unit testing Load testing Usability testing Intergration testingTester Training Course 2011
  57. 57. Question Methodologies adopted while performing maintenance testing:Breadth Test and Depth TestRe-TestingConfirmation TestingSanity testing Tester Training Course 2011
  58. 58. Question What is the normal order of activities in which software testing is orgranized? Unit, integration, system, validation System, integration, unit, validation Unit, integration, validation, system None of the aboveTester Training Course 2011
  59. 59. Question Beta testing is performed at developing organization’s site where as Alpha testing is performed by people at their own locations.TrueFalse Tester Training Course 2011
  60. 60. Question Beta testing is Performed by customers at their own site Performed by customers at their software developer’s site Performed by an independent test team Useful for test bespoke softwareTester Training Course 2011
  61. 61. Question Impact Analysis helps to decideExit CriteriaHow much regression testing should be doneHow many more test cases need to writtenDifferent tools to perform regression testing Tester Training Course 2011