Presentation for NSW NESA course SDD. Testing & Evaluation

1. Testing and EvaluatingTesting and Evaluating
Software Design and DevelopmentSoftware Design and Development
Chapter 6 summaryChapter 6 summary

2. ContentsContents
Testing the SoftwareTesting the Software
 Comparison of the Solution with the original DesignComparison of the Solution with the original Design
SpecificationSpecification
 Generating relevant Test Data for Complex SolutionsGenerating relevant Test Data for Complex Solutions
 Levels of TestingLevels of Testing
 Live Test DataLive Test Data
 BenchmarkingBenchmarking
 Quality AssuranceQuality Assurance
Reporting on the Testing ProcessReporting on the Testing Process
 DocumentationDocumentation
 CommunicationCommunication

3. Testing the Software SolutionTesting the Software Solution
Testing is carried out to find errorsTesting is carried out to find errors
Software developers must continually ensure that the original specificationsSoftware developers must continually ensure that the original specifications
are being met in the software product being developedare being met in the software product being developed
Construction of appropriate test data and adequate testing prevents theConstruction of appropriate test data and adequate testing prevents the
occurrence of major problems in the finished program that are expensive tooccurrence of major problems in the finished program that are expensive to
fix.fix.
Test data must be sufficient to ensure the program is completely operationalTest data must be sufficient to ensure the program is completely operational
and free of logic errors.and free of logic errors.
Test data must be constructed to test every part of a program, including theTest data must be constructed to test every part of a program, including the
mainline of the program and any modules used by the program.mainline of the program and any modules used by the program.

4. What should be tested?What should be tested?
Test data should testTest data should test
 All parts of the programAll parts of the program
 Each path of execution - test data should be sufficient to test theEach path of execution - test data should be sufficient to test the
termination and correct exit of each loop. To test each path in nested Iftermination and correct exit of each loop. To test each path in nested If
statement / CASE structure.statement / CASE structure.
 Boundary conditions - the values of variables or expressions thatBoundary conditions - the values of variables or expressions that
determine the choice of available options to be taken. exampledetermine the choice of available options to be taken. example
At the very minimum to execution paths should always be tested rather thanAt the very minimum to execution paths should always be tested rather than
boundary conditions. Large programs should be tested using other testingboundary conditions. Large programs should be tested using other testing
mechanisms.mechanisms.
Testing should include :Unit testing, program testing and system testingTesting should include :Unit testing, program testing and system testing

5. Unit or Module TestingUnit or Module Testing
Structured programming results in modules, each of which can be tested individuallyStructured programming results in modules, each of which can be tested individually
Subprograms can be tested as a black box where data is entered and results are shown.Subprograms can be tested as a black box where data is entered and results are shown.
InIn black box testingblack box testing, only the inputs and outputs are checked. The processes that achieve, only the inputs and outputs are checked. The processes that achieve
these results are ignored.these results are ignored.
Subprograms can also be tested as a white box where the algorithm of the subprogram isSubprograms can also be tested as a white box where the algorithm of the subprogram is
understood and each path of execution can be tested appropriately.understood and each path of execution can be tested appropriately.
White box testingWhite box testing examines all the data paths in a module.examines all the data paths in a module.
A driver program may be developed to test modules in a program. The driver substitutes forA driver program may be developed to test modules in a program. The driver substitutes for
the main program, calling the subprogram and supplying the necessary values for anythe main program, calling the subprogram and supplying the necessary values for any
variables.variables.
A driver is a temporary section of code that is created to test an individual procedure orA driver is a temporary section of code that is created to test an individual procedure or
module by calling it up and executing it.module by calling it up and executing it.

6. Program TestingProgram Testing
It is a minimum requirement that the test data tests each logical pathway and programIt is a minimum requirement that the test data tests each logical pathway and program
branch that can be entered.branch that can be entered.
All modules are called and executed at least once or when expectedAll modules are called and executed at least once or when expected
This way, the entire program can be completely tested.This way, the entire program can be completely tested.
Screen elements such as menus and buttons must be tested to ensure they areScreen elements such as menus and buttons must be tested to ensure they are
functioning correctly.functioning correctly.

7. System TestingSystem Testing
During system testing, the program is tested in a variety of operating environments.During system testing, the program is tested in a variety of operating environments.
The software may function in the development system or a controlled environmentThe software may function in the development system or a controlled environment
however, when implemented on the users computer, problems may arise.however, when implemented on the users computer, problems may arise.
The effect of hardware, operating systems and other software may create errors thatThe effect of hardware, operating systems and other software may create errors that
have not been previously detected.have not been previously detected.
System testing is carried out to detect errors at the software / hardware interface,System testing is carried out to detect errors at the software / hardware interface,
Carried out by users rather than developers. System tests treat the program as a blackCarried out by users rather than developers. System tests treat the program as a black
box.box.

8. Live Test DataLive Test Data
Live data is real data, used to ensure that a program works under real-lifeLive data is real data, used to ensure that a program works under real-life
conditions.conditions.
Real data is supplied by the client or can be generated by CASE tools.Real data is supplied by the client or can be generated by CASE tools.
Stress testing involves increasing the load on a program in an attempt to make it fail.Stress testing involves increasing the load on a program in an attempt to make it fail.
Live Test Data should include:Live Test Data should include:
 Larger File SizesLarger File Sizes
 Mix of Transaction TypesMix of Transaction Types
 Response TimesResponse Times
 Volume dataVolume data
 Interfaces between ModulesInterfaces between Modules
 Comparison with Program Test DataComparison with Program Test Data

9. Live dataLive data is used to ensure that the system response times are appropriate. Response times
are dependent on all the system components, together with their interactions with each other
and other processes that may be occurring concurrently. Response times should be tested
on minimum hardware using typical data of different types.
Interface testsInterface tests will ensure that the correct numbers of parameters are sent to and from the
module and that the format is correct. Interface tests will also detect conflicts between local
and global variables.
A variety of different transaction types and sequences of data entry should be tested with live
data. Module and program testing usually involve testing specific transactions or processes
one at a time. During system testing, transactions occur in random order and checked to see
if any errors arise
Software developers generate test data to test the limits of the system that may not be tested
under normal use. (to ensure the system has scalability)
A program developed to access files should be tested with a range of file types and sizes.
The use of large files will highlight problems associated with data access.

10. BenchmarkingBenchmarking
A benchmark is a standard against which performance of a computer program can be
assessed against expected outcomes and other similar products on the market.
Established programs are often used as a benchmark to indicate the quality and
performance expectations of a new product.
Benchmarks allow users of software products to make informed purchasing decisions

11. Quality AssuranceQuality Assurance
Quality Assurance is a set of procedures used to certify that a generated product meets specifiedQuality Assurance is a set of procedures used to certify that a generated product meets specified
criteria with respect to quality and reliability.criteria with respect to quality and reliability.
 CCorrectness - Does it do what it is supposed to do?orrectness - Does it do what it is supposed to do?
 RReliability - Does it do it all of the time?eliability - Does it do it all of the time?
 EEfficiency - Does it do it the best way possible?fficiency - Does it do it the best way possible?
 IIntegrity - Is it secure?ntegrity - Is it secure?
 MMaintainability - Can it be understood?aintainability - Can it be understood?
 (Can Roger Eat Icy Mushrooms?)(Can Roger Eat Icy Mushrooms?)
Quality Assurance is not just about testing a product once it has been completed. Its aboutQuality Assurance is not just about testing a product once it has been completed. Its about
periodically performing inspections, reviews and tests on the system being developed.periodically performing inspections, reviews and tests on the system being developed.
Quality assurance techniques should be implemented throughout the software developmentQuality assurance techniques should be implemented throughout the software development
process.process.

12. Alpha & Beta TestingAlpha & Beta Testing
Alpha TestingAlpha Testing
 Testing of the final solution by personnelTesting of the final solution by personnel within the softwarewithin the software
development companydevelopment company prior to the product’s release.prior to the product’s release.
 The client uses the system in a controlled environment and checks to seeThe client uses the system in a controlled environment and checks to see
if it meets their requirements.if it meets their requirements.
Beta Testing(Acceptance Testing)Beta Testing(Acceptance Testing)
 Testing of the final solution by a limited number ofTesting of the final solution by a limited number of users outside theusers outside the
software development companysoftware development company using real world data and conditions.using real world data and conditions.
 The program is given to a number of potential clients who will also reportThe program is given to a number of potential clients who will also report
to the developers any problems they encounter in the program.to the developers any problems they encounter in the program.

13. Reporting on the Testing ProcessReporting on the Testing Process
Required DocumentationRequired Documentation
 A test data table should be created to show the test data to be usedA test data table should be created to show the test data to be used
and the reason why this item of test data was selected.and the reason why this item of test data was selected.
 Test requirements - What needs to be tested?Test requirements - What needs to be tested?
 Test plan - How do we implement these tests?Test plan - How do we implement these tests?
 Test data and expected results - What are the necessary inputs andTest data and expected results - What are the necessary inputs and
the expected outputs?the expected outputs?
 Test results - Do the actual results match the expected results?Test results - Do the actual results match the expected results?
 Recommendations - What needs to be done nowRecommendations - What needs to be done now

14. A desk-check table is used to document the test dataA desk-check table is used to document the test data
used and compare the expected output with the actualused and compare the expected output with the actual
output of the algorithm or program.output of the algorithm or program.
CASE Tools that aid in the testing process:CASE Tools that aid in the testing process:
 Tools to generate or acquire data to be used duringTools to generate or acquire data to be used during
testingtesting
 Tools that analyse the source codeTools that analyse the source code
 Simulation tools to mimic the roles of hardware orSimulation tools to mimic the roles of hardware or
other software that interacts with the programother software that interacts with the program

15. CommunicationCommunication
Independent testers are less likely to approach the program withIndependent testers are less likely to approach the program with
preconceptions.preconceptions.
It is essential that the user is provided with the opportunity to evaluate theIt is essential that the user is provided with the opportunity to evaluate the
solution that has been developed.solution that has been developed.
Results of the testing provides an opportunity for the users to evaluate andResults of the testing provides an opportunity for the users to evaluate and
discuss the functionality of the new system.discuss the functionality of the new system.
Alpha and Beta testing help to provide users with the opportunity to use theAlpha and Beta testing help to provide users with the opportunity to use the
program.program.
After the customer has approved a program, it can be released to the generalAfter the customer has approved a program, it can be released to the general
market.market.

16. The EndThe End

17. Boundary ConditionsBoundary Conditions
CASEWHERE age isCASEWHERE age is
<=3: output “Too young for school”<=3: output “Too young for school”
<=18: output “Could be at school”<=18: output “Could be at school”
OTHERWISE: output “Too old for school”OTHERWISE: output “Too old for school”
ENDCASEENDCASE
An appropriate set of data could therefore beAn appropriate set of data could therefore be
22 33 1818 20.20.
BackBack

18. Black box TestingBlack box Testing
BackBack

19. White box TestingWhite box Testing
BackBack