Testing and Evaluating Software Design and Development Chapter 6 summary
ContentsTesting the Software Comparison of the Solution with the original Design Specification Generating relevant Test Data for Complex Solutions Levels of Testing Live Test Data Benchmarking Quality AssuranceReporting on the Testing Process Documentation Communication
Testing the Software SolutionTesting is carried out to find errorsSoftware developers must continually ensure that the original specificationsare being met in the software product being developedConstruction of appropriate test data and adequate testing prevents theoccurrence of major problems in the finished program that are expensive tofix.Test data must be sufficient to ensure the program is completely operationaland free of logic errors.Test data must be constructed to test every part of a program, including themainline of the program and any modules used by the program.
What should be tested?Test data should test All parts of the program Each path of execution - test data should be sufficient to test the termination and correct exit of each loop. To test each path in nested If statement / CASE structure. Boundary conditions - the values of variables or expressions that determine the choice of available options to be taken. exampleAt the very minimum to execution paths should always be tested rather thanboundary conditions. Large programs should be tested using other testingmechanisms.Testing should include :Unit testing, program testing and system testing
Unit or Module TestingStructured programming results in modules, each of which can be tested individuallySubprograms can be tested as a black box where data is entered and results are shown.In black box testing, only the inputs and outputs are checked. The processes that achievethese results are ignored.Subprograms can also be tested as a white box where the algorithm of the subprogram isunderstood and each path of execution can be tested appropriately.White box testing examines all the data paths in a module.A driver program may be developed to test modules in a program. The driver substitutes forthe main program, calling the subprogram and supplying the necessary values for anyvariables.A driver is a temporary section of code that is created to test an individual procedure ormodule by calling it up and executing it.
Program TestingIt is a minimum requirement that the test data tests each logical pathway and programbranch that can be entered.All modules are called and executed at least once or when expectedThis way, the entire program can be completely tested.Screen elements such as menus and buttons must be tested to ensure they arefunctioning correctly.
System TestingDuring system testing, the program is tested in a variety of operating environments.The software may function in the development system or a controlled environmenthowever, when implemented on the users computer, problems may arise.The effect of hardware, operating systems and other software may create errors thathave not been previously detected.System testing is carried out to detect errors at the software / hardware interface,Carried out by users rather than developers. System tests treat the program as a blackbox.
Live Test DataLive data is real data, used to ensure that a program works under real-lifeconditions.Real data is supplied by the client or can be generated by CASE tools.Stress testing involves increasing the load on a program in an attempt to make it fail.Live Test Data should include: Larger File Sizes Mix of Transaction Types Response Times Volume data Interfaces between Modules Comparison with Program Test Data
Live data is used to ensure that the system response times are appropriate. Response timesare dependent on all the system components, together with their interactions with each otherand other processes that may be occurring concurrently. Response times should be testedon minimum hardware using typical data of different types.Interface tests will ensure that the correct numbers of parameters are sent to and from themodule and that the format is correct. Interface tests will also detect conflicts between localand global variables.A variety of different transaction types and sequences of data entry should be tested with livedata. Module and program testing usually involve testing specific transactions or processesone at a time. During system testing, transactions occur in random order and checked to seeif any errors ariseSoftware developers generate test data to test the limits of the system that may not be testedunder normal use. (to ensure the system has scalability)A program developed to access files should be tested with a range of file types and sizes.The use of large files will highlight problems associated with data access.
BenchmarkingA benchmark is a standard against which performance of a computer program can beassessed against expected outcomes and other similar products on the market.Established programs are often used as a benchmark to indicate the quality andperformance expectations of a new product.Benchmarks allow users of software products to make informed purchasing decisions
Quality AssuranceQuality Assurance is a set of procedures used to certify that a generated product meets specifiedcriteria with respect to quality and reliability. Correctness - Does it do what it is supposed to do? Reliability - Does it do it all of the time? Efficiency - Does it do it the best way possible? Integrity - Is it secure? Maintainability - Can it be understood? (CREIM)Quality Assurance is not just about testing a product once it has been completed. Its aboutperiodically performing inspections, reviews and tests on the system being developed.Quality assurance techniques should be implemented throughout the software developmentprocess.
Alpha & Beta TestingAlpha Testing Testing of the final solution by personnel within the software development company prior to the product’s release. The client uses the system in a controlled environment and checks to see if it meets their requirements.Beta Testing(Acceptance Testing) Testing of the final solution by a limited number of users outside the software development company using real world data and conditions. The program is given to a number of potential clients who will also report to the developers any problems they encounter in the program.
Reporting on the Testing ProcessRequired Documentation A test data table should be created to show the test data to be used and the reason why this item of test data was selected. Test requirements - What needs to be tested? Test plan - How do we implement these tests? Test data and expected results - What are the necessary inputs and the expected outputs? Test results - Do the actual results match the expected results? Recommendations - What needs to be done now
A desk-check table is used to document the test dataused and compare the expected output with the actualoutput of the algorithm or program.CASE Tools that aid in the testing process: Tools to generate or acquire data to be used during testing Tools that analyse the source code Simulation tools to mimic the roles of hardware or other software that interacts with the program
CommunicationIndependent testers are less likely to approach the program withpreconceptions.It is essential that the user is provided with the opportunity to evaluate thesolution that has been developed.Results of the testing provides an opportunity for the users to evaluate anddiscuss the functionality of the new system.Alpha and Beta testing help to provide users with the opportunity to use theprogram.After the customer has approved a program, it can be released to the generalmarket.
Boundary ConditionsCASEWHERE age is <=3: output “Too young for school” <=18: output “Could be at school” OTHERWISE: output “Too old for school”ENDCASEAn appropriate set of data could therefore be 2 3 18 20. Back