Testing Software Solutions


Published on

Published in: Technology, Education
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Testing Software Solutions

  1. 1. Testing & Evaluating Software Solutions Testing and Evaluation of Software Solutions
  2. 2. What is Testing? <ul><li>Testing is a process that checks to see whether the solution meets the design specifications and is free from errors. </li></ul><ul><li>This chapter is concerned with more advanced testing techniques that prepare the product for release on the software market. </li></ul>Testing & Evaluation of Software Solutions Testing the Software Solutions Reporting on the testing process
  3. 3. What is Testing? <ul><li>Testing is done throughout the entire software development cycle: </li></ul>Defining & Understanding the problem Planning & Design of Software Solutions Implementation of Software Solutions Feasibility Study Desk Checking Algorithms` Debugging Tools
  4. 4. Comparison against original design specifications <ul><li>The firsts step of testing a solution is to ensure that the program that you have created meets the design specifications outlined during the ‘defining & understanding the problem’ stage. </li></ul><ul><li>Design Specifications – are a list of criteria that must be met in the end product. </li></ul><ul><li>These specifications are no different to the specifications that you would have for buying a new car. </li></ul>
  5. 5. Comparison against original design specifications <ul><li>Outline a list of criteria that you would use for purchasing a car? </li></ul>
  6. 6. Comparison against original design specifications <ul><li>The following guidelines are used to ensure you have created measurable specifications: </li></ul><ul><ul><li>Clarify any vague terms (e.g. ‘most’, ‘some’ and ‘usually’) </li></ul></ul><ul><ul><li>Avoid use of vague verbs (e.g. ‘processed’, ‘eliminated’ and ‘handled’) as they may be interpreted in more that one way. </li></ul></ul><ul><ul><li>Calculations should be accompanied by examples. </li></ul></ul>
  7. 7. Comparison against original design specifications <ul><li>The following guidelines are used to ensure you have created measurable specifications: </li></ul><ul><ul><li>Don not use incomplete statements such as ‘e.t.c’ and ‘so on’. </li></ul></ul><ul><ul><li>Ensure the specification can be physically measured. </li></ul></ul><ul><ul><li>Incorporate pictures and diagrams to show structure. </li></ul></ul>
  8. 8. Comparison against original design specifications <ul><li>Design specifications are often referred to as your ‘objectives’. </li></ul><ul><li>The aim of the project is to implement and solve each objective. </li></ul><ul><li>Objectives are written in such a way that a pass or fail can be allocated to them. </li></ul>
  9. 9. Comparison against original design specifications <ul><li>Identify a tool that could be implemented to test the successful completion of objectives. </li></ul>
  10. 10. Comparison against original design specifications <ul><li>Each objective is tested and if faults are encountered then the aspect of the product that failed is altered until the objective is realised. </li></ul><ul><li>This is why it is important to test your solution continually throughout the development process. </li></ul>
  11. 11. Testing Hierarchy <ul><li>Testing </li></ul><ul><li>Alpha </li></ul><ul><li>Beta </li></ul><ul><li>White-Box </li></ul><ul><li>Black-Box </li></ul><ul><li>Live Data </li></ul><ul><li>Module </li></ul><ul><li>Program </li></ul><ul><li>System </li></ul><ul><li>Large Files </li></ul><ul><li>Response Times </li></ul><ul><li>Volume Data </li></ul>
  12. 12. Broad Categories of Testing <ul><li>Alpha Testing </li></ul><ul><li>Testing by the personnel </li></ul><ul><li>within the software </li></ul><ul><li>development company. </li></ul><ul><li>Beta Testing </li></ul><ul><li>Testing of the final solution by a </li></ul><ul><li>limited number of users outside </li></ul><ul><li>the company in real world </li></ul><ul><li>conditions. </li></ul>
  13. 13. What is Black-Box Testing? <ul><li>Often referred to as ‘functional testing’ as it checks the inputs against expected and actual outputs. </li></ul><ul><li>Does not identify the source of the problem but rather to identify that a problem exists. </li></ul><ul><li>Once the problem has been identified it is necessary to move to ‘white-box’ testing to find and correct the error. </li></ul>
  14. 14. Black-Box Procedures <ul><li>1. Range (Boundary) Checking </li></ul><ul><li>Tests the upper and lower ranges of inputted data. </li></ul><ul><li>E.g. Numbers that may cause errors. </li></ul><ul><li>2. Equivalence Partitioning </li></ul><ul><li>Testing the validation procedures of input fields. </li></ul><ul><li>E.g. Inputting 22a for the age field. </li></ul>
  15. 15. What is White-Box Testing? <ul><li>Is called ‘structural testing’ as it concentrates on processes involved in the program rather than input-output. </li></ul><ul><li>This helps with the generation of test data as an explicit knowledge of the internal workings of the item being tested is needed. </li></ul>
  16. 16. White-Box Procedures <ul><li>1. Statement Coverage Testing </li></ul><ul><li>Test data would be chosen to test each statement in a module. </li></ul><ul><li>E.g. A flag (Test true and false) </li></ul><ul><li>2. Decision Condition Testing </li></ul><ul><li>Test data selected to test each decision. </li></ul><ul><li>E.g. Testing selection and repetition statements. </li></ul>
  17. 17. Generating relevant test data for complex solutions <ul><li>Often the process of generating test data is a expensive and time-consuming event due to the large numbers of variables used within a solution. </li></ul><ul><li>For this reason developers use a wide range of procedure given the situation which are organised into two main techniques: </li></ul><ul><ul><li>Black-Box Testing </li></ul></ul><ul><ul><li>White-Box Testing </li></ul></ul>
  18. 18. Black-Box - Question 1 <ul><li>Create a set of boundary (Range) test data items for an egg sorting machine. The machine works as follows: </li></ul><ul><li>“ Eggs are graded as ‘SMALL’ if they weigh less than 50 grams, ‘MEDIUM’ if their weight is from 50 grams to less than 60 grams., ‘LARGE’ if their weight is from 60 grams to less than 70 grams and ‘EXTRA LARGE’ if their weight is 70 grams or more.” </li></ul>
  19. 19. Answers <ul><li>49 </li></ul><ul><li>50 </li></ul><ul><li>59 </li></ul><ul><li>60 </li></ul><ul><li>69 </li></ul><ul><li>70 </li></ul><ul><li>71 </li></ul>Small Medium Large Extra Large
  20. 20. White-Box - Question 2 <ul><li>Create a set of test data for the module below employing the ‘statement coverage’ method? </li></ul><ul><li>BEGIN </li></ul><ul><li>Get Wage </li></ul><ul><li>Get Tax </li></ul><ul><li>CaseWhere Wage </li></ul><ul><li><$20,000 : Module A </li></ul><ul><li><$40,000 : Module B </li></ul><ul><li>Otherwise : Module C </li></ul><ul><li>EndCase </li></ul><ul><li>IF Tax = True THEN </li></ul><ul><li>Module D </li></ul><ul><li>ELSE </li></ul><ul><li>Module E </li></ul><ul><li>END </li></ul>
  21. 21. Answers <ul><li>$19,999, True </li></ul><ul><li>$19,999, False </li></ul><ul><li>$39,999, True </li></ul><ul><li>$39,999, False </li></ul><ul><li>$40,001, True </li></ul><ul><li>$40,001, False </li></ul>
  22. 22. White-Box - Question 3 <ul><li>Create a set of test data for the module below employing the ‘Decision coverage’ method? </li></ul><ul><li>BEGIN </li></ul><ul><li>Get Wage </li></ul><ul><li>Get Tax </li></ul><ul><li>CaseWhere Wage </li></ul><ul><li><$20,000 : Module A </li></ul><ul><li><$40,000 : Module B </li></ul><ul><li>Otherwise : Module C </li></ul><ul><li>EndCase </li></ul><ul><li>IF Tax = True THEN </li></ul><ul><li>Module D </li></ul><ul><li>ELSE </li></ul><ul><li>Module E </li></ul><ul><li>END </li></ul>
  23. 23. Levels of Testing <ul><li>There are various levels of testing a system. Each is progressive and must be undertaken to ensure success. </li></ul><ul><li>Testing will commence at the lowest level. </li></ul>3. System Testing 2. Program Testing 1. Module Testing
  24. 24. Module (Unit) Testing <ul><li>Each individual module is tested to make sure that it is free from errors. </li></ul><ul><li>Treats the modules as stand-alone applications that do not require any other components of the program to function. </li></ul><ul><li>Normally, the use of libraries of code ensures that most modules used are appropriate. </li></ul>
  25. 25. Module (Unit) Testing <ul><li>Identify the tool that is used to substitute the mainline of the program so that the module can be tested. </li></ul>
  26. 26. Program Testing <ul><li>Ensures that all the modules work correctly together. </li></ul><ul><li>This level concentrates on the interface and the relationship of each module to the main program. </li></ul><ul><li>Uses both bottom-up and top-down testing of the program to ensure that the program meets the criteria set out in the design specifications. </li></ul>
  27. 27. System Testing <ul><li>Involves testing the program in environments outside development. </li></ul><ul><li>Different processors, operating systems, software configurations and peripheral devices may cause errors to occur. </li></ul><ul><li>This testing is done by those outside the development team. </li></ul>
  28. 28. The System 0110011001010101110011 Identify the names of the five (5) elements in a computer system.
  29. 29. Question 4 <ul><li>PaintPro is a new graphic manipulation software that allows users to scan images or input saved images and modify them via a variety of different filters. </li></ul><ul><li>Identify what area system testing would concentrate and the reasons why. </li></ul>
  30. 30. Question 4 - Answer <ul><li>Software </li></ul><ul><li>If scanned images are to be inputted the program must be able to collaborate with appropriate scanner drivers. </li></ul>
  31. 31. Use of Live Test Data <ul><li>Live test data is real test data that the program will work with. </li></ul><ul><li>Up to this point the program has been tested with data created by the development team for the purpose of finding errors. </li></ul><ul><li>Like testing a new car on a test track. </li></ul><ul><li>Results will be obtained regarding how the car performs within the environment in which it will be used. </li></ul>
  32. 32. Use of Live Test Data
  33. 33. Use of Live Test Data <ul><li>For most products live test data should be created to test each of the following conditions: </li></ul><ul><ul><li>Large file sizes </li></ul></ul><ul><ul><li>Mix of transaction times </li></ul></ul><ul><ul><li>Response times </li></ul></ul><ul><ul><li>Volume data </li></ul></ul><ul><ul><li>Interfaces between modules </li></ul></ul><ul><ul><li>Comparison with program test data </li></ul></ul>
  34. 34. Large File Sizes <ul><li>During testing small data files are used. </li></ul><ul><li>It is appropriate to test the program with large data sets to highlight problems with data access. </li></ul><ul><li>For example a program that is created process transactions within a video store will now have to process 10,000 customers and access a video stock list of 15,000 videos. </li></ul>
  35. 35. Large File Sizes <ul><li>The limits of the program need to be tested to reveal any inefficiencies such as: </li></ul><ul><ul><li>Slow processing times. </li></ul></ul><ul><ul><li>Slow network transfer rates. </li></ul></ul><ul><ul><li>Effect on other users or programs. </li></ul></ul>
  36. 36. Question 5 <ul><li>Identify TWO different types of errors that </li></ul><ul><li>could be found when testing large file sizes. </li></ul>
  37. 37. Mix of Transaction Types <ul><li>Highlighting any errors that may occur during transactions being completed. </li></ul><ul><li>This may include: </li></ul><ul><ul><li>Multiple users using the program. </li></ul></ul><ul><ul><li>Manipulation of data in different ways. </li></ul></ul>
  38. 38. Response Times <ul><li>Are the times required to complete a process. </li></ul><ul><li>Any processes that are likely to take more than one second should provide user feedback. </li></ul><ul><li>Both heavy and light loads should be checked. </li></ul>
  39. 39. Volume Data <ul><li>Stress testing to see how the program performs with volumes of data beyond normal expectations is a way of checking efficiency and robustness. </li></ul><ul><li>Multiple users or a large number of processes occurring simultaneously may be required. </li></ul><ul><li>CASE tools can also be used. </li></ul>
  40. 40. Question 6 <ul><li>Banking software programs are faced with a multitude of customers trying to access records continuously because of the introduction of online-banking, EFTPOS and ATM’s. </li></ul><ul><li>Identify the areas that you would target regarding live testing such a system. </li></ul>
  41. 41. Question 6 - Answer <ul><li>Large File Sizes – Trying to see the maximum withdrawals, deposits ect. </li></ul><ul><li>Mix of transaction types – Changing different applications i.e. BPAY, EFTPOS, different computers. </li></ul><ul><li>Response Times – Measure the time taken on different Internet Connections. </li></ul><ul><li>Volume Test Data – 50+ users accessing one account. </li></ul>
  42. 42. Interfaces Between Modules <ul><li>In this section we are interested in testing the performance of interfaces that connect modules and programs. </li></ul><ul><li>Tests should examine the accuracy of data being passed as well as response times and the ability to cope with large volumes of data. </li></ul>
  43. 43. Comparison with test data <ul><li>This involves the use of test data that checks the expected outputs of processing. </li></ul><ul><li>Because the outputs are already known, this testing process ensures the final product is able to perform processing correctly. </li></ul>
  44. 44. Benching Marking <ul><li>Bench marking is the process of evaluating a product through a series of tests to meet a standard of excellence. </li></ul><ul><li>This process involves comparing competitors products to measure quality assurance. </li></ul>
  45. 45. Question 7 <ul><li>Identify the areas that a company may </li></ul><ul><li>employee in which to base its benchmarking </li></ul><ul><li>tests. </li></ul><ul><li>i.e. Overall Performance. </li></ul>
  46. 46. Question 7 - Answer <ul><li>Overall Performance </li></ul><ul><li>Quality </li></ul><ul><li>Usability </li></ul><ul><li>Robustness </li></ul><ul><li>Maintainability </li></ul><ul><li>Productivity </li></ul>
  47. 47. Quality Assurance <ul><li>The quality of a product is measured against how well the product meets or exceeds users’ expectations. </li></ul><ul><li>The following factors are considered: </li></ul><ul><ul><li>Correctness: Does it do what it is supposed to do? </li></ul></ul><ul><ul><li>Reliability: Does it do it all the time? </li></ul></ul><ul><ul><li>Efficiency: Does it do it the best way possible? </li></ul></ul><ul><ul><li>Useability: Is it designed for the end-user. </li></ul></ul><ul><ul><li>Flexibility: Can it be modified? </li></ul></ul>
  48. 48. Quality Assurance <ul><li>Portability: Will it work with other hardware? </li></ul><ul><li>Re-useability: Can it be used again? </li></ul><ul><li>Integrity: Is it secure? </li></ul><ul><li>It is used to assure customers of the quality of the product. </li></ul><ul><li>Employs a set of measurable standards set out by the ISO (International Standards Organisation) </li></ul>
  49. 49. Question 8 <ul><li>A software application has been written to control the </li></ul><ul><li>operation of a digital video camera. The camera performs </li></ul><ul><li>all the usual video camera functions and some new ones. </li></ul><ul><li>For example, the camera can be set to commence </li></ul><ul><li>filming once it detects movement. This is useful for wildlife </li></ul><ul><li>photographers. </li></ul><ul><li>Describe the types of tests that could be conducted by </li></ul><ul><li>The quality assurance company to determine </li></ul><ul><li>reliability, efficiency and useability of this product. </li></ul>
  50. 50. Question 8 - Answers <ul><li>Reliability – Beta test using live test data to test how the camera performs under large volumes of data. i.e multiple animals or long sequences. </li></ul><ul><li>Efficiency – Benchmarking could take place by giving a competitors product to compare. </li></ul><ul><li>Useability – Beta testing and surveying users regrading useability of the camera and its functions. </li></ul>