Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Software testing


Published on

Published in: Technology, Education
  • Be the first to comment

Software testing

  1. 1. Software Testing Strategies based on Chapter 13 1
  2. 2. Software Testing Testing is the process of exercising a program with the specific intent of finding errors prior to delivery o the end user. errors requirements conformance performance an indication of quality 2
  3. 3. Who Tests the Software? developer Understands the system but, will test "gently" and, is driven by "delivery " 3 independent tester Must learn about the system, but, will attempt to break it and, is driven by quality
  4. 4. Software Testing Goals Types of tests Levels of tests Test measures Test plan
  5. 5. Goals Verification: Have we built the software right? Bug-free, meets specs Validation: Have we built the right software? Meets customers’ needs Are Verification and Validation different or variation of the same idea My argument: “meeting specs” should equal “meeting customer needs”... not generally true (my kitchen, satellite systems)
  6. 6. Software Testing Goals Types of tests Levels of tests Test measures Test plan most of this discussion focuses on verification (more specifically bug testing)
  7. 7. Types of tests Black box White box
  8. 8. Black box tests input output interfa ce 1. Does it perform the specified functions? 2.Does it handle obvious errors in input? 3.Ariane5 – lousy error handling 4.Classic ints vs floats, yards vs meters Black box should catch these if there is adequate “test coverage”
  9. 9. Example: ordered list of ints L=create() L.insert(5) L.insert(-1) L.insert(-1) p=L.getFirst() print (p) L.delete(p) p=L.getFirst() print(p) p=L.getNext(p) print(p) p=L.getNext(p) class OrdInts create getFirst getNext insert delete print -1 -1 5 error
  10. 10. Black box tests Advantage: black box tester≠developer is unbiased by implementation details. e.g., Use Case testing, just work through all the Use Cases Disadvantage: black box tester is uninformed about implementation details unnecessary tests – test same thing in different way insufficient tests – can miss the extremes, especially if actual use follows a different pattern
  11. 11. black box tests Code Input x x x x choose good distribution of input – hope good distribution of code tested
  12. 12. unnecessary tests Input Code large range of input may exercise a small part of code e.g., operator test of satellite control stations, run through each input and output light/key option. Testing same functions,
  13. 13. insufficient tests Input Code a small range of input may exercise a large range of code but can you ‘know’ this without knowing the code? Did we miss the 20%
  14. 14. sufficient tests Input Code complex code a small range of input may exercise a small but important/error-prone region
  15. 15. White box tests Based on code test 1 test 2
  16. 16. Example: ordered list of ints class ordInts { public: … private: int vals[1000]; int maxElements=1000; … }
  17. 17. Example: ordered list of ints bool testMax() { L=create(); num=maxElements; for (int i=0; i<=num; i++) print i L.insert(i) print maxElements; }
  18. 18. White box tests Advantage: design tests to achieve good code coverage and avoid duplication can stress complicated, error-prone code can stress boundary values (fault injection) Disadvantage: tester=developer may have bias if code changes, tests may have to be redesigned (is this bad?)
  19. 19. Example: ordered list of ints L=create() L.insert(1) L.insert(2) L.insert(3) class OrdInts create getFirst 1 2 3 insert delete print … L.insert(1001) p=L.getFirst() print(p) p=L.getNext(p) print p … … getNext
  20. 20. Types of tests Black box: test based on interface, through interface White box: test based on code, through code Testing strategy can/should include all approaches! My experience: Black Box = non developer, outside testing idiots in your case who? White Box = developer, part of development process in your case who? Gray Box = hated, non developer within developer organization in your case who?
  21. 21. Software Testing White-Box Testing Black-Box Testing requirements output . .. our goal is to ensure that all statements and conditions have een executed at least once ... 21 input events
  22. 22. White-Box Testing Basis Path Testingwe compute the cyclomatic First, complexity: number of simple decisions + 1 or number of enclosed areas + 1 In this case, V(G) = 4 22
  23. 23. White-Box Testing Basis Path Next, we derive the Testing independent paths: 1 Since V(G) = 4, there are four paths 2 3 4 5 7 8 6 Path 1: Path 2: Path 3: Path 4: 1,2,4,7,8 1,2,3,5,7,8 1,2,3,6,7,8 1,2,4,7,2,4,...7,8 Finally, we derive test cases to exercise these paths. A number of industry studies have indicated that the higher V(G), the higher the probability 23 or errors.
  24. 24. White-Box Testing Loop Testing Simple loop Nested Loops Concatenated Unstructured Loops Loops 24 Why is loop testing important?
  25. 25. White-Box Testing Equivalence Partitioning Black-Box & Boundary Testing Value Analysis   25 If x = 5 then … If x > -5 and x < 5 then … What would be the equivalence classes?
  26. 26. Black-Box Testing Comparison Testing Used only in situations in which the reliability of software is absolutely critical (e.g., human-rated systems) Separate software engineering teams develop independent versions of an application using the same specification  Each version can be tested with the same test data to ensure that all provide identical output Then all versions are executed in parallel with real-time comparison of results to ensure consistency 26
  27. 27. Levels of tests Unit white Integration System System integration black
  28. 28. Testing measures (white box) Code coverage – individual modules Path coverage – sequence diagrams Code coverage based on complexity – test of the risks, tricky part of code (e.g., Unix “you are not expected to understand this” code)
  29. 29. Levels of Testing  Unit testing  Integration testing  Validation testing  Focus is on software requirements  System testing  Focus is on system integration  Alpha/Beta testing  Focus is on customer usage  Recovery testing  forces the software to fail in a variety of ways and verifies that recovery is properly performed  Security testing  verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration  Stress testing  executes a system in a manner that demands resources in abnormal quantity, frequency, or volume  Performance Testing  test the run-time performance of software within the context of an integrated system 29
  30. 30. Unit Testing Tests the smallest individually executable code units. Usually done by programmers. Test cases might be selected based on code, specification, intuition, etc. Tools: Test driver/harness Code coverage analyzer Automatic test case generator
  31. 31. Integration Testing Tests interactions between two or more units or components. Usually done by programmers. Emphasizes interfaces. Issues: In what order are units combined? How do you assure the compatibility and correctness of externally-supplied components?
  32. 32. Integration Testing How are units integrated? What are the implications of this order? Top-down => need stubs; top-level tested repeatedly. Bottom-up => need drivers; bottom-levels tested repeatedly. Critical units first => stubs & drivers needed; critical units tested repeatedly.
  33. 33. Integration Testing Potential Problems: Inadequate unit testing. Inadequate planning & organization for integration testing. Inadequate documentation and testing of externally-supplied components.
  34. 34. Stages of Testing Testing in the Large  System Testing  End-to-End Testing  Operations Readiness Testing  Beta Testing  Load Testing  Stress Testing  Performance Testing  Reliability Testing  Regression Testing
  35. 35. System Testing Test the functionality of the entire system. Usually done by professional testers.
  36. 36. Realities of System Testing Not all problems will be found no matter how thorough or systematic the testing. Testing resources (staff, time, tools, labs) are limited. Specifications are frequently unclear/ambiguous and changing (and not necessarily complete and up-to-date). Systems are almost always too large to permit test cases to be selected based on code characteristics.