Dynamic analysis in Software Testing


Published on

Small description of dynamic analysis in software testing..

Published in: Education, Technology, Business
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Dynamic analysis in Software Testing

  1. 1.  The testing of SW is seen as the execution of the test object  The test object is provided with test data  At the beginning, the test basis is analysed to determine what must be tested  It must be determined how the individual requirements & the test cases relate to each other
  2. 2. Three types of systematic technique Static (non-execution) • examination of documentation, source code listings, etc. Functional (Black Box) • based on behaviour / functionality of software Structural (White Box) • based on structure of software
  3. 3. Black Box Testing Technique  Here the test object is seen as a black box  It is the specification-based testing techniques  Also known as input/output-driven testing techniques because they view the software as a black-box with inputs and outputs, but they have no knowledge of how the system or component is structured inside the box  In essence, the tester is concentrating on what the software does, not how it does it  Test cases are derived from the specification of the test object and behavior is observed from outside(PoO is outside the test object)  Not possible to control the operating sequence of the test object(except adequate input test data) as PoC is outside of test
  4. 4. Black Box Testing Technique  The Black-box testing attempts to find errors in : 1. To test the modules independently 2. To test the functional validity of the software so that incorrect or missing functions can be recognized 3. To look for interface errors 4. To test the system behaviour and check its performance 5. To test the maximum load or stress on the system 6.To test the software such that the user/customer accepts the system within defined acceptable limits
  5. 5. Black Box Testing Technique  Both functional and non-functional testing are considered  Functional testing is concerned with what the system does, its features or functions  Non-functional testing is concerned with examining how well the system does something, rather than what it does  Non-functional aspects (also known as quality characteristics or quality attributes) include performance, usability, portability, maintainability, etc  Techniques to test these non-functional aspects are less procedural and less formalized than those of other categories as the actual tests are more dependent on the type of system, what it does and the resources available for the tests
  6. 6. White Box Testing Technique  Structure-based testing techniques (which are also dynamic rather than static) use the internal structure of the software to derive test cases  Also called 'glass-box' techniques (implying we can see into the system) since they require knowledge of how the software is implemented, that is, how it works. For example, a structural technique may be concerned with exercising loops in the software  Here the source code is known and used for test design  While executing the test cases, the internal processing of the test object and the output is analyzed  PoC and/or PoO inside the test object  Test cases are designed to cover the program structure of the test object  This is called structural testing as structure of the test object(component hierarchy, flow control, data flow) are considered
  7. 7.  Using black box testing, the test object is seen as a black box  Test cases are derived from the specification of the test object  The behavior of the test object is watched from the outside (PoO – Point of Observation is outside the test object)  It is not possible to control the operating sequence of the test object other than choosing the adequate input test data (the PoC – Point of Control is situated outside of test object, too)  Test cases are designed by using the specification or the requirements of the test object.
  8. 8.  In white box testing, the source code is known and used for test design  While executing the test cases, the internal processing of the test object, as well as the output, is analyzed (the Point of Observation is inside of the test object)  Direct intervention in the process of the test object is possible, but should be used only in special situations, e.g. to execute negative testing when the component's interface is not capable of initiating the provoked failure (the Point of Control can be located inside the test object)  Test cases are designed to cover the program structure of the test object
  9. 9. White box testing is also called structural testing, because the test designer considers the structure (component hierarchy, flow control, data flow) of the test object  The black box testing techniques are known as functional or behavioral testing techniques, because of the observation of the input/output behavior .The functionality of the test object is the center of attention  White box testing can be applied at the lower levels of the testing, i.e., component and integration test  Black box testing is predominantly used for higher levels of testing even though it is reasonable in component tests  Any test design before the code is written (test-first programming, test-driven development) is essentially black box driven 
  10. 10.  Black box appropriate at all levels but dominates higher levels of testing  White box used predominately at lower levels to compliment black box Acceptance Acceptance System System Integration Integration Component Component
  11. 11. Black Box test design techniques Techniques defined : Equivalence partitioning Boundary value analysis State transition testing Cause-effect graphing Syntax testing Random testing
  12. 12. Black Box Testing Technique  This test technique is used when the inner structure & design of the test object is unknown or not considered  A test with all possible input data combinations would be a complete test, but this is unrealistic considering the enormous no. of combinations.  Test design must make a reasonable selection of all possible test cases
  13. 13. Equivalence Class Partitioning • Equivalence Class Partitioning :  The domain of possible input data for each input data element is divided into equivalence class  This is a group of data values where the tester assumes that the test object processes them in the same way  These classes should be tested for correct input and incorrect input as well  For every input data element that should be tested the domain of all possible correct inputs should be determined & then all values outside these domains are wrong inputs  Equivalence classes are refined if the test object’s specification tells that some elements of that class are processed differently ,they are assigned to a new sub-class
  14. 14. Equivalence Class Partitioning • Determination of Equivalence Class : This partitioning is done such that the behavior of the program is similar for every input data belonging to the same equivalence class The main idea behind defining the equivalence classes is that testing the code with any one value belonging to an equivalence class is as good as testing the software with any other value belonging to that equivalence class  Equivalence classes for a software can be designed by examining the input data and output data
  15. 15. Equivalence Class Partitioning • Determination of Equivalence Class :  The probability of failure detection is highly dependent upon the quality of the partitioning and which test cases are executed  For the input & output values, identify the restrictions & conditions in the specification  For every restriction or condition, if a continuous numerical domain is specified, then create one valid & two invalid classes  To complete the test cases, the tester must define the preconditions & the expected result for every test case
  16. 16. Equivalence Class Partitioning General guidelines for designing the equivalence If the input data values to a system can be specified by a range of values, then one valid and two invalid equivalence classes should be defined. If the input data assumes values from a set of discrete members of some domain, then one equivalence class for valid input values and another equivalence class for invalid input values should be defined.
  17. 17. Equivalence partitioning (EP) divide (partition) the inputs, outputs, etc. into areas which are the same (equivalent) assumption: if one value works, all will work one from each partition better than all from one invalid valid 0 1 100 invalid 101
  18. 18. Equivalence Class Partitioning Example#1:For a software that computes the square root of an input integer which can assume values in the range of 0 to 5000, how many equivalence classes are there ? Example#2: Design the black-box test suite for the following program. The program computes the intersection point of two straight lines and displays the result. It reads two integer pairs (m1, c1) and (m2, c2) defining the two straight lines of the form y=mx + c.
  19. 19. Equivalence Class Partitioning Rules for test case determination Restriction of the number of test cases The number of "valid" test cases is the product of the number of valid equivalence classes per parameter Because of this multiplicative combination, even a few parameters can generate hundreds of "valid test cases“ Since it is seldom possible to use that many test cases, more rules are necessary to reduce the number of "valid" test cases
  20. 20. Equivalence Class Partitioning Rules for test case restriction Combine the test cases and sort them by frequency of occurrence (typical usage profile) Prioritize the test cases in this order. That way only the "relevant" test cases (often appearing combinations) are tested Test cases including boundary values or boundary value combinations are preferred Ensure as minimum criteria that every representative of an equivalence class appears in at least one test case Representatives of invalid equivalence classes should not be combined with representatives of other invalid equivalence classes.
  21. 21. Equivalence Class Partitioning • Test Completion Criteria :  The Percentage of executed equivalence classes in comparison to the total no. of specified equivalence classes , i.e. EC-coverage=(no. of tested EC/total no. of EC)*100% • Evaluation :  It gives a complete test coverage where specified conditions & restrictions are not overlooked  Drawback: only single input or output conditions are considered while possible dependencies or interactions between conditions are ignored
  22. 22. Boundary Value Analysis This checks the “border” of the equivalence classes. On every boundary, the exact boundary value & both nearest adjacent values (inside & outside the equivalence class)are tested. Three test cases result from every boundary & if the upper boundary of one equivalence class equals the lower boundary of the adjacent class, then the respective test cases coincide as well When there is no real boundary value as it belongs to another class, it is sufficient to test the boundary with two values: one which is just inside & another just outside the class
  23. 23. Boundary Value Analysis A type of programming error frequently occurs at the boundaries of different equivalence classes of inputs The reason behind such errors might purely be due to psychological factors  Programmers often fail to see the special processing required by the input values that lie at the boundary of the different equivalence classes  For example, programmers may improperly use < instead of <=, or conversely <= for <. Boundary value analysis leads to selection of test cases at the boundaries of the different equivalence classes
  24. 24. Boundary Value Analysis We can also apply equivalence partitioning and boundary value analysis more than once to the same specification item. For example, if an internal telephone system for a company with 200 telephones has 3-digit extension numbers from 100 to 699, we can identify the following partitions and boundaries: • digits (characters 0 to 9) with the invalid partition containing non-digits • number of digits, 3 (so invalid boundary values of 2 digits and 4 digits) • range of extension numbers, 100 to 699 (so invalid boundary values of 099 and 700) • extensions that are in use and those that are not (two valid partitions, no boundaries) • the lowest and highest extension numbers that are in use could also be used as boundary values
  25. 25. Boundary Value Analysis One test case could test more than one of these partitions/boundaries For example, Extension 409 which is in use would test four valid partitions: digits, the number of digits, the valid range, and the 'in use' partition. It also tests the boundary values for digits, 0 and 9. How many test cases would we need to test all of these partitions and boundaries, both valid and invalid? We would need a non-digit, a 2-digit and 4-digit number, the values of 99, 100, 699 and 700, one extension that is not in use, and possibly the lowest and highest extensions in use. This is ten or eleven test cases - the exact number would depend on what we could combine in one test case.
  26. 26. Boundary Value Analysis Example: ] For a function that computes the square root of integer values in the range of 0 and 5000, the test cases must include the following values: {0, -1,5000,5001}.
  27. 27. Boundary Value Analysis Test Completion Criteria : Intended coverage of the boundary values(BV) can be predefined & calculated as after execution of the tests: BV-coverage = ( No. of tested BV)/(Total No. of BV)*100%
  28. 28. State Transition Testing In many cases the current input as well as the history of execution or events of inputs, influences the outputs & how the test object will behave. To show the dependence on history state diagrams are used The system or test object starts from an initial state & come to a different state. Events trigger transitions. It consists of states(nodes),transitions(links),inputs(link weights) & outputs(link weights). A state transition test should execute all specified functions of a certain state at least once. Precondition: stack is initialized; state is “empty” Input: Push(“Hello”) Expected result: Stack contains “hello” Post-Condition: State of the stack is “filled”
  29. 29. State Transition Testing The system or test object starts from an initial state and can then come into different states • Events trigger state transitions where an event normally is a function invocation • State transitions can involve actions • Besides the initial state, the other special state is the endstate • Finite state machines, state diagrams, or state transition tables model this behavior • A finite state machine as follows: "An abstract machine (e.g., program, logic circuit, car's transmission) for which the number of states and input symbols are both finite and fixed. A finite state machine consists of states (nodes), transitions (links), inputs (link weights), and outputs (link weights) •
  30. 30. State Transition Testing  'finite state machine’ simply means that the system can be in a (finite) number of different states, and the transitions from one state to another are determined by the rules of the 'machine‘  This is the model on which the system and the tests are based. Any system where a different output is got for the same input, depending on what has happened before, is a finite state system  A finite state system is often shown as a state diagram
  31. 31. State Transition Testing  A state transition model has four basic parts: • the states that the software may occupy (open/closed or funded/insufficient funds); • the transitions from one state to another (not all transitions are allowed); • the events that cause a transition (closing a file or withdrawing money); • the actions that result from a transition (an error message or being given your cash)
  32. 32. State Transition Testing
  33. 33. State Transition Testing  Deriving tests only from a state graph (also known as a state chart) is very good for seeing the valid transitions, but we may not easily see the negative tests, where we try to generate invalid transitions  To see the total number of combinations of states and transitions, both valid and invalid, a state table is useful  The state table lists all the states down one side of the table and all the events that cause transitions along the top (or vice versa)  Each cell then represents a state-event pair. The content of each cell indicates which state the system will move to, when the corresponding event occurs while in the associated state  This will include possible erroneous events - events that are not expected to happen in certain states. These are negative test conditions
  34. 34. State Transition Testing
  35. 35. State Transition Testing
  36. 36. State Transition Testing Transition tree for the stack
  37. 37. State Transition Testing  The transition tree is built from a transition diagram in the following way: 1. The initial or start state is the root tree 2. For every possible transition from the initial state to a following state in the state transition diagram, the transition tree receives a branch from its root to a node, representing this next state 3. The process for step 2 is repeated foe every leaf in the tree(every newly added node) until one of the following two end-conditions is fulfilled: A) The corresponding state is already included in the tree on the way from the root to the node. This end condition corresponds to one pass of a cycle in the transition diagram. B) The corresponding state is a final state,and therefore has no further transitions to be considered.
  38. 38. State Transition Testing  Test Completion Criteria : Criteria for test intensity & Completion can be defined as: 1. Every state has been reached at least once 2. Every transition has been executed at least once 3. Every transition violating the specification has been checked 4. Percentage can be defined using the proportion of actually executed test requirements to possible ones
  39. 39. Cause Effect Graphing It uses the dependencies for identification of the test cases. The logical relationships between the causes & their effects in a component or a system are displayed here. It must be possible to find the cause & effects from the specification Every cause is described as a condition that consists of input conditions or their combinations & they are connected with logical operators It uses the dependencies for identification of the test cases known as cause-effect graphing
  40. 40. Cause Effect Graphing The logical relationships between the causes and their effects in a component or a system are displayed in a so-called causeeffect graph  It must be possible to find the causes and effects from the specification  Every cause is described as a condition that consists of input conditions (or combinations of those). The conditions are connected with logical operators (e.g., AND, OR and NOT)  A condition, and therefore a cause, can be TRue or false  The effects are treated in the same way and noted in the graph
  41. 41. Decision Table The techniques of equivalence partitioning and boundary value analysis are often applied to specific situations or inputs If different combinations results in different actions being taken, this can be more difficult to show using equivalence partitioning and boundary value analysis, which tend to be more focused on the user interface The other two specification-based techniques, decision tables and state transition testing are more focused on business logic or business rules A decision table is a good way to deal with combinations of things (e.g. inputs). This technique is sometimes also referred to as a 'cause-effect' table. The reason for this is that there is an associated logic diagramming technique called 'cause-effect graphing' which was sometimes used to help derive the decision table
  42. 42. Decisiontransformed into a decision table from Table The graph is which the test cases can be taken. Criteria for test completion is to execute every column in the decision table by at least one test case. This verifies all sensible combinations & their corresponding effects. Steps to transform a graph into a table: 1. Choose an effect 2. Looking into the graph, find combinations of causes that have this effect and combinations that do not have this effect 3.one column into the table for every one of these causecombinations and the caused states of the remaining effects 4.Check if decision table entries occur several times and if yes, delete them
  43. 43. Decision Table at an automated teller Example, withdrawing money machine (ATM) shall illustrate how to prepare a causeeffect graph. In order to get money from the machine, the following conditions must be fulfilled[]: The bankcard is valid The PIN must be correctly entered The maximum number of PIN inputs is three There is money in the machine, and in the account The following actions are possible at the machine: Reject card Ask for another PIN input "Eat" the card Ask for an alternate dollar amount Pay the requested amount of money
  44. 44. Decision Table Cause-effect graph of the ATM
  45. 45. Decision Table Optimized decision table for the ATM
  46. 46. Decision Table Empty decision table : Loan repayment Schedule
  47. 47. Decision Table Decision table with input combinations
  48. 48. Decision Table Decision table with combinations and outcomes
  49. 49. Decision Table Decision table with additional outcomes
  50. 50. Decision Table :Example 2 • You are a new customer opening a credit card account, you will get a 15% discount on all your purchases today. If you are an existing customer and you hold a loyalty card, you get a 10% discount. If you have a coupon, you can get 20% off today (but it can't be used with the 'new customer' discount). Discount amounts are added, if applicable.
  51. 51. Decision Table :Example 2 Decision table for credit card example
  52. 52. Test Completion Criteria A minimum requirement is to execute every column in the decision table by at least one test case  This verifies all sensible combinations of conditions and their corresponding effects
  53. 53. Use Case Testing With the increasing use of object-oriented methods for software development, the Unified Modeling Language (UML) is used more frequently  There exist quite a number of (research) results and approaches to directly derive test cases from UML diagrams and to generate these tests more or less automatically In order to detect requirements, use cases or business cases are described  These are then compiled into use case diagrams  The diagrams serve the purpose of defining requirements on a relatively abstract level and describing typical user system interactions
  54. 54. Use Case Testing Use case diagram for ATM
  55. 55. Use Case Testing The individual use cases in this example are "Get money", "PIN query", and "Eat card" Relationships between use cases may be "include" and "extend" "Include" conditions are always involved, and "extend" connections can lead to extensions of a use case under certain conditions at a certain point (extension point) Thus, the "extend" conditions are not always executed as there are alternatives
  56. 56. Use Case Testing Use case diagrams mainly serve to show the external view of a system It shall explain the external view of the system from the viewpoint of the user or the relation to neighboring systems Such external connections are shown as lines to "actors“ Pre- and post conditions : For every use case, there exist certain preconditions that must be fulfilled to make it possible to execute the use case  A precondition for getting money at the ATM is, for example, that the bankcard is valid  After executing a use case there exist post-conditions Example, after successfully entering the correct PIN, it is possible to get money But first the amount must be entered and it must be confirmed that the money is available  Preconditions and post-conditions are also applicable for the flow of use cases in a diagram, i.e., the path through the diagram
  57. 57. Use Case Testing Useful for system and acceptance testing : Use cases and use case diagrams serve as the basis for determining test cases in use case based testing  As the external view is modeled, the technique is useful for both system testing and acceptance testing If the diagrams are used to model the interactions between different subsystems, test cases can also be derived for integration testing
  58. 58. Use Case Testing Typical system use is tested : The diagrams show the "normal", "typical", or "probable" flows and often their alternatives Thus, the use case based test checks typical use of a system  It is especially important for acceptance of a system that it runs relatively stable in "normal" use  Use case based testing has a high relevance for the customer, and therefore for the developer and tester as well
  59. 59. Use Case Testing Every use case has a purpose and shall achieve a certain result. Events may occur that lead to further alternatives or activities. After the execution, there are post-conditions The information necessary for determining the test cases are: Start situation and preconditions Other possible conditions Expected results Post-conditions
  60. 60. Use Case Testing : test completion criteria A possible criterion is that every use case and every possible sequence of use cases in the diagram is tested at least once by a test case As alternatives and extensions are use cases too, this criterion also requires their execution
  61. 61. Black Box Techniques Syntax test It describes a technique for identification of the test cases that may be applied if a formal specification of the syntax of the inputs is available This may be the case for testing interpreters of command languages, compilers, and protocol analyzers  The rules of the syntax definition are used to specify test cases that cover both the compliance to and violation of the syntax rules for the inputs
  62. 62. Black Box Techniques Random test Random testing generates values for the test cases by random selection  If a statistical distribution of the input values is given (e.g., normal distribution), then it can be used for the selection of test values This ensures the derivation of test cases that are preferably close to reality,making it possible to use statistical models for predicting or certifying system reliability
  63. 63. Black Box Techniques Smoke test A smoke test is commonly understood as a "quick and dirty“ test that is primarily aimed at verifying a minimum reliability of the test object The test is concentrated on the main functions of the test object The output of the test is not evaluated in detail. The main outcome of interest is a crash or serious misbehavior of the test object A test oracle is not used, which contributes to making this test inexpensive and easy The term "smoke test" is derived from testing electrical circuits in which short circuits lead to smoke rising  A smoke test is often used to decide if the test object is mature enough to proceed with further testing by the more comprehensive test techniques  Another use of smoke tests is the first test and the fast test of software updates
  64. 64. White Box Testing The basis for this is the source-code of the test object.So,the technique is called code-based testing techniques or structural testing techniques. The generic idea is to execute every part of code of the test object at least once. But expected results should be determined using the requirements or specifications, not the code & this is done to check if execution results in a failure. Types are: 1. Statement Coverage 2. Branch Coverage 3. Test of conditions (branch condition & condition determination testing) 4. Path coverage
  65. 65. White Box Testing Coverage-based : It attempts to execute (or cover) certain elements of a program Examples are Statement Coverage , Branch Coverage etc. Fault-based : It targets to detect certain types of faults These faults that a test strategy focuses on constitutes the fault model of the strategy Example is mutation testing
  66. 66. White Box Testing White box testing assumes that the tester can take a look at the code for the application block and create test cases that look for any potential failure scenarios. During white box testing, you analyze the code of the application block and prepare test cases for testing the functionality to ensure that the class is behaving in accordance with the specifications and testing for robustness. Input The following input is required for white box testing: Requirements Functional specifications High-level design documents Detailed design documents Application block source code
  67. 67. White Box Testing Process Create test plans. Identify all white box test scenarios and prioritize them. Profile the application block. This step involves studying the code at run time to understand the resource utilization, time spent by various methods and operations, areas in code that are not accessed, and so on. Test the internal subroutines. This step ensures that the subroutines or the nonpublic interfaces can handle all types of data appropriately. Test loops and conditional statements. This step focuses on testing the loops and conditional statements for accuracy and efficiency for different data inputs. Perform security testing. White box security testing helps you understand possible security loopholes by looking at the way the code handles security.
  68. 68. White Box Testing Process One white-box testing strategy is said to be stronger than another strategy, if all types of errors detected by the first testing strategy is also detected by the second testing strategy, and the second testing strategy additionally detects some more types of errors When two testing strategies detect errors that are different at least with respect to some types of errors, then they are called complementary
  69. 69. White Box Testing Process Stronger and complementary testing strategies
  70. 70. Statement Coverage The statement coverage strategy aims to design test cases so that every statement in a program is executed at least once The principal idea governing the statement coverage strategy is that unless a statement is executed, it is very hard to determine if an error exists in that statement  Unless a statement is executed, it is very difficult to observe whether it causes failure due to some illegal memory access, wrong result computation, etc But executing some statement once and observing that it behaves properly for that input value is no guarantee that it will behave correctly for all input values
  71. 71. Statement Coverage This analysis focuses on each statement of the test object The test cases shall execute a predefined minimum quota or even all statements of the test object The first step is to translate the source code into a control flow graph which makes it easier to specify in detail the control elements that must be covered In graph, the statements are represented as nodes & the control flow between the statements as edges If sequence of unconditional statements appear in the program, they are treated as single node, as execution of the first statement of the sequence ensures execution all of the rest
  72. 72. The edges of the graph must be traversed in the following order: a, b, f, g, h, d, e
  73. 73. Statement Coverage The Value of the Technique  Unreachable code can be detected  If complete coverage of all statements is required, and some statements cannot be executed by any test cases, then this may be an indication for unreachable source code (dead statements).  Empty ELSE-parts are not considered  If a condition statement (IF) has statements only after it is fulfilled, i.e., after the THEN clause, and there is no ELSE, then the control flow graph has a (THEN-) edge, starting at the condition, with (at least) one node and a second outgoing (ELSE-) edge without nodes  The control flow of both these edges is reunited at the terminating (ENDIF-) node  For statement coverage an empty (ELSE-) edge (between IF and ENDIF) is irrelevant, such as when there is no requirement to cover it during the test  Possible missing statements after ELSE are not detected by a test using this criterion
  74. 74. Statement Coverage : Example int compute_gcd(x, y) int x, y; { while (x! = y){ if (x>y) then x= x – y; else y= y – x; } return x; } By choosing the test set {(x=3, y=3), (x=4, y=3), (x=3, y=4)}, we can exercise the program such that all statements are executed at least once
  75. 75. Statement Coverage Different combinations of edges of the graph can achieve complete coverage The cost of testing should always be minimized, which means reaching the goal with the least possible number of test cases The expected results and the expected behavior of the test object should be identified in advance from the specification  After execution, the expected and actual results, and the behavior of the test object, must be compared to detect any difference or failure Test Completion Criteria : It can be defined as Statement Coverage=(No. of executed statements/total No. of statements)*100% If complete coverage of all statements is required & some statements can’t be executed by any test cases, that may be unreachable source code(dead statements)
  76. 76. Branch Coverage Here the attention is on the edges of the graph The execution of each statement is not considered, rather the execution of decisions. The result of decision determines which statement is executed next. Testing should make sure every decision is executed with both possible outcome(TRUE & FALSE).That’s decision coverage is another name for it In the branch coverage-based testing strategy, test cases are designed to make each branch condition to assume true and false values in turn  Branch testing is also known as edge testing as in this testing scheme, each edge of a program’s control flow graph is traversed at least once
  77. 77. Branch Coverage Additional test cases necessary In the previous example additional test cases are necessary if all branches of the control flow graph must be executed during the test  For 100% statement coverage a test case executing the following order of edges was sufficient: a, b, f, g, h, d, e The edges c, i, and k have not been executed in this test case The edges c and k are empty branches of a condition, while the edge i is the return to the beginning of the loop  Additionally, three further test cases are necessary: a, b, c, d, e a, b, f, g, i, g, h, d, e a, k, e
  78. 78. Branch Coverage  Connection (edge) coverage of the control flow graph  Together, all four test cases result in a complete coverage of the edges of the control flow graph  With that, all possible branches of the control flow in the source code of the test object have been tested  Some edges have been executed more than once. This seems to be redundant, however, it cannot always be avoided  In the example, the edges a and e are executed in every test case because there is no alternative to these edges  For each test case, other than the precondition and post-condition, the expected result and expected behavior must be determined and then compared to the actual result and behavior  It is reasonable to record which of the branches have been executed in which test case  This helps to find faults, especially missing code in empty branches
  79. 79. Branch Coverage It is obvious that branch testing guarantees statement coverage and thus is a stronger testing strategy compared to the statement coverage-based testing In the previous example Euclid’s GCD computation algorithm , the test cases for branch coverage can be {(x=3, y=3), (x=3, y=2), (x=4, y=3), (x=3, y=4)}.
  80. 80. Branch Coverage Test Completion Criteria : Branch Coverage = (No. of executed branches/Total No. of branches)*100 In contrast to statement coverage it detects missing statements in empty branches A branch coverage of 100% should be aimed for The test can be categorized as sufficient only if, in addition to all statements, every possible branch of the control flow is considered during test execution
  81. 81. Test of Conditions If a decision is based on several(part) conditions connected by logical operators, then the complexity of the condition should be considered in the test & there comes the Branch Condition Testing An atomic part of a condition is a condition that has no logical operators such as AND, OR, and NOT, but at the most, includes relation symbols such as ">" or "=". A condition in the source code of the test object can consist of multiple atomic partial conditions Branch condition testing The goal is that each atomic(partial) condition in the test shall adopt the values TRUE or FALSE. This is a weaker criterion than statement or branch coverage because it is not required that different logical values for the result of the complete condition are included in the test
  82. 82. Test of Conditions It meets the criteria of both statement & branch coverage This is very comprehensive as it takes into account the complexity of composed conditions but is expensive due to growing no. of possible combinations An example for a composed condition is: x > 3 OR y < 5. The condition consists of two atomic partial conditions (x>3; y<5) connected by the logical operator OR  The goal of the branch condition testing is that every atomic part of conditions is evaluated once for each of the logical values Both parts of the condition have each resulted in both logical values. The result of the complete condition , however, is equal for both combinations Branch condition testing is therefore a weaker criterion than statement or branch coverage because it is not required that different logical values for the result of the complete condition are included in the test
  83. 83. Test of Conditions Branch Condition Combination Testing Branch condition combination testing, also called multiplecondition coverage requires that all True-False combinations of the atomic partial conditions be exercised at least once  All variations should be built, if possible Branch condition combination testing includes statement and branch coverage The complete condition gives both logical values as results Thus, branch condition combination testing meets the criteria of statement, as well as branch coverage
  84. 84. Branch Condition Combination Testing It is a more comprehensive criterion that also takes into account the complexity of composed conditions  But it is a very expensive technique due to the growing number of atomic conditions which make the number of possible combinations grow exponentially (to 2n with n atomic parts of conditions) A problem results from the fact that all combinations cannot always be implemented by test data For the combined condition of 3<=x AND x<5 not all combinations with the according values for the variable x can be produced because the parts of conditions depend on each other
  85. 85. Condition Determination Testing Here all combinations are not considered & consideration is given to every possible combination of logical values where the modification of the logical value of an atomic condition can change the logical value of the whole For every logical combination of the conditions, it must be decided which test cases are sensitive to faults & for which combinations faults can be masked & those are not considered Stated in another way, for a test case, every atomic condition has a meaningful impact on the result Test cases in which the result does not depend on a change of an atomic condition need not be designed So less no. of test cases need execution. For n no. of Boolean operands of the conditions amount lies between n+1 to 2n
  86. 86. Condition Determination Testing Combination of logical expressions are specially error-prone & so, a comprehensive test is very important. Combined complex conditions can be split into a tree-structure and then execute branch coverage to avoid this expensive technique This intensive testing can be avoided if they have been subjected to review in which the correctness is verified The example with the two atomic condition parts (x>3, y<5). Four combinations are possible (2*2): 1) x=6 (T), y=3 (T), x>3 OR y<5 (T) 2) x=6 (T), y=8 (F), x>3 OR y<5 (T) 3) x=2 (F), y=3 (T), x>3 OR y<5 (T) 4) x=2 (F), y=8 (F), x>3 OR y<5 (F)
  87. 87. Condition Determination Testing For the first combination the following applies: If the logical value is calculated wrong for the first condition part (i.e., an incorrect condition is implemented) then the fault can change the logical value of the first condition part from true (T) to false (F) But the result of the complete condition stays unchanged (T) The same applies for the second condition part For the first combination incorrect results of each condition part are masked because they have no effect on the result of the complete condition and thus failures will not become visible Consequently the test with the first combination can be left out If the logical value of the first condition part in the second test case is calculated wrongly as false, then the result value of the total condition changes from true (T) to false (F)  A failure then becomes visible because the value of the complete condition has also changed
  88. 88. Condition Determination Testing Because of the weak significance, branch condition testing should be abandoned for complex conditions For complex conditions, condition determination testing should be applied for test case design, because the complexity of the conditional expression is taken into account The method also leads to statement and branch coverage, which means they need not be used additionally Test Completion Criteria: Analogous to the previous techniques, the proportion between the executed and all the required logical values of the condition (parts) can be calculated  This can serve as criteria for termination of the tests  For the techniques, which concentrate attention to the complexity of the conditions in the source code, it is reasonable to try to achieve a complete verification (100% coverage)  If there are no complex condition expressions, branch coverage can be seen as sufficient
  89. 89. Path Coverage Until now, test case determination focused on the statements or branches of the control flow, as well as the complexity of conditions  If the test object includes loops or repetitions, the previous deliberations are not sufficient for an adequate test Path coverage requires the execution of all different paths through the test object If the test object includes loops or repetitions ,this method only requires the execution of all different paths through the test object. Both statement & branch coverage focuses on statement & branch of control flow & complexity of the conditions A path describes the possible order of single program parts in a program fragment. Paths consider dependencies between branches but branch coverage are independent type.
  90. 90. Path Coverage
  91. 91. Path Coverage The program fragment represented by the graph includes a loop This DO-WHILE loop is executed at least once In the WHILE-condition it is decided at the end of the loop whether the loop must be repeated, i.e., if a jump to the start of the loop is necessary When using branch coverage for test design the loop has been considered in two test cases: Loop without repetition: a, b, f, g, h, d, e Loop with single return (i) and a single repetition: a, b, f, g, i, g, h, d, e
  92. 92. Path Coverage Usually a loop is repeated more than once. Further possible sequences of branches through the graph of the program are a, b, f, g, i, g, i, g, h, d, e a, b, f, g, i, g, i, g, i, g, h, d, e a, b, f, g, i, g, i, g, i, g, i, g, h, d, e etc. This shows that there is an indefinite number of paths in the control flow graph Even with restrictions on the number of loop repetitions, the number of paths increases indefinitely
  93. 93. Path Coverage double calculate_price ( double baseprice, double specialprice, double extraprice, int extras, double discount) { double addon_discount; double result; if (extras >= 3) addon_discount = 10; else if (extras >= 5) addon_discount = 15; else addon_discount = 0; if (discount > addon_discount) addon_discount = discount; result = baseprice /100.0*(100-discount) extraprice/100.0*(100-addon_discount); return (result); } + specialprice +
  94. 94. Path Coverage  In this example the test object consists of altogether three IF-statements whereof two are nested and the third is placed separately from the others  All statements (nodes) are reached by the following sequence of edges in the graph: a, b, c, j, k, l, n a, d, e, f, i, j, k, l, n a, d, g, h, i, j, k, l, n  These sequences are sufficient to achieve 100% statement coverage.  But not all branches (edges) have been covered yet  The edge m is still missing. A sequence might look as follows: a, b, c, j, m, n  This additional sequence should have replaced the first sequence (test case) above and with the resulting three test cases, branch coverage of 100% is achieved
  95. 95. Path Coverage But, even for this simple program fragment, there are still possibilities to traverse the graph differently, and thus consider all paths of the graph Until now, the following paths have not been executed: a, d, e, f, i, j, m, n a, d, g, h, i, j, m, n Altogether, six different paths through the source code result (the three possible paths through the graph before edge j multiplied with the two possible paths after edge j) There is the precondition that the conditions are independent from each other and the edges can be combined freely
  96. 96. Intuitive testing Basis of this method is the skill, experience, and knowledge of the tester The tester selects test cases to uncover expected problems and their symptoms  A more systematic approach for this cannot be described  The test cases are based on the experience of where faults have occurred in the past or the tester's assumptions where faults might occur in the future This type of test case design is also called "error guessing" and is used very often in practice Knowledge in developing similar applications and using similar technologies should also be used when designing test cases, in addition to experience in testing  If, for example, there exist experiences with a new programming language in previous projects, it is reasonable to apply the failures found, as well as their cause in using the programming language for
  97. 97. Exploratory testing If the documents, which form the basis for test design, are of very low quality or do not exist at all , so-called "exploratory testing" may help In the extreme case only the program exists  The technique is also applicable when time is severely restricted because it uses much less time than other techniques The approach is mainly based on the intuition and experience of the tester Here minimum planning and maximum test execution is done The planning involves the creation of a test charter, a short declaration of the scope of a short (1 to 2 hour) time-bound test effort, the objectives and possible approaches to be used
  98. 98. Exploratory testing The test design and test execution activities are performed in parallel typically without formally documenting the test conditions, test cases or test scripts This does not mean that other, more formal testing techniques will not be used  For example, the tester may decide to use boundary value analysis but will think through and test the most important boundary values without necessarily writing them down Some notes will be written during the exploratory-testing session, so that a report can be produced afterwards
  99. 99. Exploratory testing Test logging is undertaken as test execution is performed, documenting the key aspects of what is tested, any defects found and any thoughts about possible further testing A key aspect of exploratory testing is learning: learning by the tester about the software, its use, its strengths and its weaknesses As its name implies, exploratory testing is about exploring, finding out about the software, what it does, what it doesn't do, what works and what doesn't work  The tester is constantly making decisions about what to test next and where to spend the (limited) time
  100. 100. Exploratory testing It makes sense to restrict exploratory testing to certain elements of the program (certain tasks or functions)  The elements are further broken down: The term "test charter" is used for such smaller parts  The test of a "charter" should not take more than one or two hours of uninterrupted test time