STA UNIT-2

1. UNIT 2
There are several similarities and differences between techniques used for testing software
and hardware
Software Application
a) Does not integrate over time
b) Fault present in the application will remain and no
c) new faults will creep in unless the application is changed
d) Built-in self test meant for hardware product, rarely, can be applied to software designs
and code
e) It only detects faults that were present when the last change was made
6/10/2017 1Abhimanyu Mishra(CSE)

2. There are several similarities and differences between techniques used for testing software
and hardware
Hardware Application
a) Does integrate over time
b) VLSI chip, that might fail over time due to a fault thatdid not exist at the time chip was
manufactured and tested
c) BIST intended to actually test for the correct functioning of a circuit
d) Hardware testers generate test based on fault-models
6/10/2017 2Abhimanyu Mishra(CSE)

3. 6/10/2017 3Abhimanyu Mishra(CSE)
Software testers generate tests to test for correct functionality.
Sometimes such tests do not correspond to any general fault model
For example: to test whether there is a memory leak in an application, one performs a combination of stress
testing and code inspection
A variety of faults could lead to memory leaks
Hardware testers use a variety of fault models at different levels of abstraction
Example:
transistor level faults low level
gate level, circuit level, function level faults higher level

4. 6/10/2017 4Abhimanyu Mishra(CSE)
Software testers might not or might use fault models during test generation even
though the model exist
Mutation testing is a technique based on software fault models
Test Domain a major difference between tests for hardware and software is in the domain of tests

5. 6/10/2017 5Abhimanyu Mishra(CSE)
TESTING AND VERIFICATION
Program verification aims at proving the correctness of progress by showing that is contains no errors.
This is very different from testing that aims at uncovering errors in a program.
While verification aims at showing that a given program works for all possible inputs that satisfy a set of
conditions, testing aims to show that the given program is reliable to that, no errors of any significance were found.
Program verification and testing are best considered as complimentary techniques.
In the developments of critical applications, such as smart cards or control of nuclear plants, one often makes use of
verification techniques to prove the correctness of some artifact created during the development cycle, not
necessarily the complete program.
Regardless of such proofs, testing is used invariably to obtain confidence in the correctness of the application.
Testing is not a perfect process in that a program might contain errors despite the success of a set of tests;
verification might appear to be a perfect process as it promises to verify that a program is free from errors.
Verification reveals that it has its own weakness.
The person who verified a program might have made mistakes in the verification process’ there might be an
incorrect assumption on the input conditions; incorrect assumptions might be made regarding the components that
interface with the program.
Thus, neither verification nor testing is a perfect technique for proving the correctness of program.

6. 6/10/2017 6Abhimanyu Mishra(CSE)
DEFECT MANAGEMENT
Defect Management is an integral part of a development and test process in many software
development organizations. It is a sub process of a the development process. It entails the
following:
Detect prevention
Discovery
Recording and reporting
Classification
Resolution
Production

7. 6/10/2017 7Abhimanyu Mishra(CSE)
Defect Prevention
It is achieved through a variety of process and tools: They are, Good coding techniques.
Unit test plans.
Code Inspections.
Defect Discovery
Defect discovery is the identification of defects in response to failures observed during
dynamic testing or found during static testing.
It involves debugging the code under test.
Defect Classification
Defects found are classified and recorded in a database.
Classification becomes important in dealing with the defects.
Classified into High severity-to be attended first by developer.
Low severity.

8. 6/10/2017 8Abhimanyu Mishra(CSE)
Resolution
Each defect, when recorded, is marked as ‘open’ indicating that it needs to be resolved. It
required careful scrutiny of the defects, identifying a fix if needed, implementing the fix,
testing the fix, and finally closing the defect indicating that every recorded defect is resolved
prior to release.
Defect Prediction
Organizations often do source code Analysis to predict how many defects an application might
contain before it enters the testing the phase.
Advanced statistical techniques are used to predict defects during the test process.
Tools are existing for Recording defects, and computing and reporting defect related statistics.
BugZilla - Open source
Fog-Buzz - commercially available tools.

9. 6/10/2017 9Abhimanyu Mishra(CSE)
EXECUTION HISTORY
Execution history of a program, also known as execution trace, is an organized collection of
information about various elements of a program during a given execution. An execution slice
is an executable subsequence of execution history. There are several ways to represent an
execution history, Sequence in which the functions in a given program are executed against a
given test input,Sequence in which program blocks are executed.
Sequence of objects and the corresponding methods accessed for object oriented languages
such as Java An execution history may also included values of program variables.
A complete execution history recorded from the start of a program’s execution until its
termination represents a single execution path through the program.
It is possible to get partial execution history also for some program elements or blocks or
values of variables are recorded along a portion of the complete path.

10. 6/10/2017 10Abhimanyu Mishra(CSE)
TEST GENERATION STRATEGIES

11. 6/10/2017 11Abhimanyu Mishra(CSE)
Test generation uses a source document. In the most informal of test methods, the source document
resides in the mind of the tester who generates tests based on knowledge of the requirements.
Fig summarizes the several strategies for test generation. These may be informal techniques that
assign value to input variables without the use of any rigorous or formal methods. These could also
be techniques that identify input variables, capture the relationship among these variables, and use
formal techniques for test generation such as random test generation and cause effect graphing.
Another set of strategies fall under the category of model based test generation. These strategies
require that a subset of the requirements be modeled using a formal notation.
FSMs, state charts, pettiness and timed I/O automata are some of the well known and used formal
notations for modeling various subset requirements.
Sequence & activity diagrams in UML also exist and are used as models of subsets of requirements.
There also exist techniques to generate tests directly from the code i.e. code based test generation.
It is useful when enhancing existing tests based on test adequacy criteria.
Code based test generation techniques are also used during regression testing when there is often a
need to reduce the size of the suite or prioritize tests, against which a regression test is to be
performed.

12. 6/10/2017 12Abhimanyu Mishra(CSE)
STATIC TESTING
Static testing is carried out without executing the application under test.
i) This is in contrast to dynamic testing that requires one or more executions of the application
under test.
ii) It is useful in that it may lead to the discovery of faults in the application, ambiguities and
errors in the requirements and other application-related document, at a relatively low cost, This
is especially so when dynamic testing expensive. Static testing is complementary to dynamic
testing.
iii) This is carried out by an individual who did not write the code or by a team of individuals.
iv) The test team responsible for static testing has access to requirements document,
application, and all associated documents such as design document and user manual. Team also
has access to one or more static testing tools.
v) A static testing tool takes the application code as input and generates a variety of data useful
in the test process.

13. 6/10/2017 13Abhimanyu Mishra(CSE)

14. 6/10/2017 14Abhimanyu Mishra(CSE)
WALKTHROUGHS
•Walkthroughs and inspections are an integral part of static testing.
•Walkthrough are an integral part of static testing.
•Walkthrough is an informal process to review any application-related document.
eg:
requiremnts are reviewed---->requirements walkthrough code is reviewed---->code
walkthrough
(or)
peer code review
Walkthrough begins with a review plan agreed upon by all members of the team.
Advantages:

15. 6/10/2017 15Abhimanyu Mishra(CSE)
both functional and non functional requirements are reviewed.
A detailed report is generated that lists items of concern regarding the requirements.
INSPECTIONS
Inspection is a more formally defined process than a walkthrough. This term is usually associated with
code.
Several organizations consider formal code inspections as a tool to improve code quality at a lower
cost than incurred when dynamic testing is used.
Inspection plan:
statement of purpose
work product to be inspected this includes code and associated documents needed for inspection. Team
formation, roles, and tasks to be performed.
rate at which the inspection task is to be completed
Data collection forms where the team will record its findings such as defects discovered, coding
standard violations and time spent in each task.
Members of inspection team
Moderator: in charge of the process and leads the review.
Leader: actual code is read by the reader, perhaps with help of a code browser and with monitors for
all in the team to view the code.
Recorder: records any errors discovered or issues to be looked into.
Author: actual developer of the code.

16. 6/10/2017 16Abhimanyu Mishra(CSE)

17. 6/10/2017 17Abhimanyu Mishra(CSE)
CFG clearly shows that the definition of x at block 1 is used at block-3 but not at block 5.In
fact the definition of x at block 1 is considered killed due to its redefinition at block 4.
CFG indicates the use of variable y in the block 3.If y is not defined along the path from
start to block 3,then there is a data-flow error as a variable is used before it is defined.
Several such errors can be detected by static analysis tools.
->compute complexity metrics, used as a parameter in deciding which modules to inspect
first.
.

18. 6/10/2017 18Abhimanyu Mishra(CSE)
Model-Based Testing and Model checking:
•Model based testing refers to the acts of modeling and the generation of tests from a formal model of
application behavior.
•Model checking refers to a class of techniques that allow the validation of one or more properties from a
given model of an application.

19. 6/10/2017 19Abhimanyu Mishra(CSE)

20. 6/10/2017 20Abhimanyu Mishra(CSE)
Above diagram illustrates the process of model-checking. A model, usually finite state is
extracted from some source. The source could be the requirements and in some cases, the
application code itself.
One or more desired properties are then coded to a formal specification language. Often, such
properties are coded in temporal logic, a language for formally specifying timing properties.
The model and the desired properties are then input to a model checker. The model checker
attempts to verify whether the given properties are satisfied by the given model.

21. 6/10/2017 21Abhimanyu Mishra(CSE)
CONTROL FLOW GRAPH
A CFG captures the flow of control within a program. Such a graph assists testers in the
analysis of a program to a understand its behaviour in terms of the flow of control. A CFG can
be constructed
manually without much difficulty for relatively small programs, say containing less than about
50 statements.
However, as the size of the program grows, so does the difficulty of constructing its CFG and
hence arises the need for tools.
A CFG is also known by the names flow graph or program and it is not to be confused with
program-dependence graph(PDG).

22. 6/10/2017 22Abhimanyu Mishra(CSE)
TYPES OF TESTING
Framework consists of a set of five classifies that serve to classify testing techniques that fall
under the dynamic testing category.Dynamic testing requires the excution of program under
test.Static testing consists of testing for the review and analysis of the program.
five classifiers of testing:-
o1.C1:source of test generation
o 2.C2:life cycle phase in which testing takes place
o 3.C3:goal of a specific testing activity.
o 4.C4:characteristics of the artifact under test
o 5.C5:test process

23. 6/10/2017 23Abhimanyu Mishra(CSE)
TYPES OF TESTING
Framework consists of a set of five classifies that serve to classify testing techniques that fall
under the dynamic testing category .Dynamic testing requires the execution of program under
test. Static testing consists of testing for the review and analysis of the program.
five classifiers of testing:-
o1.C1:source of test generation
o 2.C2:life cycle phase in which testing takes place
o 3.C3:goal of a specific testing activity.
o 4.C4:characteristics of the artifact under test
o 5.C5:test process
Classifier C1: Source of test generation
Black box Testing: Test generation is an essential part of testing. There are a variety of ways to
generate tests, listed in table. Tests could be generated from informally or formally specified
requirements and without the aid of the code that is under test. Such form of testing is
commonly referred to as black box testing.

24. 6/10/2017 24Abhimanyu Mishra(CSE)

25. 6/10/2017 25Abhimanyu Mishra(CSE)
Model based or specification based testing:
Model based or specification based testing occurs when the requirements are formally specified as for example,
using one or more mathematical or graphical notations such as, z, state charts, event sequence graphs
White box testing:
White box testing refers to the test activity where in code is used in the generation of or the assessment of the test
cases.
Code could be used directly or indirectly for test generation.
In the direct case, a tool, or a human tester examines the code and focuses on a given path to be covered. A test is
generated to cover path.
In the indirect case, test generated using some black box testing is assessed against some code based coverage
criterion.
Additional tests are then generated to cover the uncovered positions of the code by the analyzing which parts of
the code are feasible.
Control flow, data flow, and mutation testing can be used for direct as well as indirect code-based test generation.

26. 6/10/2017 26Abhimanyu Mishra(CSE)
Pair wise testing:
Set of values for each input is obtained from the components requirement.
Interface mutation:
The interface itself, such as function coded in /c or CORBA component written in an IDL,serves to extract the
information needed to perform interface mutation.
pair wise testing is a black box testing
interface mutation is a white box testing
Ad-hoc testing:
In ad-hoc testing, a tester generates tests from requirements but without the use of any systematic method.
Random testing:
Random testing uses a systematic method to generate tests. Generation of tests using random testing requires
modeling the input space randomly.
.

27. 6/10/2017 27Abhimanyu Mishra(CSE)
Classifier C2: Life cycle phase
Testing activities take place throughout the software life cycle.
Each artifact produced is often subject to testing at different levels of rigor and using different testing techniques.
Unit testing:
Programmers write code during the early coding phase.
They test their code before it is integrated with other system components.
This type of testing is referred to as the unit testing.
System testing:
When units are integrated and a large component or a subsystem formed, programmers do integration testing of
the sub system.
System testing is to ensure that all the desired functionality is in the system and works as per its requirements.
Note: test designed during unit testing are not likely to be used during integrating and system testing.
.

28. 6/10/2017 28Abhimanyu Mishra(CSE)
Acceptance testing:
-beta testing
Carefully selected set if customers are asked to test a system before commercialization.
This form of testing is referred to as beta testing.
In case of contract software, the customer who contracted the development performs acceptability testing prior to
making the final decisions as to whether to purchase the application for deployment.

29. 6/10/2017 29Abhimanyu Mishra(CSE)

30. 6/10/2017 30Abhimanyu Mishra(CSE)
Classifier C3: Goal-directed testing
There exists a variety of goals of course finding any hidden errors is the prime goal of testing, goal-oriented testing
books for specific type of failure.
Robustness testing:
•Robustness testing refers to the task of testing an application for robustness against unintended inputs. It differs from
functional testing in that the tests for robustness are derived from outside of the valid (or expected) input space,
whereas in the former the tests are derived from the valid input space.
Stress testing:
•In stress testing, one checks for the behavior of an application under stress. Handling of overflow of
data storage, for example buffers, can be checked with the help of stress testing.
Performance testing:
•The term performance testing refers to that phase of testing where an application tested specifically with
performance requirements in the view.
•Ex: An application might be required to process 1,000billing transactions per minute on a specific intel processer-
based machine and running a specific OS.
Load testing:
•The term load testing refers to that phase of testing in which an application is loaded with respect to one or more
applications. The goal is to determine if the application continues to perform as required under various load
conditions.

31. 6/10/2017 31Abhimanyu Mishra(CSE)
Classifier C4: Artifact under test
Table 1.7 is a partial list of testing techniques named after the artifact that is being tested. For ex, during the design
phase one might generate a design using SDL notation. This form of testing is known as design testing.

32. 6/10/2017 32Abhimanyu Mishra(CSE)
Classifier C5: Test process models
Software testing can be integrated into the software development life cycle in a variety of ways.
This leads to various models for the tests process listed in the table 1.8