4. • One of the practical methods commonly used to detect the presence
of errors (failures) in a computer program is to test it for a set of
inputs.
Our program
The output
is correct?I1, I2, I3,
…, In, …
Expected results
= ?
Obtained results
“Inputs”
Testing………….?Testing………….?
5. Developer Independent Tester
Understands the
system,
tests "gently”,
Test is driven by
“delivery”.
learns about the system,
attempts to break the
system
Test is driven by
“quality”.
Who Tests the Software….?Who Tests the Software….?
7. 7
• An early start to testing reduces the cost, time to rework and
error free software that is delivered to the client.
• However in Software Development Life Cycle (SDLC) testing can be
started from the Requirements Gathering phase and lasts till the
deployment of the software.
• However it also depends on the development model that is being
used.
• For example in Water fall model formal testing is conducted in the
Testing phase,
• But in incremental model, testing is performed at the end of every
increment/iteration and at the end the whole application is tested.
When to start Testing …?When to start Testing …?
8. 8
• Unlike when to start testing it is difficult to determine when to
stop testing, as testing is a never ending process and no one can
say that any software is 100% tested.
• Following are the aspects which should be considered to stop
the testing:
• Testing Deadlines.
• Management decision
• Completion of test case execution.
• Completion of Functional and code coverage to a certain point.
• Bug rate falls below a certain level
When to stop Testing …?When to stop Testing …?
24. 24
• A unit is the smallest testable part of software.
• In procedural programming a unit may be an individual
program, function, procedure, etc. In OOP, the smallest unit is a
method.
– Unit testing is often neglected but it is, in fact, the most
important level of testing.
Unit Testing
25. 25
METHOD
• Unit Testing is performed by using
the method White Box Testing
When is it performed?
• First level of testing & performed
prior to Integration Testing
Who performs it?
• performed by software developers
themselves or their peers.
Unit Testing
26. 26
Integration Testing is a level of the software testing process
where individual units are combined and tested as a group.
Integration testing tests interface between components,
interaction to different parts of system.
Integration Testing
27. 27
Big Bang is an approach to
Integration Testing where all or most
of the units are combined together
and tested at one go.
This approach is taken when the
testing team receives the entire
software in a bundle.
Top Down is an approach to
Integration Testing where top level
units are tested first and lower level
units are tested step by step after
that.
This approach is taken when top
down development approach is
followed.
Integration Testing Approaches
28. 28
Bottom Up is an approach to
Integration Testing where
bottom level units are tested
first and upper level units
step by step after that. This
approach is taken when
bottom up development
approach is followed.
Sandwich/Hybrid is an approach
to Integration Testing which
is a combination of Top
Down and Bottom Up
approaches.
Integration Testing
Approaches
30. 30
Functional Testing:
Goal: Test functionality of system
Test cases are designed from the requirements analysis
document (better: user manual) and centered around
requirements and key functions (use cases).The system is
treated as black box
Unit test cases can be reused, but new test cases have to be
developed as well.
Performance Testing:
Goal: Try to violate non-functional requirements
Test how the system behaves when overloaded.
Try unusual orders of execution
Check the system’s response to large volumes of data
What is the amount of time spent in different use cases?
System Testing
31. Types of Performance Testing
• Stress Testing
– Stress limits of system
• Volume testing
– Test what happens if large
amounts of data are handled
• Configuration testing
– Test the various software and
hardware configurations
• Compatibility test
– Test backward compatibility
with existing systems
• Timing testing
– Evaluate response times and
time to perform a function
• Security testing
– Try to violate security
requirements
• Environmental test
– Test tolerances for heat,
humidity, motion
• Quality testing
– Test reliability, maintain-
ability & availability
• Recovery testing
– Test system’s response to
presence of errors or loss
of data
• Human factors testing
– Test with end users.
32. Acceptance Testing
Goal: Demonstrate
system is ready for
operational use
Choice of tests is
made by client
Many tests can be
taken from integration
testing
Acceptance test is
performed by the
client, not by the
developer.
Alpha test:
Client uses the software at
the developer’s environment.
Software used in a controlled
setting, with the developer
always ready to fix bugs.
Beta test:
Conducted at client’s
environment (developer is not
present)
Software gets a realistic
workout in target environment
36. 36
Errors
An error is a mistake, misconception, or misunderstanding on the part
of a software developer.
Faults /Defects /Bugs
• A fault (defect) is introduced into the software as the result of an
error.
• It is an anomaly in the software that may cause it to behave
incorrectly, and not according to its specification.
Failures
• A failure is the inability of a software system or component to
perform its required functions within specified performance
requirements
A fault in the code does not always produce a failure.
Faults Failures & ErrorsFaults Failures & Errors
37. 37
LOC Code
1 program double ();
2 var x,y: integer;
3 begin
4 read(x);
5 y := x * x;
6 write(y)
7 end
Fault: The fault that causes the failure is in
line 5. The * operator is used instead of +.
Error: The error that conduces to this fault
may be:
• a typing error (the developer has
written * instead of +)
• a conceptual error (e.g., the developer
doesn't know how to double a number)
Failure: x = 3 means y =9 Failure!
• This is a failure of the system since the
correct output would be 6
Fault - Failure - Error - illustrationsFault - Failure - Error - illustrations
48. 48
1. ‘‘If an input condition for the software-under-test is specified as a
range of values, select one valid equivalence class that covers the
allowed range and two invalid equivalence classes, one outside each
end of the range.’’
For example, suppose the specification for a module says that an
input, the length of a Wire in millimetres, lies in the range 1–499;
then select
One valid equivalence class that includes all values from 1 to
499.
Two invalid classes
That consists of all values less than 1, and
That consists of all values greater than 499.
Equivalence Partitioning
49. 49
2. ‘‘If an input condition for the software-under-test is specified as a
number of values, then select one valid equivalence class that includes
the allowed number of values and two invalid equivalence classes that
are outside each end of the allowed number.’’
For example, if the specification for a real estate-related module says that
a house can have one to four owners, then we select
– One valid equivalence class that includes all the valid
number of owners
– Two invalid equivalence classes
• Less than one owner and
• More than four owners.
Equivalence Partitioning
50. 50
3. ‘‘If an input condition for the software-under-test is specified as a set
of valid input values, then select one valid equivalence class that
contains all the members of the set and one invalid equivalence class
for any value outside the set.’’
For example, if the specification for a paint module states that the
colours RED, BLUE, GREEN and YELLOW are allowed as inputs, then
select
– One valid equivalence class that includes the set RED,
BLUE, GREEN and YELLOW, and
– One invalid equivalence class for all other inputs.
Equivalence Partitioning
51. 51
4. ‘‘If an input condition for the software-under-test is specified as a “must
be” condition, select one valid equivalence class to represent the “must
be” condition and one invalid class that does not include the “must be”
condition.’’
For example, if the specification for a module states that the first
character of a part identifier must be a letter, then select
• One valid equivalence class where the first character is a letter
and
• One invalid class where the first character is not a letter.
Equivalence Partitioning
52. 52
Write Test Cases using Equivalence Partitioning for a requirement that is
stated as follows:
“In the examination grading system, if the student scores 0 to less than 40
then assign E Grade, if the student scores between 40 to 49 then assign
D Grade, if the student scores between 50 to 69 then assign C Grade, if
the student scores between 70 to 84 then assign B Grade, and if the
student scores 85 to 100 then assign A Grade.”
In the above problem definition, after analysis, we identify set of Output
Values and corresponding set of input values producing same output.
This analysis results in:
Values from 0 to 39 produce E Grade
Values from 40 to 49 produce D Grade
Values from 50 to 69 produce C Grade
Values from 70 to 84 produce B Grade
Values from 85 to 100 produce A Grade
Equivalence Partitioning - An Illustration
53. 53
Based on EP, we identify following input values to test each boundary:
For EP in range 0 to 39:
EP for producing E Grade, input values are 0 to 39,
Here Minimum Value= 0, Maximum Value = 39, Precision is 1.
Thus, input values for testing for this EP for Grade E are:
Minimum Value= 0
(Minimum Value+ precision)= 1
(Maximum Value- precision)= 38
Maximum Value= 39
Thus, input values for Boundary Values 0 and 39 are:
0, 1, 38, 39
Output Value is: Grade E
Equivalence Partitioning - An Illustration
54. 54
On the similar lines of analysis, we arrive at following input values for
other
EPs and corresponding outputs as:
For 40 to 49, we have 40, 41, 48, 49 and Output Value is “Grade D”
For 50 to 69, we have 50, 51, 68, 69 and Output Value is “Grade C”
For 70 to 84, we have 70, 71, 83, 84 and Output Value is “Grade B”
For 85 to 100, we have 85, 86, 99, 100 and Output Value is “Grade A”
In Addition to this, we have EP with input values -1 and 101 and
corresponding output value is “Error”.
Thus for -1 and 101, we have Output Value as “Error”
Equivalence Partitioning - An Illustration
55. 55
For these boundaries, test cases based on EP technique are documented
in Table. Test Case Design for a given example using EP Technique
Equivalence Partitioning - An Illustration
56. 56
Boundary Value Analysis (BVA) technique is a test case design technique
that allows to identify appropriate number of input values to be
supplied to test the system.
Human brain is evolved in emotional side and less evolved in logical side
Hence during software development, people always get confused in
using < and <=; > and >= relational operators.
As such large number of errors tends to occur at boundaries of the input
domain.
BVA leads to selection of test cases that exercise boundary values.
BVA based test case design helps to write test cases that exercise
bounding values.
Boundary Value Analysis
57. 57
“Bugs lurk in corners and
congregate at boundaries…”
---Boris Beizer
The test cases developed based on equivalence class partitioning can be
strengthened by use of a technique called boundary value analysis.
Many defects occur directly on, and above and below, the edges of
equivalence classes.
BVA is a software design technique to determine test cases off-by-one
errors
Boundary Value Analysis
58. 58
1. If an input condition for the software-under-test is specified as a
range of values, develop valid test cases for the ends of the range,
and invalid test cases for possibilities just above and below the
ends of the range.
For example if a specification states that an input value for a module
must lie in the range between -1.0 and +1.0,
Valid tests that include values -1.0, +1.0 for ends of the range, as well as
Invalid test cases for values just above and below the ends -1.1, +1.1,
should be included.
Boundary Value Analysis
59. 59
2. If an input condition for the software-under-test is specified as a
number of values, develop valid test cases for the minimum and
maximum numbers as well as invalid test cases that include one lesser
than minimum and one greater than the maximum
For example, for the real-estate module mentioned previously that
specified a house can have four owners,
Valid Tests that includes tests for minimum and maximum values 1, 5
owners
Invalid tests that include one lesser than minimum and one greater
than maximum 0, 6 are developed.
Boundary Value Analysis
60. 60
If the input or output of the software-under-test is an ordered set, such
as a table or a linear list, develop tests that focus on the first and last
elements of the set
Example: Loan application
Boundary Value Analysis
61. 61
Using BVA- Range
When the programming element is a range type, we can arrive at test cases
using BVA as follows:
For a range of values bounded by a and b, then test:
(Minimum Value - precision)
Minimum Value
(Minimum Value + precision),
(Maximum Value - precision)
Maximum Value
(Maximum Value + precision)
Boundary Value Analysis
62. 62
Write Test Cases using BVA for a requirement that is stated as follows:
“In the examination grading system, if the student scores 0 to less than 40 then
assign E Grade, if the student scores between 40 to 49 then assign D Grade, if the
student scores between 50 to 69 then assign C Grade, if the student scores
between 70 to 84 then assign B Grade, and if the student scores 85 to 100 then
assign A Grade.”
In the above problem definition, after analysis, we identify following Boundary
Values:
0 to 39
40 to 49
50 to 69
70 to 84
85 to 100
BVA - An Illustration
63. 63
Based on BVA, we identify following input values to test each boundary: For
Boundary Values in range 0 to 39:
For 0 to 39 boundary values, Minimum Value= 0, Maximum Value = 39,
Precision is 1.
Thus, input values for testing for this boundary values are:
(Minimum Value-precision)= -1 Minimum Value= 0
(Minimum Value+ precision)= 1 (Maximum Value- precision)= 38
Maximum Value= 39 (Maximum Value + precision)= 40
Thus, input values for Boundary Values 0 and 39 are:
-1, 0, 1, 38, 39, 40.
On the similar lines of analysis, we arrive at following input values for other
Boundary values:
For 40 to 49, we have 39, 40, 41, 48, 49, 50
For 50 to 69, we have 49, 50, 51, 58, 59, 60
For 70 to 84, we have 69, 70, 71, 83, 84, 85
For 85 to 100, we have 84, 85, 86, 99, 100, 101
BVA - An Illustration