Unit 4- Testing.pptx

© Kalasalingam academy of research and education
222MCA3302 / Software
Engineering and Software Testing
UNIT IV
TESTING

© Kalasalingam academy of research and education SOFTWARE ENGINEERING
Unit 4 Testing
Lesson 1.
Lesson 2. Types of Testing
Lesson 3 Configuration Management
Lesson 1.
Testing Fundamentals
2

Course Progress
Lesson 1.
Lesson 3. Configuration Management
3

What is Software Testing?
Software testing is defined as an activity to check whether the actual results match the
expected results and to ensure that the software system is Defect free.
Software testing also helps to identify errors, gaps or missing requirements in contrary to the
actual requirements. It can be either done manually or using automated tools.
4
SOFTWARE ENGINEERING

Why Software Testing is Important?
264 people dead ,1994
5

Therac 25-Radiation Therapy
3 Dead and 3 Critically Injured, 1985
6

Failed Satellite Launch
$1.2 billion dollars lost ,1999
7

US bank accounts
In may of 1996, a software bug caused the bank accounts of 823 customers of a major U.S. bank
to be credited with 920 million US dollars.
8

Nissan
Nissan cars have to recall over 1 million cars from the market due to software failure in the
airbag sensory detectors. There has been reported two accident due to this software failure.
9

Note
Software bugs can be Expensive and even Dangerous
10

Definition of Testing
According to IEEE – “Testing means the process of analyzing a software item to detect the
differences between existing and required condition (i.e. bugs) and to evaluate the feature of
the software item”.
According to Myers – “Testing is the process of analyzing a program with the intent of finding an
error”.
11

Objectives:
According to Glen Myers the primary objectives of testing are:
- Testing is a process of executing a program with the intent of finding an error.
- A good test case is one that has a high probability of finding and as yet undiscovered
error.
- A successful test is one that uncovers as yet undiscovered error.
Testing cannot show the absence of errors and defects, it can show only error and detects
present. Hence, objective of testing is to design tests that systematically uncover different
errors and to do so with a minimum amount of time and effort
12

Who does Testing?
Software Tester
Software Developer
Project Lead/Manager
End User
13

When to Start Testing?
During the requirement gathering phase, the analysis and verification of requirements are also
considered as testing.
Reviewing the design in the design phase with the intent to improve the design is also
considered as testing.
Testing performed by a developer on completion of the code is also categorized as testing.
14

When to Stop Testing?
Testing Deadlines
Completion of test case execution
Completion of functional and code coverage to a certain point
Bug rate falls below a certain level and no high-priority bugs are identified
Management decision
15

Software Testing Life Cycle (STLC)
16
Each of these stages has
a definite Entry and Exit
criteria, Activities &
Deliverables associated
with it.
Requirement
Analysis
Test
Planning
Test Case
Development
Environment
Setup
Test
Execution
Test Cycle
Closure

What is Entry and Exit Criteria?
Entry Criteria: Entry Criteria gives the prerequisite items that must be completed before testing
can begin.
Exit Criteria: Exit Criteria defines the items that must be completed before testing can be
concluded
17

Requirement Analysis
Activities
-Identify types of tests to be performed.
-Gather details about testing priorities and focus.
-Prepare Requirement Traceability Matrix (RTM).
-Identify test environment details where testing is supposed to be carried out.
-Automation feasibility analysis (if required).
Deliverables
-RTM
-Automation feasibility report. (if applicable)
18

Test Planning
Activities
-Preparation of test plan/strategy document for various types of testing
-Test tool selection
-Test effort estimation
-Resource planning and determining roles and responsibilities.
-Training requirement
Deliverables
-Test plan /strategy document.
-Effort estimation document
19

Test Environment Setup
 can be done in parallel with Test Case Development Stage.
Activities
-Understand the required architecture, environment set-up and prepare hardware and
software requirement list for the Test Environment.
-Setup test Environment and test data
-Perform smoke test on the build
Deliverables
-Environment ready with test data set up
-Smoke Test Results.
20

Test Execution
Activities
-Execute tests as per plan
-Document test results, and log defects for failed cases
-Map defects to test cases in RTM
-Retest the Defect fixes
-Track the defects to closure
Deliverables
-Completed RTM with the execution status
-Test cases updated with results
-Defect reports
21

Test Cycle Closure
Activities
-Evaluate cycle completion criteria based on Time, Test coverage, Cost,Software, Critical Business
Objectives, Quality
-Prepare test metrics based on the above parameters.
-Document the learning out of the project
-Prepare Test closure report
-Qualitative and quantitative reporting of quality of the work product to the customer.
-Test result analysis to find out the defect distribution by type and severity.
Deliverables
-Test Closure report
-Test metrics
22

We have learnt the fundamentals of software
Testing.
First Lesson Summary
Objectives
Testing Life cycle
Test Planning

Unit 4 Testing
Lesson 1.
Lesson 2.
Types of Testing
24

Course Progress
Lesson 1.
25

Software Testing Levels
26
Acceptance Testing
System Testing
Integration Testing
Unit Testing

Different Types of Software Testing
Functional testing types include:
Unit testing
Integration testing
System testing
Sanity testing
Smoke testing
Interface testing
Regression testing
Acceptance testing
27

Unit testing
The objective of Unit Testing is to isolate a section of code and verify its correctness
Examples:
-In a program we are checking if loop, method or function is working fine
-Misunderstood or incorrect, arithmetic precedence.
-Incorrect initialization
28

Unit testing
Unit Test Considerations
The module interface is tested to ensure that information properly flows into and out of the
program unit under test.
The local data structure is examined to ensure that data stored temporarily maintains its
integrity during all steps in an algorithm's execution.
Boundary conditions are tested to ensure that the module operates properly at boundaries
established to limit or restrict processing.
All independent paths (basis paths) through the control structure are exercised to ensure that all
statements in a module have been executed at least once.
And finally, all error handling paths are tested.
29

Unit testing
Tests of data flow across a module interface are required before any other test is initiated.
If data do not enter and exit properly, all other tests are moot.
In addition, local data structures should be exercised and the local impact on global data should
be ascertained (if possible) during unit testing.
Selective testing of execution paths is an essential task during the unit test.
Test cases should be designed to uncover errors due to erroneous computations, incorrect
comparisons, or improper control flow.
30

Unit testing
Basis path and loop testing are effective techniques for uncovering a broad array of path errors.
Among the more common errors in computation are
(1) Misunderstood or incorrect arithmetic precedence,
(2) Mixed mode operations,
(3) Incorrect initialization,
(4) Precision inaccuracy,
(5) Incorrect symbolic representation of an expression.
31

Unit testing
Comparison and control flow are closely coupled to one another (i.e., change of flow frequently
occurs after a comparison). Test cases should uncover errors such as
(1) Comparison of different data types,
(2) Incorrect logical operators or precedence,
(3) Expectation of equality when precision error makes equality unlikely,
(4) Incorrect comparison of variables,
(5) Improper or nonexistent loop termination,
(6) Failure to exit when divergent iteration is encountered, and
(7) Improperly modified loop variables
32

Unit testing
Among the potential errors that should be tested when error handling is evaluated are
1. Error description is unintelligible.
2. Error noted does not correspond to error encountered.
3. Error condition causes system intervention prior to error handling.
4. Exception-condition processing is incorrect.
5. Error description does not provide enough information to assist in the location of the
cause of the error.
33

Unit testing
Boundary testing is the last (and probably most important) task of the unit test step.
Software often fails at its boundaries.
That is, errors often occur when the nth element of an n-dimensional array is processed, when
the ith repetition of a loop with i passes is invoked, when the maximum or minimum allowable
value is encountered.
Test cases that exercise data structure, control flow, and data values just below, at, and just
above maxima and minima are very likely to uncover errors.
34

Unit testing
Unit Test Procedures
Unit testing is normally considered as an adjunct to the coding step.
After source level code has been developed, reviewed, and verified for correspondence to
component level design, unit test case design begins.
A review of design information provides guidance for establishing test cases that are likely to
uncover errors in each of the categories discussed earlier.
Each test case should be coupled with a set of expected results.
35

Unit testing
Advantage of Unit Testing
 Can be applied directly to object code and does not require processing source code.
 Performance profilers commonly implement this measure.
Disadvantages of Unit Testing
Insensitive to some control structures (number of iterations)
Does not report whether loops reach their termination condition
Statement coverage is completely insensitive to the logical operators
36

Integration Testing
 Big Bang Approach
 Incremental Approach
which is further divided into
-Top Down Approach
-Bottom Up Approach
-Sandwich Approach - Combination of Top Down and Bottom Up
37

Integration Testing
Here all the components are integrated together at once and then tested.
Advantages:
Convenient for small systems.
Disadvantages:
Fault Localization is difficult.
Given the sheer number of interfaces that need to be tested in this approach, some interfaces link to
be tested could be missed easily.
Since the Integration testing can commence only after "all" the modules are designed, the testing
team will have less time for execution in the testing phase.
Since all modules are tested at once, high-risk critical modules are not isolated and tested on
priority. Peripheral modules which deal with user interfaces are also not isolated and tested on
priority.
38

Bottom-up Integration
It uses Driver modules for testing
39

Bottom-up Integration
Advantages:
Fault localization is easier.
No time is wasted waiting for all modules to be developed unlike Big-bang approach
Disadvantages:
Critical modules (at the top level of software architecture) which control the flow of application
are tested last and may be prone to defects.
An early prototype is not possible
40

Top-down Integration
It uses Stub modules for testing
41

Top-down Integration
Advantages:
Fault Localization is easier.
Possibility to obtain an early prototype.
Critical Modules are tested on priority; major design flaws could be found and fixed first.
Disadvantages:
Needs many Stubs.
Modules at a lower level are tested inadequately.
42

Hybrid/ Sandwich Integration
43

System Testing
Here the entire software is tested.
The last high order testing stage falls outside the boundary of software engineering.
System testing verifies that all elements mesh properly and that overall system
function/performance is achieved.
44

System Testing
Software is the only one element of a larger computer-based system.
Ultimately, software is incorporated with other system elements (e.g., hardware, people,
information), and a series of system integration and validation tests are conducted.
These tests fall outside the scope of the software process and are not conducted solely by
software engineers.
However, steps taken during software design and testing can greatly improve the probability of
successful software integration in the larger system.
A classic system testing problem is "finger-pointing."
This occurs when an error is uncovered, and each system element developer blames the other
for the problem.
45

System Testing
Rather than indulging in such nonsense, the software engineer should anticipate potential
interfacing problems and
 (1) Design error-handling paths that test all information coming from other elements of the
system,
 (2) Conduct a series of tests that simulate bad data or other potential errors at the software
interface,
 (3) Record the results of tests to use as "evidence" if finger-pointing does occur, and
 (4) Participate in planning and design of system tests to ensure that software is adequately
tested.
 System testing is actually a series of different tests whose primary purpose is to fully exercise
the computer-based system.
Although each test has a different purpose, all work to verify that system elements have been
properly integrated and perform allocated functions.
46

Alpha Testing
The alpha test is conducted at the developer's site by a customer. The software is used in a
natural setting with the developer "looking over the shoulder" of the user and recording errors
and usage problems. Alpha tests are conducted in a controlled environment.
47

Beta Testing
The beta test is conducted at one or more customer sites by the end-user of the software.
Unlike alpha testing, the developer is generally not present. Therefore, the beta test is a
"live" application of the software in an environment that cannot be controlled by the
developer.
The customer records all problems (real or imagined) that are encountered during beta
testing and reports these to the developer at regular intervals. As a result of problems
reported during beta tests, software engineers make modifications and then prepare for
release of the software product to the entire customer base.
48

Smoke Vs Sanity Testing
49

Smoke Testing
Smoke testing is an integration testing approach that is commonly used when “shrink –
wrapped” software products are being developed.
It is designed as a pacing mechanism for time-critical projects, allowing the software team to
assess its project on a frequent basis. In essence, the smoke testing approach encompasses the
following activities:
1. Software components that have been translated into code are integrated into a “build.” A
build includes all data files, libraries, reusable modules, and engineered components that
are required to implement one or more product functions.
2. A series of tests is designed to expose errors that will keep the build from properly
performing its function. The intent should be to uncover “show stopper” errors that have
the highest likelihood of throwing the software project behind schedule.
3. The build is integrated with other builds and the entire product (in its current form) is
smoke tested daily. The integration approach may be top down or bottom up
50

Smoke Testing
The daily frequency of testing the entire product may surprise some readers.
However, frequent tests give both managers and practitioners a realistic assessment of
integration testing progress.
McConnell describes the smoke test in the following manner:
“The smoke test should exercise the entire system from end to end. It does not have to be
exhaustive, but it should be capable of exposing major problems. The smoke test should be
thorough enough that if the build passes, you can assume that it is stable enough to be tested
more thoroughly”.
51

Smoke Testing
Smoke testing provides a number of benefits when it is applied on complex, time critical
software engineering projects:
Integration risk is minimized. Because smoke tests are conducted daily, incompatibilities and
other show-stopper errors are uncovered early, thereby reducing the likelihood of serious
schedule impact when errors are uncovered.
 The quality of the end-product is improved. Because the approach is construction
(integration) oriented, smoke testing is likely to uncover both functional errors and
architectural and component-level design defects. If these defects are corrected early, better
product quality will result.
52

Smoke Testing
Error diagnosis and correction are simplified. Like all integration testing approaches, errors
uncovered during smoke testing are likely to be associated with “new software increments”—
that is, the software that has just been added to the build(s) is a probable cause of a newly
discovered error.
 Progress is easier to assess. With each passing day, more of the software has been integrated
and more has been demonstrated to work. This improves team morale and gives managers a
good indication that progress is being made.
53

Validation Testing
Validation Testing – Criterion, which is established during requirement analysis, is tested and it
provides final assurance that software meets all functional behavior, performance requirement
etc.
54

Validation Testing
At the culmination of integration testing, software is completely assembled as a package,
interfacing errors have been uncovered and corrected, and a final series of software tests—
validation testing—may begin.
Validation can be defined in many ways, but a simple (albeit harsh) definition is that validation
succeeds when software functions in a manner that can be reasonably expected by the
customer.
At this point a battle-hardened software developer might protest: "Who or what is the arbiter
of reasonable expectations?"
Reasonable expectations are defined in the Software Requirements Specification— a document
that describes all user-visible attributes of the software.
The specification contains a section called Validation Criteria. Information contained in that
section forms the basis for a validation testing approach
55

Validation Testing
Validation Test Criteria
Software validation is achieved through a series of black-box tests that demonstrate conformity
with requirements.
A test plan outlines the classes of tests to be conducted and a test procedure defines specific
test cases that will be used to demonstrate conformity with requirements.
Both the plan and procedure are designed to ensure that all functional requirements are
satisfied, all behavioral characteristics are achieved, all performance requirements are attained,
documentation is correct, and human engineered and other requirements are met (e.g.,
transportability, compatibility, error recovery, maintainability).
56

Validation Testing
After each validation test case has been conducted, one of two possible conditions exists:
(1) The function or performance characteristics conform to specification and are accepted or
(2) A deviation from specification is uncovered and a deficiency list is created. Deviation or
error discovered at this stage in a project can rarely be corrected prior to scheduled delivery. It
is often necessary to negotiate with the customer to establish a method for resolving
deficiencies
57

Configuration Review
An important element of the validation process is a configuration review.
The intent of the review is to ensure that all elements of the software configuration have been
properly developed, are cataloged, and have the necessary detail to bolster the support phase
of the software life cycle
58

Smoke Testing Vs Sanity Testing
59
Smoke Testing Sanity testing
Smoke testing is performed to ascertain that the
critical functionalities of the program is working is fine
Sanity testing is done to check the new
functionality/ bugs have been fixed
The objective of this testing is to verify the stability of
the system in order to proceed with rigorous testing
The objective of this testing is to verify the
rationality of the system in order to proceed with
rigorous testing
Smoke testing is performed by developers/ testers It is usually performed by testers
Smoke testing is usually documented or scripted Sanity testing is usually documented or unscripted
Smoke testing is a subset of Acceptance testing Sanity testing is a subset of Regression testing
Smoke testing exercises the entire system from end to
end
Smoke testing exercises the only particular
component of the entire system
Smoke testing is like General Health Check up Sanity testing is like a specialised Health Check up

What is Interface Testing?
It verifies whether the communication between two different software systems is done
correctly.
A connection that integrates two components is called interface.
Example:
-API's, web services, etc
60

How to do Interface Testing?
Interface Testing includes testing of two main segments:
-Web server and application server interface
-Application server and Database server interface.
61

The interface testing is done to
Check servers are executed properly or not
Errors are handled properly by an application
Check the outcomes when connection to a web server is reset in between
62

Interface testing Vs Integration Testing
Interface testing:
-An integration test type is concerned with testing the interfaces between the
components
Integration testing:
-Testing is performed to expose the defects in the interfaces and interaction between
the integrated components or system
63

What is Regression Testing?
To confirm that a recent program or code change has not adversely affected existing features.
64

Regression Testing
Each time a new module is added as part of integration testing, the software changes.
New data flow paths are established, new I/O may occur, and new control logic is invoked.
These changes may cause problems with functions that previously worked flawlessly.
In the context of an integration test strategy, regression testing is the re-execution of some
subset of tests that have already been conducted to ensure that changes have not propagated
unintended side effects.
In a broader context, successful tests (of any kind) result in the discovery of errors, and errors
must be corrected.
Whenever software is corrected, some aspect of the software configuration is changed.
Regression testing is the activity that helps to ensure that changes do not introduce unintended
behavior or additional errors.
65

Regression Testing
Regression testing may be conducted manually, by re-executing a subset of all test cases or using
automated capture/playback tools.
Capture/playback tools enable the software engineer to capture test cases and results for
subsequent playback and comparison.
The regression test suite (the subset of tests to be executed) contains three different classes of
test cases:
◦ A representative sample of tests that will exercise all software functions.
◦ Additional tests that focus on software functions that are likely to be affected by the change.
◦ Tests that focus on the software components that have been changed.
 As integration testing proceeds, the number of regression tests can grow quite large. Therefore,
the regression test suite should be designed to include only those tests that address one or more
classes of errors in each of the major program functions. It is impractical and inefficient to re-
execute every test for every program function once a change has occurred. 66

Need of Regression Testing
Regression Testing is required when there is a
-Change in requirements and code is modified according to the requirement
-New feature is added to the software
-Defect fixing
-Performance issue fix
67

How to do Regression Testing?
68

Non-functional testing types
Performance Testing
Load testing
Stress testing
Volume testing
Security testing
Compatibility testing
Install testing
Recovery testing
Reliability testing
Usability testing
Compliance testing
69

Load Testing
Testing the maximum load for an application in which it can with stand without crashing
For example: Client -Server Application
Gmail
70

How to perform Load Testing?
First connect some fixed number users or clients with server application and do normal
operations (Assume if server responds correctly)
Increase some additional number clients connection with server and perform normal operation
(Assume if server responds correctly)
Further increase number of clients connection with server and repeat this until server
withstands without any application crash
At some point, the application fails to respond properly
That is maximum load supported by a particular application
Automated tools are used to perform such tests
Example: Load Runner, Silk Performer
71

Volume Testing
Testing the maximum amount data (volume data) supported by a particular application without
any crashing
72

How to perform Volume Testing?
First connect some fixed number users or clients with server application and do data upload
and download operations (Assume if server responds correctly)
Increase the size of data upload and download (Assume if server responds correctly)
Further increase the size of data upload and download and repeat this until server withstands
without any application crash
That is maximum amount of data supported by a particular application
73

Stress Testing
It is the combination of both load and volume Testing
 calculating the maximum amount stress supported by a particular application without any
crashing
74

How to perform Stress Testing?
First connect some fixed number users or clients with server application and do some fixed size
of data upload and download operations (Assume if server responds correctly)
Increase number clients connections and at the same time increase the size of data upload and
download (Assume if server responds correctly)
Further increase number of connections and the size of data upload and download and repeat
this until server withstands without any application crash
That is maximum number clients with maximum data supported by a particular application
75

Performance Testing
For real-time and embedded systems, software that provides required function but does not
conform to performance requirements is unacceptable.
Performance testing is designed to test the run-time performance of software within the context
of an integrated system.
Performance testing occurs throughout all steps in the testing process. Even at the unit level, the
performance of an individual module may be assessed as white-box tests are conducted.
However, it is not until all system elements are fully integrated that the true performance of a
system can be ascertained.
76

Performance Testing
Performance tests are often coupled with stress testing and usually require both hardware and
software instrumentation.
That is, it is often necessary to measure resource utilization (e.g., processor cycles) in an exacting
fashion.
External instrumentation can monitor execution intervals, log events (e.g., interrupts) as they
occur, and sample machine states on a regular basis.
By instrumenting a system, the tester can uncover situations that lead to degradation and possible
system failure.
77

Security Testing
Any computer-based system that manages sensitive information or causes actions that can
improperly harm (or benefit) individuals is a target for improper or illegal penetration.
Penetration spans a broad range of activities:
 Hackers who attempt to penetrate systems for sport;
 Disgruntled employees who attempt to penetrate for revenge;
 Dishonest individuals who attempt to penetrate for illicit personal gain.
Security testing attempts to verify that protection mechanisms built into a system will, in fact,
protect it from improper penetration.
78

Security testing
It ensures software systems and applications are free from any vulnerabilities, threats, risks
that may cause a big loss.
Security testing of any system is about finding all possible loopholes and weaknesses of the
system which might result into a loss of information, revenue, repute at the hands of the
employees or outsiders of the Organization.
79

Security testing
The goal of security testing is to identify the threats in the system and measure its potential
vulnerabilities, so the system does not stop functioning or is exploited.
It also helps in detecting all possible security risks in the system and help developers in fixing
these problems through coding.
80

Compatibility testing
Compatibility Testing is a type of Software testing to check whether your software is capable of
running on different hardware, operating systems, applications, network environments or
Mobile devices.
81

Install testing
Most software systems have installation procedures that are needed before they can be used
for their main purpose. Testing these procedures to achieve an installed software system that
may be used is known as installation testing. These procedure may involve full or partial
upgrades, and install/uninstall processes.
82

Recovery testing
Recovery testing is the activity of testing how well an application is able to recover from
crashes, hardware failures and other similar problems. Recovery testing is the forced failure of
the software in a variety of ways to verify that recovery is properly performed.
83

Recovery Testing
Many computer based systems must recover from faults and resume processing within a pre-
specified time.
In some cases, a system must be fault tolerant; that is, processing faults must not cause
overall system function to cease.
In other cases, a system failure must be corrected within a specified period of time or severe
economic damage will occur.
Recovery testing is a system test that forces the software to fail in a variety of ways and
verifies that recovery is properly performed.
If recovery is automatic (performed by the system itself), re-initialization, check-pointing
mechanisms, data recovery, and restart are evaluated for correctness.
If recovery requires human intervention, the mean-time-to-repair (MTTR) is evaluated to
determine whether it is within acceptable limits.
84

Reliability testing(The ability of the Software)
It is defined as a software testing type, that checks whether the software can perform a failure-
free operation for a specified period of time in a specified environment. ... Reliability testing in
software assures that the product is fault free and is reliable for its intended purpose
85

COMPLIANCE TESTING
It is also known as conformance testing, regulation testing, standards testing, is a type of
testing to determine the compliance of a system with internal or external standards.
86

COMPLIANCE TESTING
87
... our goal is to ensure that all
statements and conditions have
been executed at least once ...

White box Testing
Uses the control structure part of component-level design to derive the test cases
These test cases
-Guarantee that all independent paths within a module have been exercised at least once
-Exercise all logical decisions on their true and false sides
-Execute all loops at their boundaries and within their operational bounds
-Exercise internal data structures to ensure their validity
88

White Box Testing/Gloss Box Testing/ Structural
Testing
In this testing technique the internal logic of software components is tested.
 It is a test case design method that uses the control structure of the procedural design test
cases.
 It is done in the early stages of the software development.
 Using this testing technique software engineer can derive test cases that:
 All independent paths within a module have been exercised at least once.
 Exercised true and false both the paths of logical checking.
 Execute all the loops within there boundaries.
 Exercise internal data structures to ensure their validity
89

Testing
Advantages:
As the knowledge of internal coding structure is prerequisite, it becomes very easy to find out
which type of input/data can help in testing the application effectively.
 The other advantage of white box testing is that it helps in optimizing the code.
 It helps in removing the extra lines of code, which can bring in hidden defects.
 We can test the structural logic of the software.
 Every statement is tested thoroughly.
 Forces test developer to reason carefully about implementation.
 Approximate the partitioning done by execution equivalence.
Reveals errors in "hidden" code
90

Testing
Disadvantages:
 It does not ensure that the user requirements are fulfilled.
 As knowledge of code and internal structure is a prerequisite, a skilled tester is needed to carry
out this type of testing, which increases the cost.
 It is nearly impossible to look into every bit of code to find out hidden errors, which may create
problems, resulting in failure of the application.
 The tests may not be applicable in real world situation.
 Cases omitted in the code could be missed out
91

Loop Testing
Loop testing is a white-box testing technique that focuses exclusively on the validity of loop
constructs. Four different classes of loops can be defined: simple loops, concatenated loops,
nested loops, and unstructured loops.
Simple loops – The following set of tests can be applied to simple loops, where n is the
maximum number of allowable passes through the loop.
 Skip the loop entirely.
 Only one pass through the loop.
 Two passes through the loop.
 m passes through the loop where m < n.
 n – 1, n, n + 1 passes through the loop
92

Loop Testing
Nested loops – If we were to extend the test approach for simple loops to nested loops, the
number of possible tests would grow geometrically as the level of nesting increases. This would
result in an impractical number of tests. Beizer suggests an approach that will help to reduce the
number of tests:
 Start at the innermost loop. Set all other loops to minimum values.
 Conduct simple loop tests for the innermost loop while holding the outer loops at their
minimum iteration parameter (e.g., loop counter) values. Add other tests for out-of-range or
excluded values.
 Work outward, conducting tests for the next loop, but keeping all other outer loops at
minimum values and other nested loops to "typical" values.
 Continue until all loops have been tested
93

Loop Testing
Concatenated loops – Concatenated loops can be tested using the approach defined for simple
loops, if each of the loops is independent of the other.
However, if two loops are concatenated and the loop counter for loop 1 is used as the initial
value for loop 2, then the loops are not independent.
When the loops are not independent, the approach applied to nested loops is recommended.
Unstructured loops – Whenever possible, this class of loops should be redesigned to reflect the
use of the structured programming constructs
94

Basic Path Testing
Basis path testing is a white-box testing technique first proposed by Tom McCabe.
The basis path method enables the test case designer to derive a logical complexity measure of
a procedural design and use this measure as a guide for defining a basis set of execution paths.
Test cases derived to exercise the basis set are guaranteed to execute every statement in the
program at least one time during testing.
Before the basis path method can be introduced, a simple notation for the representation of
control flow, called a flow graph (or program graph) must be introduced.
The flow graph depicts logical control flow using the notation illustrated in Figure below. Each
structured construct has a corresponding flow graph symbol.
95

Basis Path Testing
Enables the test case designer to derive a logical complexity measure of a procedural design
Uses this measure as a guide for defining a basis set of execution paths
Test cases derived to exercise the basis set are guaranteed to execute every statement in the
program at least one time during testing
96

Flow graph Notation
A circle in a graph represents a node, which stands for a sequence of one or more procedural
statements
A node containing a simple conditional expression is referred to as a predicate node
Each compound condition in a conditional expression containing one or more Boolean operators
(e.g., and, or) is represented by a separate predicate node
A predicate node has two edges leading out from it (True and False)
An edge, or a link, is a an arrow representing flow of control in a specific direction
An edge must start and terminate at a node
An edge does not intersect or cross over another edge
Areas bounded by a set of edges and nodes are called regions
When counting regions, include the area outside the graph as a region, too
97

1
2
0
3
4
5
6
7 8
9
10
11
1
2
3
4
6
7 8 5
9
10
11
R1
R2
R3
R4
FLOW CHART FLOW GRAPH
0

Independent Path Testing
Defined as a path through the program from the start node until the end node that introduces
at least one new set of processing statements or a new condition (i.e., new nodes)
Must move along at least one edge that has not been traversed before by a previous path
Basis set for flow graph on previous slide
Path 1: 0-1-11
Path 2: 0-1-2-3-4-5-10-1-11
Path 3: 0-1-2-3-6-8-9-10-1-11
Path 4: 0-1-2-3-6-7-9-10-1-11
The number of paths in the basis set is determined by the cyclomatic complexity
99

Cyclomatic Complexity
Provides a quantitative measure of the logical complexity of a program
Defines the number of independent paths in the basis set
Provides an upper bound for the number of tests that must be conducted to ensure all statements have been
executed at least once
Can be computed three ways
-The number of regions
-V(G) = E – N + 2, where E is the number of edges and N is the number of nodes in graph G
-V(G) = P + 1, where P is the number of predicate nodes in the flow graph G
 Results in the following equations for the example flow graph
Number of regions = 4
V(G) = 14 edges – 12 nodes + 2 = 4
V(G) = 3 predicate nodes + 1 = 4
100

Deriving the Basis Set and Test Cases
Using the design or code as a foundation, draw a corresponding flow graph
Determine the cyclomatic complexity of the resultant flow graph
Determine a basis set of linearly independent paths
Prepare test cases that will force execution of each path in the basis set
101

Sample Code
IF A = 10 THEN
IF B > C THEN
A = B
ELSE A = C
ENDIF
ENDIF
Print A
Print B
Print C
102

Sample Code-Flow Graph

V(G) = E – N + 2
V(G) = P + 1
V(G)=Number of Regions
104

Conclusion
Cyclomatic complexity calculation is one of the standard approaches to calculate the source
code complexity and determine the risk that the source code possesses for any future
modification and maintenance.
Lower the Cyclomatic complexity better is the code quality in terms of complexity.
105

Advantages:
White box testing is very thorough as the entire code and structures are tested.
It results in the optimization of code removing error and helps in removing extra lines of code.
It can start at an earlier stage as it doesn’t require any interface as in case of black box testing.
Easy to automate.
106

Disadvantages:
Main disadvantage is that it is very expensive.
Redesign of code and rewriting code needs test cases to be written again.
Testers are required to have in-depth knowledge of the code and programming language as
opposed to black box testing.
Missing functionalities cannot be detected as the code that exists is tested.
Very complex and at times not realistic.
107

Black Box Testing
108
Executable Code
INPUT OUTPUT

Black Box Testing
Incorrect or missing functions
Interface errors
Errors in data structures or external database access
Behaviour or performance errors
Initialization and termination errors
109

How to do Blackbox Testing?
Initially, the requirements and specifications of the system are examined.
Tester chooses valid inputs (positive test scenario) to check whether SUT processes them
correctly. Also, some invalid inputs (negative test scenario) are chosen to verify that the SUT is
able to detect them.
Tester determines expected outputs for all those inputs.
Software tester constructs test cases with the selected inputs.
The test cases are executed.
Software tester compares the actual outputs with the expected outputs.
Defects if any are fixed and re-tested.
110

Types of Blackbox Testing
Functional testing - This black box testing type is related to the functional requirements of a
system; it is done by software testers.
Non-functional testing - This type of black box testing is not related to testing of specific
functionality, but non-functional requirements such as performance, scalability, usability.
Regression testing - Regression Testing is done after code fixes, upgrades or any other system
maintenance to check the new code has not affected the existing code.
111

Black Box Testing techniques
Equivalence Class Testing: It is used to minimize the number of possible test cases to an
optimum level while maintains reasonable test coverage.
Boundary Value Testing: Boundary value testing is focused on the values at boundaries. This
technique determines whether a certain range of values are acceptable by the system or not. It
is very useful in reducing the number of test cases. It is most suitable for the systems where an
input is within certain ranges.
112

Equivalence Partitioning
Test design technique
Divide input test data in to partitions
Test each partition once ( The assumption is that any input with in a partition is equivalent i.e.,
produces the same output)
113

Date (1 to 31)
114
Invalid Partition Valid Partition Invalid Partition
0 1 32
-1 2 33
-2 3 .
-3 . .
. . .
. 31 .

Username (6 to 10 characters)
115
Invalid Partition Valid Partition Invalid Partition
0 char 6 char 11 char
1 char 7 char 12 char
2 char 8 char .
3 char . .
. . .
5 char 10 char .

Age (18 to 80 Years) except 60 to 65 years
116
Invalid Partition Valid Partition Invalid Partition Valid Invalid
0 18 year 60 year 66 year 81
1 year 19 year 61 year 82
2 year . . .
3 year . . .
. . . .
17char 59 year 65 year 80 year

Condition Testing
Condition testing is a test case design method that exercises the logical conditions contained in a
program module.
 A simple condition is a Boolean variable or a relational expression, possibly preceded with one NOT
(¬) operator. A relational expression takes the form
E1 <relational-operator> E2
Where E1 and E2 are arithmetic expressions and <relational-operator> is one of the following: <, ≤, =,
≠ (nonequality), >, or ≥.
117

Condition Testing
A compound condition is composed of two or more simple conditions, Boolean operators, and
parentheses. We assume that Boolean operators allowed in a compound condition include OR ( | ),
AND (&) and NOT (¬). A condition without relational expressions is referred to as a Boolean
expression.
 Therefore, the possible types of elements in a condition include a Boolean operator, a Boolean
variable, a pair of Boolean parentheses (surrounding a simple or compound condition), a relational
operator, or an arithmetic expression
118

Condition Testing
If a condition is incorrect, then at least one component of the condition is incorrect. Therefore,
types of errors in a condition include the following:
 Boolean operator error (incorrect/missing/extra Boolean operators).
 Boolean variable error.
 Boolean parenthesis error.
 Relational operator error.
 Arithmetic expression error.
The condition testing method focuses on testing each condition in the program
119

Condition Testing
Condition testing strategies generally have two advantages.
 Measurement of test coverage of a condition is simple.
 The test coverage of conditions in a program provides guidance for the generation of
additional tests for the program.
 The purpose of condition testing is to detect not only errors in the conditions of a program but
also other errors in the program.
If a test set for a program P is effective for detecting errors in the conditions contained in P, it is
likely that this test set is also effective for detecting other errors in P.
In addition, if a testing strategy is effective for detecting errors in a condition, then it is likely
that this strategy will also be effective for detecting errors in a program.
120

Difference between Error, Failure and Defect
Error – The term error is used to refer to the discrepancy between computed, observed or
measured value and the specified value. In other terms errors can be defined as the difference
between actual output of software and correct output.
Fault – It is a condition that causes a system to fail in performing its required function.
Failure – A software failure occurs if the behavior of the software is different from specified
behaviour. It is a stage when system becomes unable to perform a required function according
to the specification mentioned.
Defect: A error found by tester is called Defect
Bug: A defect accepted by development team then it is called Bug
121

Testing Principles
All tests should be traceable to customer requirements
Tests should be planned long before testing begins
The Pareto principle applies to software testing
 Testing should begin “in the small” and progress toward testing “in the large”
Exhaustive testing is not possible
To be most effective, testing should be conducted by an independent third party
122

Attributes of Good Test
A good test has a high probability of finding an error
A good test is not redundant
A good test should be “best of breed”.
A good test should be neither too simple nor too complex.
123

Test Case Design
The test case design provides the developer with a systematic approach to test and also
provide a mechanism that can help to ensure the completeness of tests and provide the highest
likelihood errors in software.
Since, effort and time both is needed to execute the test cases and also machine time is
needed to execute the program for those test cases also, effort is needed to evaluate the
results.
Hence, the fundamental goals of a testing activity are to minimize the test cases and maximize
the error unfolded.
According to Yamaura there is only one rule in designing test cases and it is that it should cover
all features, but do not make too many test cases.
An ideal set of test cases is one that includes all the possible inputs to the program. This is
called exhaustive testing.
124

We have learnt about the various types of testing.
Second Lesson Summary
Unit Testing
Integration Testing
System Testing
Blackbox Testing
White box testing
Alpha Testing
Smoke Testing

Unit 4 Testing
Lesson 1.
Lesson 3.
Configuration Management
126

Course Progress
Lesson 1.
127

Software Configuration Management (SCM) is the discipline of controlling the evolution of
software systems.
SCM is indispensable for the development and maintenance of large, long-lived software.
Ideal:
-Software is developed from stable/frozen requirements
-The concept is that it is easier to hit a stationary target than a moving target
Reality:
-Not applicable for most real-world systems
The only constant is “CHANGE”
-An effective software project need to have a strategy to tackle “CHANGE”
128
Software Configuration Management

Software changes constantly.
Change causes chaos.
SCM helps to prevent change chaos:
-Tracks development history,
-Prevents or reconciles conflicting changes,
-Tracks configurations and baselines,
-Tracks error reports and fixes,
-Automates configuration selection,
-keeps software in well-defined state.
129
Why is it needed?

Problems of identification and tracking:
-“This program worked yesterday. What happened?”
-“I fixed this error last week. Why is it back?”
-“Where are all my changes from last week?”
-“This seems like an obvious fix. Has it been tried before?”
-“Who is responsible for this change?”
130
Symptoms of the Change Chaos

Problems of version selection
-“Has everything been compiled? Tested?”
-“How do I configure for test, with my updates and no others?”
-“How do I exclude this incomplete/faulty change?”
-“I can’t reproduce the error in this copy!”
-“Exactly which fixes went into this configuration?”
-“Oh shoot. I need to merge 250 files!”
131
Change Chaos Continued

Software Delivery Problems
“Which configuration does this customer have?”
“Did we deliver a consistent configuration?”
“Did the customer modify the code?”
“The customer skipped the previous two releases. What happens if we send him the
new one?”
These problems sound familiar to anyone who has worked in software development.
132
More Chaos ….

Configuration management was first developed for the aerospace industry in the 1950’s. It was
used among others in the Apollo Space Program for tracking thousands of changes.
SCM began in the 1970’s with SCCS.
SCM is now a successful part of the software tools market, with at least 50 tools available.
SCM tools deliver recognized benefits to users.
SCM is a technically challenging area.
133
History of SCM

1.Identification and Establishment – Identifying the configuration items from products that
compose baselines at given points in time (a baseline is a set of mutually consistent
Configuration Items, which has been formally reviewed and agreed upon, and serves as the
basis of further development). Establishing relationship among items, creating a mechanism to
manage multiple level of control and procedure for change management system.
Suppose after some changes, the version of configuration object changes from 1.0 to 1.1.
Minor corrections and changes result in versions 1.1.1 and 1.1.2, which is followed by a major
update that is object 1.2. The development of object 1.0 continues through 1.3 and 1.4, but
finally, a noteworthy change to the object results in a new evolutionary path, version 2.0. Both
versions are currently supported.
134
Processes involved in SCM

A software configuration audit complements the formal technical review of the process and
product. It focuses on the technical correctness of the configuration object that has been
modified. The audit confirms the completeness, correctness and consistency of items in the
SCM system and track action items from the audit to closure.
137
Configuration auditing

Providing accurate status and current configuration data to developers, tester, end users,
customers and stakeholders through admin guides, user guides, FAQs, Release notes, Memos,
Installation Guide, Configuration guide etc .
138
Reporting

CFEngine, Bcfg2 server,
Vagrant, SmartFrog,
CLEAR CASETOOL (CC), SaltStack,
CLEAR QUEST TOOL, Puppet,
SVN- Subversion, Perforce,
TortoiseSVN, IBM Rational team concert,
 IBM Configuration management version management,
Razor, Ansible, etc
139
SCM Tools

These activities help in exposing errors, provide increased project communication, keeping the
project in schedule, and verification that the design satisfies the requirements.
-Milestones
-Walkthroughs
-Inspections
140
Milestones, Walkthroughs and Inspections

These are a set of occasions in project design where the proper progress of the project can be
assessed in such a way that corrective measures could be taken if necessary.
 The two major milestones are –
 Preliminary Design Review (PDR) :- Its normally held near the end of architectural design and
prior to detailed design
Critical design Review (CDR) :- Its normally held at the end of detailed design and prior to
implementation.
141
Milestones

The major goal of PDR is to demonstrate the externally observable characteristics and
architectural structure of the product which
would satisfy the customer’s requirements. Functional characteristics, performance attributes,
external interface, user dialogs, report formats, exception conditions and exception handling
and future enhancements are reviewed during PDR.
 The CDR provides a final management decision point , to build or cancel the system
142
Milestones..

A structured walkthrough is an in-depth, technical review of some aspects of a software
system. Walkthroughs can be anytime, during any phase of a software project.
 A walkthrough team consists of 4 to 6 people. The person whose material is being reviewed is
responsible for providing copies of the review materials to the members of the walkthrough
group in advance of the walkthrough session and the team members are responsible for
understanding the reviewing material before the session.
 During the walkthrough the reviewed “walks through” the material while the reviewers look
for errors, request clarification and explore problem areas in the material under review.
 High-level managers should not attend walkthrough sessions as the aim of walkthroughs is
error detection not corrective action. Its important to note that the material is reviewed not the
person whose material is being reviewed.
143
Walkthroughs

Design inspections are conducted by teams of trained inspectors who have a check list of items
to be examined.
 Special forms are used to record problems encountered.
 A typical inspection team consists of a Moderator or Secretary, a Designer, an Implementer
and a Tester. The Designer, Implementer and Tester may or may not be the people responsible
for the actual design, implementation and testing of the product being inspected.
The team members are trained for their specific roles and typically conduct a dual 2-hrs
sessions per day.
144
Inspections

Inspections
Design inspections are conducted by teams of trained inspectors who have a check list of items to be
examined.
 Special forms are used to record problems encountered.
 A typical inspection team consists of a Moderator or Secretary, a Designer, an Implementer and a Tester.
The Designer, Implementer and Tester may or may not be the people responsible for the actual design,
implementation and testing of the product being inspected.
The team members are trained for their specific roles and typically conduct a dual 2-hrs sessions per
day.
SOFTWARE ENGINEERING 145

We have learned about different User Interface design
process, Software Configuration Management System and real
time SCM tools. We will learn about the different types testing
activities to verify that the developed software meets the
user requirement in next unit.
Third Lesson Summary
Software Configuration Management
History
SCM Process
SCM Tools

References
Software Engineering: A Practitioner’s approach, by Roger. S. Pressman, 5th edition
Software Engineering, by Ivan Sommerville, Ninth edition.
“Fundamentals of Software Engineering by Rajib Mall, 7th edition
Software Engineering by Gaurav Gupta and Deepika Gupta, 2016.
Software Engineering 1st Edition by Udit Agarwal
147

Thank You!
SOFTWARE ENGINEERING 148

Unit 4- Testing.pptx

More Related Content

Similar to Unit 4- Testing.pptx

More from LSURYAPRAKASHREDDY

Recently uploaded

Unit 4- Testing.pptx