The document discusses the evolving discipline of software engineering, highlighting the growing demand for skilled professionals and the importance of structured processes in software development. It emphasizes the critical role of software testing as a sub-process that integrates with the overall quality assurance and defines the Testing Maturity Model (TMM) that helps organizations improve their software testing processes. The TMM is structured in five levels, describing the journey from ad-hoc practices to defined and optimized methods for effective quality assurance.

1. IT 8076 Software Testing
UNIT - I
Dr. S. Roselin Mary,
Professor & Head
Department of Computer Science and Engineering
(NBA Accredited)
Anand Institute of Higher Technology, Chennai

2. The Evolving profession of software
Engineering
• S/W systems are becoming more challenging
to build
• People with s/w development skills are in
demand
• New methods, techniques and tools are
available to support development and
maintenance tasks
• Towards software engineering
• Need for software engineering as a discipline
6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
2

3. • Role of IEEE and ACM
6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
3

4. • Using an engineering approach to s/w
development implies that
– The development of the process is well understood.
– Projects are planned.
– Life cycle models are defined and adhered to.
– Standards are in place for product and process
– Measurements are employed to evaluate product and
process quality.
– Components are reused.
– Validation and Verification process play a key role in
quality determination.
– Engineers should have proper education, training and
certification.
6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
4

5. • A test specialist is one whose education is based on the principles,
practices and processes that constitute the software engineering
discipline and whose specific focus is on one area of that discipline,
software testing.
• A test specialist who is trained as an engineer should have
knowledge on the following
– Test related principles,
– Processes,
– Measurements,
– Standards,
– Plans,
– Tools and methods
– How to apply them to the testing tasks.
• Testing concepts, are not an isolated collection of technical and
managerial activities, it should be integrated within the context of a
quality testing process. It grows in competency and uses
engineering principles to guide improvement growth.
6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
5

6. The Role of process in software quality
• The need for high quality software products has pressured those in the
software profession
– to identify and quantify quality factors such as usability, testability,
maintainability and reliability
– to identify engineering practices that support the production of quality
products having these favorable attributes.
• Identified Practices that contribute to the development of high-quality
software are
– Project Planning
– Requirements Management
– Development of formal specification
– Structured design with use of information hiding and encapsulation,
– design and code reuse,
– inspections and reviews,
– product and process measurements,
– education and training of software professionals,
– development and application of CASE tools,
– use of effective testing techniques and integration of testing activities into the
entire life cycle.6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
6

7. • Process, in the software engineering domain, is the set of
methods, practices, standards, documents, activities, policies,
and procedures that software engineers use to develop and
maintain a software system and its associated artifacts, such
as project and test plans, design documents, code, and
manuals.
• Adding individual practices to an existing software development
practices in an adhoc way is not satisfactory.
•
• The software development process is similar to any other
engineering activity, it must be engineered. It must be
designed, implemented, evaluated and maintained.
• Similar to other engineering process, a software development
process must evolve in a consistent and predictable manner.
• Both best technical and managerial practices must be
integrated in a systematic manner.6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
7

8. 6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
8

9. • Models(CMM,SPICE,..) were developed to address process
issues.
• Models allow an org/- to evaluate its current software process
and to capture its state.
• Models provide a strong support for incremental process
improvement, consistent with historical process evolution and
the appln of quality principles
• Widely accepted s/w process improvement models in industry
are high level models. They focus on the s/w process as a whole ,
not on a specific s/w development sub process(design, testing, …)
• CMM, Bootstrap and ISO-9000 are not adequately addressing the
testing process issues.
• But Testing Maturity Model (TMM) is addressing testing process
issues
6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
9

10. Testing as a process
Validation is the process of evaluating a
software system or component during, or
at the end of, the development cycle in
order to determine whether it satisfies
specified requirements.
Verification is the process of evaluating a
software system or component to
determine whether the products of a
given development phase satisfy the
conditions imposed at the start of that
phase
Validation is usually associated with
traditional execution-based testing, that
is, exercising the code with test cases.
Verification is usually associated with
activities such as inspections and reviews
of software deliverables.
Is it the right product? Is the product right?
6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
10
Definitions of Testing:
• Testing is generally described as a group of procedures carried out to evaluate some
aspect of a piece of software.
• Testing can be described as a process used for revealing defects in software, and for
establishing that the software has attained a specified degree of quality with respect
to selected attributes.
The definitions also describe testing as a dual-purpose process²one that reveals defects, as well as one
that is used to evaluate quality attributes of the software such as reliability, security, usability, and
correctness.

11. • The debugging process begins after testing has been carried out and the tester has
noted that the software is not behaving as specified.
• Debugging / fault localization is the process of (1) locating the fault or defect, (2)
repairing the code, and (3) retesting the code.
• Testing as a process has economic, technical and managerial aspects.
Economic aspects are related to the reality that resources and time are available to
the testing group on a limited basis. In fact, complete testing is in many cases not
practical because of these economic constraints. An organization must structure its
testing process so that it can deliver software on time and within budget, and also
satisfy the client’s requirements.
Technical aspects of testing relate to the techniques, methods, measurements, and
tools used to insure that the software under test is as defect-free and reliable as
possible for the conditions and constraints under which it must operate.
In the Managerial aspects of testing , an organizational policy for testing must be
defined and documented. Testing procedures and steps must be defined and
documented. Testing must be planned, testers should be trained, the process should
have associated quantifiable goals that can be measured and monitored. Testing as a
process should be able to evolve to a level where there are mechanisms in place for
making continuous improvements.
6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
11

12. Overview of Testing Maturity Model
• Issues that lead to TMM:
– There is a demand for s/w of high quality with low defects
– Process is important in the s/w engineering discipline
– s/w testing is an important s/w development sub process
– Existing s/w evaluation and improvement models haven’t adequately
addressed testing issues.
• Benefits of Test process Improvement:
– Smarter testers
– Higher quality s/w
– Ability to meet budget and scheduling goals
– Improved planning
– Ability to meet quantifiable testing goals
• Test process Improvement is supported by the set of levels and Maturity
goals in the TMM
• Achievement of maturity goals results in incremental improvement of an
organizations testing process
6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
12

13. The following four sources served as the principal inputs to TMM
development:
1. Capability Maturity Model (CMM):
– It is a staged process improvement model
– It prescribes the stages that an organization must proceed through
in an orderly fashion to improve its s/w development process
– It has 5 levels /stages
– Each level has a set of Key Process Areas(KPA) that an organization
needs to focus on to achieve maturity at that level.
– It also has an assessment procedure for an organization to
evaluate the current state of its s/w process
2. Gelperin and Hetzel’s Evolutionary Testing Model
– describes the evolution of the testing process in industry over a 40
year period
3. Beizer’s Testing model
– Describes the evolution of the individual tester’s thinking
4. Software Testing practices Survey Report
– Identifies best test practices in industry as of 1993
6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
13

14. TMM Levels
• It follows staged architecture for process improvement models
• It contains 5 levels from adhoc and unmanaged to managed, defined,
measured and optimizable
• Internal structure of the TMM is the best in testing practices that can
be learned and applied in a systematic way to support a quality
testing process that improves in incremental steps
• Characteristics of each level are described in terms of testing
capability organizational goals and roles & responsibilities for the key
players in the testing process
• Except level 1, each level has a structure that contains the following:
– a set of maturity goals
• maturity goals identify testing improvement goals
• They should be addressed to achieve maturity at that level
– Supporting maturity subgoals
• They define scope, boundaries and needed accomplishments for a particular
level
– Activities, tasks and responsibilities (ATR)
• ATRs address implementation and organizational adaptation issues at each TMM
level. Supporting activities and tasks are identified and responsibilities are
assigned to appropriate groups6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
14

15. 6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
15
Commitment ability to perform
ATs to improve testing capability Technical ATs constitute
Quality Testing practice
Cooperating/
supporting view

16. 6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
16

17. • testing is a chaotic process
• No documented set of specification for s/w behavior
• Tests are developed in an adhoc way
• Testing & debugging are interleaved to get the bugs
out of s/w
• Objective of testing is to show the s/w works
• s/w products are released without quality assurance
• Lack of resources, tools, properly trained staff
Level 1: Initial
6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
17

18. 6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
18
Level 2
Primary Activity :
Post code , Execution based
testing
TMM Testing is separated from
debugging.
It is defined as a phase that
follows coding
It is a planned activity
The primary goal of testing at this
phase is to show that the s/w
meets its given specifications.
Basic testing techniques and
methods have use of blackbox
and whitebox testing stratergy
and validation cross reference
matrix
Multilevel testing :Unit,
Integration, System and
Acceptance levels
Drawbacks:
As testing plan occurs late, many
quality problem occurs
Defects are propagated from the
requirements and design phases
into the code
No review programs

19. 6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
19
Level 3
Primary Activity :
Testing integrated into
the entire s/w life cycle
Planning for testing begins
at the requirements phase
and continues throughout
the life cycle
Test objectives are
established with respect
to the requirements based
on user needs
There is a technical
training organization with
a testing focus
Testing is monitored and
deviation is observed
Drawbacks:
No formal review
programs
No formal test
measurement programs to
quantify the number of
process and product
attributes

20. 6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
20
Level 4
Primary Activity :
Reviews and Test
measurements
Testing at this level
becomes a process that is
measured and quantified.
Reviews at all phases
An extension of the V-
model
s/w products are tested for
quality attributes
Test cases from all projects
are collected and recorded
in a test case database for
reuse and regression testing
Defects are logged and
given a severity level.
Drawbacks:
Lack of defect prevention
Lack of support for the
collection, analysis and
dissemination of test
related metrics.

21. 6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
21

22. 6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
22

23. 6-Dec-19
Dr.S.Roselin Mary
Professor, HOD/CSE
23
Level 5
Primary Activity :
Monitoring costs and
effectiveness
Testing at this level becomes a process that is
defined and managed.
Testing can be fine tuned and continuously
improved
Defect prevention and quality control are practiced
Stratistical sampling, measurements of confidence
levels, trustworthiness and reliability
Automated Test tools support the running and
rerunning of Testcases. They support for test case
design, maintenance of test related items and
defect collection and analysis
Process reuse is supported by Process Asset Library
(PAL)