Software Testing principle Testers Role Defects & Defect classes

1. IT 8076 Software Testing
UNIT – I – PART 2
Dr. S. Roselin Mary,
Professor & Head
Department of Computer Science and Engineering
(NBA Accredited)
Anand Institute of Higher Technology, Chennai

2. • Error is a mistake , misconception/misunderstanding
on the part of a s/w developer.
• A Fault/Defect is introduced into s/w as a result of an
error . It is an anomaly in the s/w that may cause it to
behave incorrectly, and not according to its
specification
• A Failure is the inability of a software
system/component to perform its required functions
within specified performance requirements
A fault in the code doesn’t
always produce a failure.
Justify.
12/08/2019 2
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE

3. • s/w that easily reveals its faults as failures is said
to be more testable.
• A Test case in a practical sense is a test related
item which contains the following information:
– A set of test Inputs: These are data items received
from an external source by the code under test. The
external source can be h/w, s/w or human.
– Execution conditions: These are conditions required
for running the test, for example, a certain state of a
database, or a configuration of a h/w device.
– Expected outputs: These are the specified results to
be produced by the code under test
12/08/2019 3
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE

4. • Test
A test is a group of related test cases, or a group of related
test cases and test procedures.
Test set  group of related tests
Test suite group of related tests that are associated with a
database and usually run together
• Test Oracle
A test oracle is a document, or piece of software that allows
testers to determine whether a test has been passed or
failed.
• Test Bed
A test bed is an environment that contains all the hardware
and software needed to test a software component or a
software system.
12/08/2019 4
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE

5. • Software Quality – IEEE std definitions
1. Quality relates to the degree to which a system, system component,
or process meets specified requirements.
2. Quality relates to the degree to which a system, system component,
or process meets customer or user needs, or expectations.
• Metrics
A metric is a quantitative measure of the degree to which a system,
system component, or process possesses a given attribute.
There are product and process metrics.
E.g:
- software product metric is software size, usually measured in lines of
code (LOC).
- Process metrics are costs and time required for a given task.
• Quality metrics:
A Quality metric is a quantitative measurement of the degree to which
an item possesses a given quality attribute
12/08/2019 5
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE

6. Examples of Quality attributes:
• correctness—the degree to which the system performs its intended
function
• reliability—the degree to which the software is expected to perform
its required functions under stated conditions for a stated period of
time
• usability—relates to the degree of effort needed to learn, operate,
prepare input, and interpret output of the software
• integrity—relates to the system’s ability to withstand both intentional
and accidental attacks
• portability—relates to the ability of the software to be transferred
from one environment to another
• maintainability—the effort needed to make changes in the software
• interoperability—the effort needed to link or couple one system to
another.
• Testability –
– the amount of effort needed to test the software to ensure it performs
according to specified requirements ,
– the ability of the software to reveal defects under testing conditions
(some software is designed in such a way that defects are well hidden
during ordinary testing conditions).
12/08/2019 6
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE

7. • Software Quality Assurance Group
The software quality assurance (SQA) group in an organization has ties
to quality issues. The group serves as the customers’ representative
and advocate. Their responsibility is to look after the customers’
interests.
The software quality assurance (SQA) group is a team of people with
the necessary training and skills to ensure that all necessary actions
are taken during the development process so that the resulting
software conforms to established technical requirements.
• Review
A Review is a group meeting whose purpose is to evaluate a software
artifact or a set of software artifacts.
• Audit
An audit is a special type of review conducted by SQA group to assess
the compliance with specifications, standards and contractual
agreements.
12/08/2019 7
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE

8. • Software testing Principles:
Principle – Defn
• a general/fundamental law, doctrine/assumption
• A rule or code of conduct
• The laws/facts of nature underlying the working of an artificial device
Software Engineering Principles
• laws, rules/ doctrines that relate to software systems, how to build them and how they behave
Testing principles
• guide testers in defining how to test software systems and provide rules of conduct for testers
as professionals
Glenford Myer’s Principles / Execution based testing principles:
1. Testing is the process of exercising a software component using a selected
set of testcases to reveal defects and to evaluate quality
– Supports testing as an execution based activity to detect defects
– Supports the separation of testing from debugging
– Locate the defect & repair the defect
– Software component – to represent any unit of software ranging in size &
complexity from an individual procedure/method to entire software
SOFTWARE TESTING PRINCIPLES
12/08/2019 8
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE

9. – defects – deviations in the software that have a –ve impact on its specified QA’s
– Bertolino –“Guide to Software Engg Body of Knowledge” – Book – Testing as a dynamic
process that executes a program on values inputs
– Reviews & other static analysis techniques
– Explain
2. When the test objective is to detect defects, then a good test case is one that
has a high probability of revealing yet undetected defects.
– Supports careful test design
– Provides a criterion with which to evaluate test case design and the effectiveness of the
testing efforts to detect defects
– Tester – consider the goal for each test case – which specific type of defect to be
detected by the test case
– Tester – scientist approach – hypothesis –to prove/disprove – explain
3. Test results should be inspected meticulously
– Failure may be overlooked
– Failure may be suspected
– Outcome of the quality test may be misunderstood – unnecessary rework/oversight of a
problem
4. Test case must contain the expected output or result
– Explain
– Quality goals- quantitative terms – explain
5. Test cases should be developed for both valid & invalid input conditions
–explain
6. The probability of the existence of additional defects in a software component
is proportional to the number of defects already detected in that component12/08/2019 9
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE

10. – Explain - defects in two projects
7. Testing should be carried out by a group that is independent of
the development group
– Explain –why
8. Tests must be repeatable and reusable
– Explain the situation-when repeatable and when reusable
9. Testing should be planned
– Test plan for each level & objective for each level
– Objective – quantitatively
– Plans- ensure – adequate time and resources allocated for testing task
– Testing can be monitored & managed
– Test planning – coordinated with project planning
– Testers can’t plan to test a component on a given date until it is
available
– Test risks – evaluated
– Probable delay in delivery of software component
– Identifying the complex and difficult components to test
– Whether adequate training on new tools
– Careful test planning avoids
• wasteful throwaway & unproductive tests
• Unplanned testcycle(test-patch-retest)
12/08/2019 10
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE

11. 10. Testing activities should be integrated into the software development
lifecycle
– Explain –for each phase
11. Testing is creative & challenging for the tester
Challenges
– Comprehensive knowledge of the software engg discipline
– Knowledge from experience& education – how software is specified, designed
& developed
– Able to manage many details
– Knowledge of fault type& it’s location
– Reason like scientist – propose hypothesis
– Familiarity with a problem domain of the software
– Create & document test cases
– Design & record procedures
– Plan for testing & allocate proper resources
– Execute tests &record results
– Analyse results
– Learn to use new tools
– Education & training
– Cooperation with requirement engineers, designers & developers
12/08/2019 11
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE

12. 1. Testing is the process of exercising a software component using a selected set of
testcases to reveal defects and to evaluate quality
2. When the test objective is to detect defects, then a good test case is one that has a high
probability of revealing yet undetected defects
3. Test results should be inspected meticulously
4. Test case must contain the expected output or result
5. Test cases should be developed for both valid & invalid input conditions
6. The probability of the existence of additional defects in a software component is
proportional to the number of defects already detected in that component
7. Testing should be carried out by a group that is independent of the development group
8. Tests must be repeatable and reusable
9. Testing should be planned
10. Testing activities should be integrated into the software development lifecycle
11. Testing is creative & challenging for the tester
12/08/2019
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE
12

13. Tester’s role in software development
organization
• tester Vs Developer –their role
• tester Vs requirements engineers - system& acceptance
test
• tester Vs designer – integration & unit test
• tester Vs project manager – test plan
• tester Vs upper management
• tester Vs SQA people
• TMM level 1, 2 – no independent group
• TMM level 3, 4 – Independent testing group
• Need for support of management
12/08/2019
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE
13

14. DEFECTS
• Origins of defects :
Effect of defects –quality
1. Education – Eg:order of precedence operators
2. Communication – interfacing modules – error
checking code absence
3. Oversight – omitted to do – omit initialization
4. Transcription – Knows what to do but makes a mistake
– misspelling a variable name
5. Process – process used by software engineer
misdirected his action – development process – no
sufficient time for detailed specification
12/08/2019
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE
14

15. 12/08/2019
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE
15

16. • Tester develops hypotheses about possible
defects. Test cases are then designed based on
hypotheses. Hypotheses are used to
• Design test cases
• Design test procedures
• Assemble test sets
• Select the testing levels
• Evaluate the results of the tests
– A successful testing experiment will prove that the
hypothesis is true. (i.e) the hypothesized defect was
present.
12/08/2019
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE
16

17. • Fault model – link between the error made and the fault / defect in
software
• It shows defects, testing and diagnosis relationship
– digital system engineers describe similar models
– link physical defects in digital components to electrical effects in the
resulting digital system
– physical defects – due to manufacturing errors, component wear out,
environmental defects
– fault models are used to generate fault list/dictionary
– from the fault dictionary
• faults can be selected
• test inputs developed for digital components
• effectiveness of test – number of faults as expressed in the model & those
revealed by test
• software engineer – not concerned with
– physical defect
– relationship between software failures and defects
– their origins – not mapped
• use the fault model concept & fault list – in memory –experience –to
test & debug
12/08/2019
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE
17

18. Creation of defect database/defect repository:
• Supports storage & retrieval of defect data from all projects in a
centrally accessible location
• Defect classification scheme- necessary for developing the defect
repository
• Repository – organized by projects – for all projects defects of each
class are logged & their frequency of occurrence, impact on
operation & useful comments
• Cataloging defects
• Supplementary information
• Staff - how utilizing
• Usage of Data
• Beizer, IEEE std, Grady, Kaner – defect classes
12/08/2019
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE
18

19. D
E
F
E
C
T
C
L
A
S
S
E
S
12/08/2019
DR.S.ROSELIN MARY, PROFESSOR &
HOD/CSE
19