This unit covers introduction to software quality, verification, validation and testing, measuring software quality factors, testing techniques, and formal technical reviews.

1. Arab Open University
2nd
Semester, 2006-2007
M301
Unit 5.3
Product Quality: Metrics,
Verification, Validation, Testing
reem.attas@arabou.org.sa

2. Reem AlAttas © 2
Topic Road Map
 Introduction to software quality
Verification, validation and testing
Measuring software quality factors
Testing techniques
Formal technical reviews

3. Reem AlAttas © 3
What is software quality?
Software that is fit for its purpose and is of
sufficiently high quality is said to be of
appropriate quality, i.e. in conformance to
the requirements and expectations of the
customer.

4. Reem AlAttas © 4
Requirements
 There are three general types of requirements
that have an effect on software product quality:
 Product operation requirements how the product will
be used by the end user?
 Product revision requirements how the product will be
changed?
 Product transition requirements how the product will
be modified for different operating environments?

5. Reem AlAttas © 5
Product operation requirements
1. Correctness: that attribute which determines how well a
system fulfils the customer’s overall objectives;
2. Reliability: that attribute which determines system can be
expected to perform its intended function;
3. Efficiency: that attribute which determines the level of
computing resources required perform its function;
4. Integrity: that attribute which determines how well the data
is secured;
5. Usability: that attribute which determines how easy the
system is to use.

6. Reem AlAttas © 6
Product revision requirements
1. Maintainability: that attribute which determines
how easily bugs can be found and fixed.
2. Flexibility: that attribute which determines the
effort required to modify an operational system.
3. Testability: that attribute which determines how
easily the system can be tested to show that the
customer’s requirements have been met.

7. Reem AlAttas © 7
Product transition requirements
1. Portability: that attribute which determines how
easily the system can be used on another machine
2. Reusability: that attribute which determines how
easy it is to reuse some of the software to make
future developments more cost-effective
3. Interoperability: that attribute which determines the
effort required to couple one system to another.

8. Reem AlAttas © 8
Primary Software Quality Factors
(SQFs)
1. Correctness.
2. Integrity.
3. Maintainability.
4. Usability.

9. Reem AlAttas © 9
Correctness Measure
The popular measure for assessing
correctness is defects per thousand lines of
code (defects per KLOC)
Defect (bug): verified lack of conformance to
requirements.

10. Reem AlAttas © 10
Integrity Measure
Integrity is measured by considering the
proportion of ‘attacks’ on a product as
opposed to bona fide uses.

11. Reem AlAttas © 11
Example

12. Reem AlAttas © 12
Topic Road Map
Introduction to software quality
 Verification, validation and testing
Measuring software quality factors
Testing techniques
Formal technical reviews

13. Reem AlAttas © 13
Verification
The process of checking that each system
description is selfconsistent, and that
different system descriptions are
consistent and complete with respect to
each other.

14. Reem AlAttas © 14
Validation
The process of checking that each system
description is consistent with the
customer’s requirements.

15. Reem AlAttas © 15
Consistency
Two system descriptions are consistent
simply means that, where they describe
the same part of the system, they do not
contradict each other.
Consistency with a customer requirement
means that the requirement is met.

16. Reem AlAttas © 16
Completeness
Every aspect of the customer’s
requirements must be met by the system
description.
More difficult task than showing
consistency.

17. Reem AlAttas © 17
Testing
The process of executing a software
system, or part of a software system, in
order to check that it meets its
requirements.
It is any form of validation or verification
that operates on program code.

18. Reem AlAttas © 18
Test Procedure
Select some test data.
Predict the results that should be
expected.
Check whether the software conforms to
our expectations.

19. Reem AlAttas © 19
Role of Testing
Help finding and fixing bugs.
Improve the customer’s confidence in a
software product.
Improve the testing process itself.
 A cost–benefit analysis of ‘bugs found’ versus
‘time taken’ can be performed to determine
when testing should stop.

20. Reem AlAttas © 20
Types of Testing
1. Usability testing.
2. Developmental testing: checks that developmental
activities have been carried out correctly.
3. Requirements-based testing: checks that a system
meets the customer’s requirements.
4. Regression testing: occurs during developmental
testing and system maintenance, and checks that fixing
one bug has not introduced others.

21. Reem AlAttas © 21
Developmental testing stages
 Unit testing, in which units of functionality (e.g.
The classes in an object-oriented system) are
tested in isolation.
 Integration testing, in which previously unit-
tested units are tested together.
 System testing, in which the completed system
is tested against the customer’s requirements.

22. Reem AlAttas © 22
Requirements testing stage
Acceptance testing, which is performed
by the customer, and after which (all being
well) the system is accepted as complete.

23. Reem AlAttas © 23
Topic Road Map
Introduction to software quality
Verification, validation and testing
 Measuring software quality factors
Testing techniques
Formal technical reviews

24. Reem AlAttas © 24
Measuring Quality
 The quality of a software product
can be measured by obtaining
values for the eleven software
quality factors (SQFs)
 Obtaining such values is not a
trivial task, as there is no
obvious way of evaluating a
product against these factors.
 What we must do is to find
properties of software products
which are measurable, and use
these to infer the values for the
SQFs.
1. Correctness
2. Reliability
3. Efficiency
4. Integrity
5. Usability
6. Maintainability
7. Flexibility
8. Testability
9. Portability
10. Reusability
11. Interoperability

25. Reem AlAttas © 25
Software Metrics
1. Accuracy
2. Auditability
3. Communication commonality
4. Completeness
5. Complexity
6. Conciseness
7. Consistency
8. Data commonality
9. Execution efficiency
10. Expandability
11. Generality
12. Hardware independence
13. Instrumentation
14. Modularity
15. Operability
16. Robustness
17. Security
18. Self-documentation
19. System independence
20. Traceability
21. Training

26. Reem AlAttas © 26
The Dependence of the SQFs on the
Metrics

27. Reem AlAttas © 27
Ex. The Relationship Between
Reliability and the Metrics
 F  value for the SQF, reliability.
 m  measured values for the metrics which relate to the SQF,
reliability.
 c  weightings indicating the relative importance of the metrics with
respect to the SQF, reliability.

28. Reem AlAttas © 28
Measuring System Complexity
1. Lines-of-code Metric: measures the
complexity of a method by counting the
number of lines-of-code.
 More LOC  more errors.
 Should comment lines be included in the line
count?

29. Reem AlAttas © 29
… Measuring System Complexity
2. McCabe’s Cyclomatic-complexity Metric: measures the
complexity of a method by counting the number of independent
paths through a method body.
 A method body starts and completes its execution defines one
independent path.
 The number of independent paths through a piece of code can be
calculated by counting the number of decision points.
 Each if statement adds a decision point.
 Each while, do-while and for loop adds a decision point.
 Each switch statement adds one decision point for each of the cases it
tests for, excluding any default case.
 Each try statement adds one decision point per catch block, but any
finally block is not a decision point, as it will always be executed.
 The operators && and || each add a decision point.

30. Reem AlAttas © 30
… Measuring System Complexity
A cyclomatic-complexity of ten or more
should be a hint that a method body
should be restructured into two or more
simpler methods.

31. Reem AlAttas © 31
Modern Object-oriented Metrics
 Depth of inheritance tree (DIT): its number of ancestors.
 Coupling between objects (CBO): the number of has-a
relationships the class has with other classes.
 Number of children (NOC): the number of children for that class.
 Response for a class (RFAC): the size of the response set for the
class, which consists of all the methods of that class together with
all the methods of other classes called by those methods.
 Lack of cohesion in methods (LCOM): its cohesiveness.
 Weighted methods per class (WMPC): its complexity of behavior
 the sum of the cyclomatic complexities of each method of the
class.

32. Reem AlAttas © 32
Topic Road Map
Introduction to software quality
Verification, validation and testing
Measuring software quality factors
 Testing techniques
Formal technical reviews

33. Reem AlAttas © 33
Testing Techniques
Black-box testing we choose test cases
by looking at the specification (i.e.
requirements and high-level design) of the
system to be tested.
White-box testing we choose test cases
by looking at the detail of the
implementation of the system to be tested.

34. Reem AlAttas © 34
A strategy for black-box testing (using
partitioning)
1. For each method in the class, determine
the input data space.
2. Partition the input data space into sub-
domains.
3. Test all sub-domains given by the case
analysis.

35. Reem AlAttas © 35
Step 1

36. Reem AlAttas © 36
Step 2

37. Reem AlAttas © 37
Step 3

38. Reem AlAttas © 38
White-box Testing Techniques
Basis-path testing.
Loop testing.

39. Reem AlAttas © 39
Basis-path testing
 Ensures that all reachable statements in a
method are tested at least once.
 Select data which will exercise the straight-through path,
i.e., loop and if conditions evaluate to false, and only
default cases selected in switch statements.
 Then, find data that deviates from the straight-through
path at the first decision point.
 The process is then repeated for each subsequent
decision point in the program, varying the flow in each
case.

40. Reem AlAttas © 40
Loop testing
Used to test loop constructs.
In Java, there are three ways in which
loops can occur:
 Simple loops.
 Nested loops.
 Concatenated loops.

41. Reem AlAttas © 41
Simple loops forms
 Test data should be found for the following cases:
 the loop is skipped entirely
 the loop is passed through exactly once
 the loop is passed through more than once, maximum n times
 m times, where 3 < m < n – 2
 pass through the loop n – 1, n and n + 1 times.

42. Reem AlAttas © 42
Nested loops form
 Approach:
 Start with the innermost loop.
 Work outwards.
 Repeat step 2 until all loops are tested.

43. Reem AlAttas © 43
Concatenated loops
 Concatenated loops can be tested separately
using sequential simple loop tests

44. Reem AlAttas © 44
Topic Road Map
Introduction to software quality
Verification, validation and testing
Measuring software quality factors
Testing techniques
 Formal technical reviews

45. Reem AlAttas © 45
Inspections (Formal Technical
Reviews)
A formal, efficient, and economical method
of finding errors in design and code.

46. TMA5 – Q3

47. Thank You!