2. Course Outline:
Introduction to software quality assurance,
The Quality Challenge,
Quality Control v/s Quality Assurance,
Quality Assurance in Software Projects (Phases),
Principles and Practices,
Quality Management,
Quality Assurance and Standards,
Quality Planning and Quality Control,
Verification and Validation,
Planning Verification and Validation,
Critical System Validation,
Reliability Validation,
Safety Assurance,
Security assessment,
Inspections and reviews,
Principles of software validation,
Software verification,
Planning for Software Quality Assurance,
Software Quality Assurance (SQA) Plans,
SQA-Organizational Level Initiatives,
SQA Planning (Observations, Numbers, Results),
Software Testing, Specification based test construction techniques,
White-box and grey-box testing,
Others comprehensive software testing techniques for SDLC,
Control flow oriented test construction techniques,
Data flow oriented test construction techniques,
Clean-room approach to quality assurance,
Product Quality and Process Quality,
Standards for process quality and standards for product quality,
Walkthroughs and Inspections, Structure, Checklist, Audits, Roles and Responsibilities (Reviews, Inspections,
etc),
How to make Reviews and Inspections most effective.
3. Books
Fundamentals of Software Testing, Bernard Hom, Wiley,
2012, (or Latest Edition).
“Software Quality Assurance: Principles and Practice”, Nina
S. Godbole, Alpha Science, 2004 (or Latest Edition).
Software Quality Engineering: Testing, Quality Assurance,
and Quantifiable Improvement, Jeff Tian, John-Wiley &
sons, 2005 (or Latest Edition).
Software Testing in the Real World: Improving the Process,
Kit, Edward, Addison-Wesley, 1998 (or Latest Edition).
Perfect Software: And other illusions about testing, Gerald
M. Weinberg, Dorest House, 2008 (or Latest Edition).
5. Learning Objectives
1. The student will be able to define both verification and
validation, and to know the difference between them.
2. The student will be able to identify the appropriate activities
and their classification as either verification or validation.
3. The student will be able to distinguish between verification
and validation, given a set of activities.
4. Given a set of software artifacts, the student will understand
the appropriate verification or validation activity to that
artifact.
5. Given a set of projects in different categories, the student
will be able to determine which category of projects would
require more V&V activity, and which would require less.
6. Given a set of defects and the phases of development in
which they were found, the student will be able to estimate
the relative cost of correcting errors early or late in the
lifecycle.
6. Why Invest in Quality?
Cost effective
Provides competitive edge
Essential for business survival
Essential for international marketing
Helps to retain customers, and
increase profits
Hallmark of world class business
7. Ways to Improve Quality
Prevention of Defects
• Process Improvement
• Complexity Reduction
• Risk Management
• Causal Analysis
Detection and Correction of Defects
• Verification
• Validation
• Rework
• Causal Analysis
8. Verification
Verification – Are we building the product
right?
Verification is any checking process
conducted on software artifacts in an
attempt to determine if they work as
specified by the designers of the system.
Includes reviews, inspections,
walkthroughs, unit testing and integration
testing.
9. Validation
Validation – Are we building the right
product?
Validation is the process of evaluating
software artifacts during the software
development process in an attempt to
determine if the system works as required
by the customers. Any evaluation activity
that involves the customer can be used for
validation purposes.
Includes program reviews, system testing,
customer acceptance testing.
10. Verification vs. Validation
Verification
• Main purpose is to detect defects in the
artifacts of the system under
development.
Validation
• Main purpose is to show that the system
under development meets user needs,
requirements, and expectations.
11. Verification & Validation Techniques
Static Methods
• Techniques applied to artifacts without
execution.
Dynamic Methods
• Techniques applied to artifacts through
execution.
Mathematically Based Methods
13. Effectiveness of Static Verification
More than 60% of program defects can be
detected by program inspections.
More than 90% of program defects may be
detectable using more rigorous
mathematical program verification.
The defect detection process is not
confused by the existence of previous
defects.
14. Dynamic: Testing (Verification)
Unit Test (Detailed Design):
• Testing the individual software modules,
components, or units.
Integration Testing (Architectural Design):
• After unit test, the system is put together in
increments. Integration testing focuses on the
interfaces between software components (OO
thread-based, cluster-based testing)
System Testing (Requirements Spec):
• One goal of system testing is to ensure that
the system functions as specified in the
specification.
15. Dynamic: Testing (Validation)
System Testing (Requirements Spec):
• Another goal of system testing is to ensure
that the system functions as the client
expected in a controlled environment.
User Acceptance Test (ConOps):
• A set of formal tests run for the client, and
specified by the client. When the system
passes these tests, the software has been
accepted by the client as meeting the
requirements.
16. Verification and Validation in the
Development Lifecycle
Code
Execute Unit
Tests
White-Box &
Black-Box
Testing
Black-Box
Testing
Requirements
Analysis
Execute
System Tests
Execute
Integration
Tests
Design
Validate the System
Verify Design
Verify Implementation
17. Mathematics-Based Verification
Verification is based on mathematical
arguments which demonstrate that a
program is consistent with its
specification.
Programming language semantics
must be formally defined.
The program must be formally
specified.
18. Why Inspection is important?
Relative cost to fix a defect
Phase in which found Cost ratio
(hours)
– Requirement 1
– Design 3-6
– Coding 10
– Testing 15-70
– Operation 40-1000
Data derived from Capers Jones.
19. Costs of Finding and Fixing Late
Delays in identifying and fixing
defects gets geometrically more
expensive as the lifecycle progresses!
$1
$10
$100
$1000
Cost
of
Fixing
a
Defect
Defects
Time in (Phase of) Development
20. Cost of Quality
Includes all costs of quality-related
activities.
Quality costs =
+Prevention costs
+Detection and Appraisal costs
+Failure costs
• Internal failure costs
• External failure costs
21. Quality Cost Components
Direct Cost
– Reviews/inspections
– Unit testing
– System testing
– Acceptance testing
– Test planning and
design
– Computer time
– Resources (terminals,
staffs, etc.)
Indirect Cost
– Rework
– Recovery
– Corrective action cost
– Failures
– Analysis meeting
– Debugging
– Retesting
– Legal fees
22. Verification or… Validation?
Reviews Either
Unit testing Verification
Integration Testing Verification
System testing Validation
Acceptance testing Validation
23. Balancing Trade-Offs
What is the real requirement?
What are the expected benefits of
V & V vs.
– cost of increasing V&V activity
– additional time required
– technical feasibility
– risk of not finding defects
» delivery
» operation
“When is it good enough?”
(Bach, 1997)
24. Summary
Verification and Validation are both
important to the quality of software.
Each are different:
• Verification determines that what we are doing
is done correctly.
• Validation determines whether we are doing
the right thing.
Sometimes, less quality is okay. Some
software systems require less V&V, as the
cost is not supportable. Some require
more no matter what it costs.
The earlier defects are discovered, the less
it will cost to correct them.
27. Static and Dynamic
Verification & Validation
Formal
specification
High-level
design
Requirements
specification
Detailed
design
Program
Prototype
Dynamic
validation
Static
verification
Sommerville, p. 421
28. Test Case Development in the
V Model
Code
Execute Unit
Tests
Test cases designed and
executed by developers
Test cases designed
and executed by
independent testers
Requirements
Analysis
Execute
System Tests
Execute
Integration
Tests
Design
Validate Requirements
Verify Design
Verify Implementation
