This document presents an overview of independent verification and validation (IV&V) provided by Maneat. It defines verification as evaluating work products to ensure they meet requirements, while validation ensures the product meets user needs. The benefits of IV&V include improved quality, reduced costs and failures. Maneat's 5-step methodology covers the full lifecycle and priorities include understanding business flows and risks. Maneat tailors its IV&V approach to each client's needs and industry.

1. Independent
Verification & Validation
presented by Maneat
Chicago, 28 Oct 2013

2. Agenda
● Trust but Verify, Verify
but also Validate
● Definitions
● Benefits of IV&V
● Objective of
Methodology
● Methodology/phases
● Independence of Review
● Priorities in System
Testing
● Levels of System Testing
● Covering the full life
cycle
● Maneat at work in IV&V
● Maneat’s approach to
IV&V
● Contacts
Page 02

3. Trust but Verify, Verify but also Validate
● It is entirely possible that a product passes when
verified but fails when validated. This can happen when,
say, a product is built as per the specifications but the
specifications themselves fail to address the user’s
needs.
● The terms ‘Verification‘ and ‘Validation‘ are
frequently used in the software testing world but the
meaning of those terms are mostly vague and debatable.
You will encounter (or have encountered) all kinds of
usage and interpretations of those terms, and it is our
humble attempt here to distinguish between them as
clearly as possible.
Page 03

4. Definitions
Criteria
Verification
Validation
Definition
The process of evaluating work-products (not the
actual final product) of a development phase to
determine whether they meet the specified
requirements for that phase.
The process of evaluating software during or at the end
of the development process to determine whether it
satisfies specified business requirements.
Objective
To ensure that the product is being built according
to the requirements and design specifications. In
other words, to ensure that work products meet
their specified requirements.
To ensure that the product actually meets the user’s
needs, and that the specifications were correct in the
first place. In other words, to demonstrate that the
product fulfills its intended use when placed in its
intended environment.
Question
Are we building the product right?
Are we building the right product?
Evaluation
Items
Plans, Requirement Specs, Design Specs, Code, Test
Cases
The actual product/software.
Activities
Reviews
Walkthroughs
Inspections
Testing
Page 04

5. Benefits of IV&V
●
●
●
●
●
Better quality of software
IV&V increases the usability and reliability
Decreased failure rates
Fewer corrective actions
It also reduces long term costs by making it easier and
less costly to reliably modify software and re-validate
software changes (less Application Maintenance &
Support)
● It improves the project schedule because problems are
detected before the User’s Acceptance test
● It decreases the project costs because IV&V can be
performed by smaller, less expensing companies than
the large ones appointed for the “big” project thing
Page 05

6. Objective of Methodology
The objective of the Methodology is to provide a framework
for developing system tests that cover
● typical business flows
AND ARE
● well-documented
● repeatable
● data-dimensional
Page 06

7. Methodology/phases
Phase
Planning
Contents
●
●
●
Planning of ISVV Activities
System Criticality Analysis: Identification of Critical Components through a set of RAMS (*)
activities (Value for Money)
Selection of the appropriate Methods and Tools
Requirements
Verification
●
Verification for: Completeness, Correctness, Testability
Design
Verification
●
●
●
Design adequacy and conformance to Software Requirements and Interfaces
Internal and External Consistency
Verification of Feasibility and Maintenance
Code
Verification
●
●
●
Verification for: Completeness, Correctness, Consistency
Code Metrics Analysis
Coding Standards Compliance Verification
Validation
●
●
Identification of unstable components/functionalities
Validation focused on Error-Handling: complementary (not concurrent!) validation regarding
the one performed by the Development team (More for the Money, More for the Time)
Compliance with Software and System Requirements
Black box testing and White box testing techniques
Experience based techniques
●
●
●
(*) RAMS: Reliability, Availability, Maintainability, Safety
Page 07

8. Independence of Review
● U.S. Department Of Health and Human Services, in the
document General Principles of Software Validation,
states the following:
○ Validation activities should be conducted using the
basic quality assurance precept of “independence of
review”
○ Self-validation is extremely difficult
Page 08

9. Priorities in System Testing
1.
2.
3.
4.
Understanding the Business Flow is key
Understanding Risks and Risk Mitigation
Make it repetitive
Testing the system’s capabilities is more important than
testing its components
5. Testing old capabilities is more important than testing
new capabilities
6. Testing typical situations is more important than testing
unlikely situations
7. Have in mind the overall architecture
Page 09

10. Levels of System Testing
No. of Test Cases
that would need to
be executed during
System Testing to
achieve different
levels of coverage.
Where to set the
bar depends on
application and
Client’s needs
Page 10

11. Covering the full life cycle (V-diagram)
Page 11

12. Maneat at work in IV&V
● Maneat uses a 5 Step Methodology for IV&V
● We conform to Best Practices, relevant international
standards, and comply with IEEE 1012
● We train our consultants before Client’s engagement
● We encourage our IV&V teams using the client’s
methodology in order to better fit within their native
environment
● Our Management Process ensures that Methodology is
followed correctly, completely, on-schedule, and within
budget
● We produce regular Activity/Task reports that
document management awareness and controls
Page 12

13. Maneat approach to IV&V
● Each Client has different needs, so we prefer to listen
each need and, applying the principles, tailoring the
work to be done, and defining together with our Client:
○ Priorities
○ Levels of System Testing
○ Coverage
● Industry focus: each industry has different needs also
○ Aerospace must require an IV&V approach fully
compliant with IEEE 1012. Maneat is capable to fulfill
this kind of requirement.
○ Automotive, using SAP as an example, may want an
efficient and effective approach to IV&V, keeping the
overall process at the proper/adequate level, but inspired
by concepts of simplicity and moderate spending. Maneat
can satisfy this need as well.
Page 13

14. Maneat approach to IV&V (cont.)
● Having an overall understanding of the entire
architecture and how it fits with day-by-day end-user
activities
● Complementing and challenging the job done by the
System Integrator ensuring independence of review
● Offering mix of Local and Offshoring (Italy) IV&V
activities to reduce costs
● Partnering with single focus companies when necessary
Page 14

15. Contacts
www.maneatusa.com
info@maneat.it
150 N Michigan Ave, Chicago, IL 60601
Phone +1 (312) 216-5160
Page 15