How to Verify and Validate a System or Process

HowtoVerifyandValidateaSystemorProcess
Presented by: Steven H. Dam, Ph.D., ESEP – Part Four of the “How to MBSE” webinar series.

AskUsYourQuestions
• Ask us your questions using
the panel on the right
• This presentation is being
recorded and will be made
available to you.
• Contact us after the webinar
through
• support@Innoslate.com
• Call 571.485.7800
• LinkedIn Innoslate User Group
• Twitter
Proprietary of SPEC Innovations® 05/26/2022 at 11:00 am ET, Dr. Steven Dam

MeetYourHost
• President and Founder of SPEC
Innovations
• Participated in the development of C4ISR
and the DoDAF
• Expert Systems Engineering Professionals
Certificate
• steven.dam@specinnovations.com
• @stevenhdam

WhatYouWillLearnToday
• How Different Customers View V&V
• How to Develop Your V&V Requirements, V&V Plans, and Test Suites from
the Beginning
• How to Capture and Report Test Results in Innoslate®
• How to Evaluate Processes using Sopatra®

V&VinDoD
• DoD focuses on “operational effectiveness,
operational suitability and survivability
(including cybersecurity), or lethality.”
[Defense Acquisition Guidebook (DAG),
Chapter 8]
• Organizationally, DoD divides into
Developmental Test and Evaluation (DT&E)
and Operational Test and Evaluation (OT&E)
• Development of a Test and Evaluation Master
Plan (TEMP) is considered critical
• DoD supports major test ranges and facilities,
worldwide
• An entire command focuses on
interoperability (Joint Interoperability Test
Command – JITC)

V&VinNASA
• According to NASA:
• “Product verification proves that an end product
(whether built, coded, bought, or reused) for any
element within the system structure conforms to its
requirements or specifications.”
• “While verification proves whether ‘the product was
done right,’ validation proves whether ‘the right
product was done.’”
• These definitions then map back to the DoD
approach:
• DT&E == Verification
• OT&E == Validation
• So, frequently different customers will frame
V&V in different terms and have different
processes and procedures, but the end goals
are the same
From NASA Systems Engineering Handbook

EachOrganizationWillDefineTheirOwnProcessesforCM
• Our process has many of the same
elements as the DoD and NASA
processes
• We include the use of our “SE
Knowledgebase” as a key part of our
process
• We currently use Innoslate® as our
primary SE Knowledgebase and tool
• It can ingest or capture all the work
products and may even be the
source of many of those products
• We can also use it to conduct
analyses and create any necessary
reports
SE Knowledgebase
Review Test Results with ERB
Develop Deployment Plan and
Conduct Beta Testing
Assess Risk and Plan Mitigation
Updated SE
Knowledgebase
Return Item to “Integrate Solution”
(if necessary)
Beta Test
Report
Deficiencies?
YES
NO
Workaround
Identified
Verify/Validate to Requirements
Verification/
Validation
Requirements
From Design &
Analysis Phase
Perform Regression Testing
Acceptable
Regression Tests?
YES
NO
Deployment Plan
Submit Changes to CM Process
Regression Test
Report
Inspection and Test
Report
Updated Test
Plan

“HowtoMBSE”WebinarSeries:
WhereDoesV&VFitintotheSystemLifecycle?
Documents View
Requirements View
Modeling Simulation
Risk Analysis
Timeline WBS
Design Engineering
Integrations
Test Center
Sopatra
EVERYWHERE!

HowtoDevelopYourV&VRequirements,V&VPlans,
andTestSuitesfromtheBeginning

CreateVerification/ValidationRequirementsandDocuments
• Start with Validation
Requirements
• Customer needs get translated into
implementation independent
requirements
• Determine then how you will validate
that these requirements have been
met
• May require a separate set of
requirements
• Once system requirements have
been decomposed to specification
level, create the verification
requirements as well

CreateTestPlansandTestSuites
• Test Plans can be developed
using built-in templates or
create your won
• Essential to be done early in the
lifecycle to identify long lead
items, such as test facilities and
automated test equipment (ATE)
• Preliminary Test Suites can also
be created as verification
requirements are developed
• Just about everything can be
planned well ahead of the actual
V&V execution activities
• This upfront planning ensures
better requirements and much
quicker V&V

EnsureRequirementsTraceabilityEarly
• The Traceability Matrix
provides a way to trace
requirements to Test
Cases
• The traceability assist
provides a check on the
probability that a
requirement and test
case should be
connected

IncludeVerificationMethodsforRequirements
• Verification Methods are
identified as labels in Innoslate®
• Analysis
• Demonstration
• Inspection
• Modeling & Simulation
• Test
• Additional labels can be added, if
needed by ensuring that the
“Verification Method” label is
used as a parent
• These labels are provided at the
Organization level

HowtoCaptureandReportTestResultsinInnoslate®

UseTestCenterandTestSuitestoCaptureResults
• Test Center provides a means to
define Test Suites for each major
V&V activity, using all methods
(analysis, demonstration,
inspection, simulation, test)
• Innoslate® can support the
simulation portion of V&V as well
• Data recorded in files can be
stored as Artifacts or a “File”
attribute can be added to the Test
Case class to associate files with
each Test Case

UseAlltheTestCaseAttributesandDiagrams
• Test Cases are a subclass of
Actions
• This feature enables the Test
case to be use in Action
Diagrams, Timelines, and other
project planning and execution
activities
• Test Cases have extra
attributes to capture V&V
related information
• Expected Result
• Actual Result
• Setup
• Add others as needed, e.g.
• Result Link
• File

CreateTestReportsandVerificationMatrices
• Test Reports are another
document type in
Innoslate®
• No templates are currently
available
• Add your own custom
templates
• Verification Matrices are
reports in the Document
View or can be developed
in Database View

HowtoEvaluateProcessesusingSopatra®

Sopatra®WasDesignedtoEvaluateProcessesandProcedures
• Sopatra® translates a text-based
Standard Operating Procedure
into an Action Diagram
• It uses other information (input
devices, output displays,
acronyms, and time distribution
databases) to better develop the
process diagram
• Best practice actions (Hear, See,
Decide) are automatically added
• Special metrics were defined to
identify the Available Operational
Time Window (AOTW) and Time
of Procedure (ToP) to calculate
the Procedure Buffer Time (PBT)

Sopatra®’sMonteCarloSimulatorProvidestheanalysisResults
• A well developed procedure will
show “all green”
• Any failed runs will show up in
“red”
• Some procedures may be able to
have an acceptable failure rate and
the check for that will highlight the
fail rate in red when exceeded

LiveDemo
Let’s dive on in.

PredicttheFuture!
A well calibrated simulation can do this
• Remember what we do and how we do it matters
• High-quality analyses save projects, products, systems,
and lives
• Develop high-quality simulations by using Innoslate’s
discrete event and Monte Carlo Simulators
• Reduce operational errors by using Sopatra

Questions and
Answers
Use the panel on the right to ask your questions

Next Webinar
in “How to MBSE”
“Webinar: How to Verify and Validate”
May 26, 2022, 11:00 am ET
Go To Webinar

MoreResources
SPEC Innovations offers training, books, videos, documentation, trials, and more
Training: specinnovations.com/training
Books: “Real MBSE” textbook and lab manual available on
Amazon
Videos: Visit the SPEC Innovations YouTube channel
LinkedIn: Innoslate and Systems Engineers User Group
Documentation: help.Innoslate.com
Trial: cloud.Innoslate.com

SPEC Innovations offers training, books, videos, documentation, trials, and more
Thank you!
Visit cloud.innoslate.com for a trial.
SPEC Innovations
@Innoslate
Innoslate User Group
Innoslate.com/blog
571.485.7800
innoslate.com

How to Verify and Validate a System or Process

Recommended

Recommended

More Related Content

Similar to How to Verify and Validate a System or Process

Similar to How to Verify and Validate a System or Process (20)

More from Elizabeth Steiner

More from Elizabeth Steiner (20)

Recently uploaded

Recently uploaded (20)

How to Verify and Validate a System or Process

Editor's Notes