Scrum is an iterative, incremental framework for delivering business value. It is not a Verification and Validation (V&V) approach. So how do we merge Scrum and V&V when a product must be subjected to formal V&V activities? How do we plan V&V work, incorporating it into a Scrum roadmap and backlog? How do we execute the V&V plan while performing development activities? Julie Bouchard and Kathy Aragon briefly describe what V&V is—and what it isn’t. They introduce V&V Navigator, a Government-developed, web-based tool to aid in identifying candidate V&V activities. Julie and Kathy demonstrate the use of Navigator to plan activities and artifacts for V&V, show how to map V&V activities into a Scrum backlog, and explore how to bake V&V into epics and stories “done” criteria. Learn ways to integrate V&V within the Scrum development process—the same as we do testing activities.

1. Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly owned subsidiary of Lockheed Martin
Corporation, for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000. SAND NO. 2011-XXXXP
V&V and Scrum
V&V has to be Agile too!

2. Agenda
 Introduction
 How to Fold V&V into Scrum
 Demo of V&V Navigator
 Currently Government Use tool
2

3. Introduction
 Julie
 V&V (software and M&S)
 Analyst, Model Developer
 Kathy
 Certified PO, Certified SM, Scaled Agile Framework Agilist)
 Software Development, V&V
 Sandia National Laboratories
 Albuquerque, NM
3

4. Introduction
Why we are doing this?
 We are passionate about Agile and Scrum.
 We are passionate about V&V and especially M&S V&V.
 It is a bad idea to develop using Scrum and then throw V&V in
at the end.
 We want the application of V&V, which is traditionally very
“waterfall-ish” to be agile.
4

5. What V&V Is and Isn’t
IS
 Required for development of systems,
software, and models
 Based on best engineering practices
 Iterative – Plan, Act, Assess
ISN’T
 Prescriptive
 One size fits all
 Easy
5
Our definitions
 Verification – ensuring that the software is implemented correctly
 Validation – ensuring that model assumptions, theory, algorithms
and results are correct/reasonable
Note: This talk isn’t a forum for argument about the Vs.
Likewise, we won’t be explaining Scrum terms.

6. So… How do we make V&V Agile?
 Plan Development Work
 Plan V&V Work
 Merge both plans
 Into Scrum Roadmap, Epics & Stories
 Execute both incrementally
 Perform development activities
 Create development artifacts, e.g. code, documentation
 Perform V&V activities
 Create V&V artifacts and evidence
6

7. So… How do we make V&V Agile?
 Merge Scrum and V&V planning and execution
 Use an approach to merge the two
 Approach 1 – Define V&V planning/execution as
stories in the product backlog
 Approach 2 – Define V&V planning/execution as tasks in stories
 Combine approaches 1 & 2
 Other approaches
We will focus on approaches 1 & 2 in this presentation.
7

8. How do we merge Scrum and V&V?
 Approach #1
 Add V&V planning work as Epics & Stories
 Add V&V activities as Epics & Stories
 Backlog has embedded V&V plans and
activities
8

9. Approach #1
Overview
 Identify V&V Activities
 Use a tool (V&V Navigator) or
other analysis
 “Front-plan” V&V Epics, as
needed
 Approach V&V planning
incrementally
9
Example questions from
Navigator:
Activity: Review Design
• Is design consistent and
implementable?
• Are requirements
traceable to the design
and adequately
addressed by the
design?

10. Approach #1
Overview
 Add V&V activities to the Roadmap as Epics
 Will be Epics for “Do It” e.g., Data Planning
 Will be Epics for “V&V” e.g., V&V Data Collection Plan
 Add Stories for V&V work
 Either in a separate V&V BL or merge with Product BL
 Stories for “Do It” and “V&V”
10

11. Approach #1 - Example
Select Stories for a Sprint
11
Story: Design Collection for
accumulation of Results.
Story: Develop potential new
Use Case.
Sprint Backlog
Story: Design V&V approach
for Input and Results Data
Epic Backlog
V&V of Data Analysis & Processing
Planning for Data
Use/Needs/Collection
Design Database
V&V of Data Collection Plan
Requirements and Use Cases
Other Epics
V&V of Data Verification Plan
User Interface Implementation
Story: Implement Error Path
for User Interface.
Legend
V&V stories
Other stories

12. Approach #1 – Example Roadmap
12
Epic Sprint 1 Sprint 2 Sprint 3 Sprint 4 to 6 Quarter 2 Quarter 3 Quarter 4
V&V of Data Analysis & Processing
Design V&V
Approach for
Input and
Results Data Story 6 Story 9
Adopt V&V
Approach for
Input Data Story3
Adopt V&V
Approach for
Results Data Story 7
Story 8
V&V of Data Characterization Plan Story 1
Design Database Story 1
User Interface Implementation Story 1 Story 2 Story 3-5
V&V of Data Collection Plan Story 1 Story 2
Requirements and Use Cases
Develop
Potential New
Use Case
Planning for Data Use/Needs/Collection
Story 1 Story 2-4
Legend
V&V stories
Other stories

13. Approach #1 – Another Example
Tie V&V activities to Development Activities
all in one combined EPIC
Application: Develop Schedule Optimization Model
V&V approach: V&V Owner determines that V&V will include
Expert Review of Optimization Algorithm
Integration testing
Epic: Develop Schedule Optimization Model
Story – Select Optimization Algorithm (Dev)
Story – Prototype Optimization Algorithm - noncomputational model (Dev)
Story – Expert Review of Optimization Algorithm (V&V)
Story – Develop Code for Optimization Algorithm (Dev)
Story – Integration Test Optimization Algorithm Code (V&V)
13

14. How do we merge Scrum and V&V?
 Approach #2
 Embed V&V planning into Story Acceptance Criteria
 Embed V&V activities into Story Tasks
 Stories have embedded
 V&V plans, activities, and Acceptance Criteria
14

15. Approach #2
Overview
 As part of Planning, Identify the V&V Activities
 Include each V&V Activity as a task in a Story
 e.g., Peer Review of Design (task) would be included in Design (story)
 Drive V&V activities via Story Acceptance Criteria
15

16. Approach #2 - Example
Acceptance Criteria In Stories
16
Story #: Design Collection for accumulation of Results
Acceptance Criteria: Design Checked-in
V&V Criteria: Peer Review of Design Complete
V&V Owner Review
Story #: Meeting to discuss potential new Use Case
Acceptance Criteria: Meeting held, notes distributed and
in repository.
V&V Criteria: none
Epic Backlog
Planning for Data
Use/Needs/Collection
Design Database
Requirements and Use Cases
Other Epics
User Interface Implementation
Story #: Implement Error Path for User Interface
Acceptance Criteria: Implementation Checked-in; User Guide
Updated
V&V Criteria: Unit Test Pass, Regression Tests Pass, Test Plan
Updated

17. Approach #2 - Acceptance
• Product Owner
 Accepts completion of story according to
the Acceptance Criteria when V&V tasks
are included
 Accepts completion of story including V&V
Criteria met when V&V Criteria is included
as part of the Acceptance Criteria
17

18. Considerations For Each Approach
 Approach #1 – V&V embedded in the BL as stories
 V&V stories are visible
 PO is responsible for V&V stories
 Story Acceptance Criteria is explicit for V&V activities
 V&V activities could be overlooked due to there not being an enforced
1:1 relationship between development and V&V
 Approach #2 – V&V embedded in story tasks
 Team is responsible for tasking V&V activities
 V&V is directly related to development activities
 Coordination/effort required to embed the right activities as tasks in a
story
 V&V tasks could become box-checking-exercise instead of a useful task.
18

19. Consider a V&V Champion
 Identify a person responsible for
 Ensuring V&V is addressed throughout development
 Understanding how to merge Scrum and V&V
 Assisting with selection of V&V methods (V&V Navigator lives here)
 The V&V Champion can be the Product Owner
 The roles require different skill sets
 V&V could be driven by the organization, not the customer
 A designated V&V Champion is especially important with
Approach 2 to ensure that V&V isn’t overlooked
19

20. V&V Navigator
 Assists planning for V&V of Modeling and Simulation
(M&S)
 Any portion of the model includes theory or
unknown science
 Some output results are intentionally variable (e.g.,
optimization, prediction of future)
 Where all answers/outputs are not known or agreed
upon
 The Process Step flow identifies what to plan for and
what to do

21. 21
Demonstration
V&V Navigator v. 2.0 can be distributed for Government Use.
Navigator

22. Final Thoughts
 Plan Development Work
 Plan V&V
 Combine both plans
 Into Scrum Roadmap, Epics & Stories
 Execute plans as per Roadmap
 Perform development activities
 Perform V&V activities
 Assess
Consider involving a V&V Champion!
22

23. QUESTIONS?
23

24. Contact Information
Julie Bouchard, jfbouch@sandia.gov
Kathy Aragon, kmarago@sandia.gov
24