This document discusses improving quality when partnering with customers. It describes a case study where phase 1 of a project went poorly, experiencing delays, cost overruns, and defects due to incomplete requirements and lack of transparency. Phase 2 applied lessons learned with improved requirements gathering, testing processes, and communication. These changes strengthened the partnership and led to on-time delivery with few defects. The key is being open about challenges, willing to modify processes, understanding the client's metrics and culture, and having oversight to avoid getting lost in details.

1. IMPROVING QUALITY BY PARTNERING WITH YOUR
CUSTOMERS:
A “HOW TO” STORY
Armand Spoto
Director, Professional Services
 May 23, 2016

2. AgendaBackground
Overview
Agile Development vs Delivery
Case Study (Phase 1)
Interactive Exercise
Case Study (Phase 2)
Summary

3. Background
20 years of experience in professional services industry
Provided strategy, project management, and
implementation of large scale programs for a number of
Fortune 500 companies
10 years providing Training and Development solutions to
the Life Science Industry
Full time Project Management and PMO responsibility for
past 7 years
Scrum master since 2014
M.S. in Industrial/Organizational Psychology in 1997 from
Radford University.

4. Who Are We?
~400 employees across three Continents
~175 P.S. employees
Project Management
Business Analysis, Insurance Domain
.Net Development
QA/Testing
80+ Carrier, Financial Inst. and Distribution
Implementations
Average 20 Deployments Weekly
Key Partners

5. SDLC
Implementation Phase Initiation Phase

6. Phase 3
Agile Development / Agile Delivery
Customer Success – Rollout Plan/Producer On Boarding/Training
Rollout Plan / Producer on Boarding/ Training
SSO –SAML
E-Submission
Other Integrations
Development
Initiation Phase
Workflow
Screen Flow
Forms matrix
Business Rules
Product Specs
Screen Mockups
Draft
Specification.
SSO, Submission,
e-Signature
Test Strategy
Phase 1

CustomerCustomer
            
LiveLive
iPipelineActivityiPipelineActivity
Common Screens
Parties, Payments
Term Life
E-app Screens
E-Signature
Phase 2
Integrations
E-Submission
IntegrationTest
Initiation Phase
Workflow
Screen Flow
Forms matrix
Business Rules
Screen Mockups
Draft
Specification.
SSO, Submission,
e-Signature
Test Strategy
IntegrationTest
LiveLive
IntegrationTest
LiveLive
Sprint Functional Testing

7. iPipeline Agile Implementation
Requirements clearly understood and documented
Business & Product Rules Defined in Initiation
Collaborative Planning
Release & Sprint Planning, aligning on integration
Continuous Involvement
Testing (End Users, Business and Project Team)
Project
Scoping
High
Level
Req.
Prioritization
Final
Assembly
Final
UAT
Delivery
Support
Agile
Sprint Sprint Sprint Sprint
Customer
Review
Customer
Review
Customer
Review
A
D C
T
A
D C
T
A
D C
T
A
D C
T
Customer
Review

8. Phase 1
THE PROJECT THAT WOULDN’T END

9. The Situation
Phase 1
•Build 4 Insurance Illustrations Products
• Two Term
• Two Guarantee Universal Life
•Develop and Test
• User Interface Screens
• Calculate Engine
• Crystal Reports (Output)
•Planned Start: February 2014
•Planned Finish: October 2014
•Budget: $475,000

10. Phase 1: Client Defects per Month

11. Phase 1: # of Requirement Changes

12. What is Your RCA?
Break up into small groups
•Spend ~10 minutes brainstorming what do you think went
wrong and why?
•What are impacts on:
• Team?
• Client?
• Budget?
•Ask any clarification question

13. The Lightbulb Moment

14. Phase 1
Results
•Averaged ~15 defects per sprint
•Mountain of missing/ambiguous requirements
•Actual Finish: March 2015 (six months over due)
•Actual Costs: $968,000
•Client very unhappy
•Team felt demoralized and frustrated

15. Phase 1: Project Debrief
“Lack of Transparency” was a direct cause to many of
the project’s issues
Incomplete regression test beds
No test cases/checkers
No visibility into requirements
Too much leverage of JIRA to document requirement
update and discrepancies

16. Phase 1: Project DebriefAdjusted Three Key Processes

17. Phase 2
The Redemption

18. The Situation
Phase 2
•Build 12 Insurance Illustrations Products
• Eight Indexed Universal Life
• Four Universal Life
• Client wanted to complete all Crystal Reports (Output) Work
•Created two full-teams
• 1 FTE Business Analyst
• 1 FTE User Interface Developer
• 2.5 FTE Calc Engine Developers
• 2 FTE Quality Assurance
•Planned Start: March 2015
•Planned Finish: April 2016
•Budget: $1,025,000

19. Requirements Gathering
Client’s “Business” has to sign-off on their requirements
Used to create developer Requirements, which is signed
off between both parties
Focus on the completion of a specific Feature/Function
of a specific product

20. BRD Document Walkthrough
 Functional Overview
 Requirements
 User Stories
 User Story Details
 UI Definitions (Windows, Menus,
Toolbars)
 Calculation Engine Checkers
 Report Design
 Configuration Settings
 iGO Integration Impacts
 Base/R&D Integration Impacts
 Content Impacts
Client Input Required
• Detailed Technical Design
• Code Review
• Unit Test Review
• Test Plans and Requirements Traceability
• Issues List

21. Testing Strategy
 Development begins on that signed-off set of features
 Develop to Checkers
 Execute testing based on firm test cases and checkers provided by client
 Regression is run twice per sprint
 Features are released to UAT based on QA results

22. Communication
Client attends all Agile Ceremonies
•Scrums
•Scope Overviews
•Sprint Planning
•Bi-weekly QA Meeting
•See all backlog items/burndowns in VersionOne
Additional touch-points
•Quarterly on-site visits
•Bi-weekly Executive touch-point
•Distribute 3 sprint Release plan every sprint

23. Phase 2: Client Defects per Month

24. Phase 2: # of Requirement Changes

25. Results
Strengthening the partnership
New work projects through 2017
More strategic discussions w/ Executives
Virtually eliminated all escalation issues
 Exposed ownership of issues to be unblocked
Broadened conversation to address issues
Remove blockers
Improve velocity

26. Summary
Exposing your dirty laundry elevates everyone’s game
Open to trying new processes, and modifying when
necessary
Your metrics aren’t my metrics
Someone has to be out of the weeds, or you’ll always be
lost
Client culture is key to understand how to modify your
processes