The document summarizes the experience of using Microsoft Team Foundation Server (TFS) to drive process improvements at an organization providing healthcare services. Key points: 1) TFS was used to visualize workflows, manage flow, standardize reporting, and automate tasks to eliminate waste and improve consistency, transparency, and decision making. 2) Custom tools were built to bridge gaps in TFS reporting capabilities and provide real-time metrics on code reviews, task creation, customizations testing, and more. 3) Significant improvements resulted, including time savings, reduced variations, earlier issue detection, and empowered teams through self-service reporting and process transparency.

1. Leveraging TFS for Driving Process
Improvement using Lean Principles
Lean and Agile Learning Network – Chicago
April 28, 2015
Chandra Mohan
Srini Kadiam

2. Introductions
Mohan Ganesan
- Passionate About
• Process Improvements
• Building Automation Tools to enable Sustainable Process Improvements
Srini Kadiam
- Passionate About
• Teams and Building High-Performance Teams
• Agile Engineering & Delivery Practices Adoption

3. Overview
• Experience report in using TFS to process
improvements
• Time period: Jan, 2014 to March, 2015
• Organization: At an organization that provides
services to the healthcare providers

4. Context
- Transitioning to Scaled Agile Framework
- Team size increased by 3 times in 6 to 8
months
- Distributed Teams
- Portfolio and Program Teams in US and
Project Teams in India
- Investments in New Products
- Focus on stabilizing the existing Platform

5. Summary
Problem Statement
- No shared understanding of
workflow
- No proper tools to support Agile
processes
- Lack of measurement for the
Flow
- Release Quality is not great and
not consistent
Our Approach
- Dedicated ALM Team
- TFS as not only Version Control
Tool
- Maximize TFS Usage
- Set up TFS Test Environment
- Collaboration and Training
- Customize TFS to support
workflow
- Applying lean and Kanban
principles
- Augmenting with custom tool
Applications
- Scrum Framework
- Release Workflow
- Business Process Workflow

6. Lean and Kanban – Guiding Principles
Lean 1 Kanban 2
Eliminate waste Visualize the workflow
Amplify learning Limit Work in Progress (WIP)
Decide as late as possible Manage flow
Deliver as fast as possible Make process policies explicit
Empower the team Improve collaboratively
Build integrity in
See the whole
1. Lean Software Development - http://en.wikipedia.org/wiki/Lean_software_development
2. Kanban Development: http://en.wikipedia.org/wiki/Kanban_(development)
Green: Focus Areas

7. Visualize - SAFe
Visibility at Portfolio and Program Level. Support for SAFe

8. Visualize Process – ALM, Take 1
• ALM in Kanban board
• Customizable board per team
• Great for small team
• Limited support for complicated
workflow
• Limited support for multi-team
workflow visualization
• We FAILED
• We pivoted our approach

9. Visualize Process – ALM Team Collaboration
ALM team Collaborated with ScrumMasters
- Workshop for training on TFS
- Understand each team’s specific needs
- Learning from each other
- Requirements for standardized reporting
- Empower ScrumMaster as champions for driving improvements
- Building the backlog
Standardized Visual Reporting (no customizations)
- Blockers / Impediments
- WIP / completed tasks
- Reporting on daily scrum standup questions
Result
- All teams are on the same page
- Standardization and removing variances

10. Visualize Entire Process
Meeting Operational, Support and other dev teams
to
- Understand their workflow
- Educate on TFS
- Migrated version control to TFS
- Created a workflow for their teams
Customize Workflow / Business Process for each
team
- Adding custom fields
- Explicit policies for each state
We have most of the teams using TFS now

11. Manage the Flow
Portals For Each Team
- Information radiators
- Collaboration and communication
- Brings team together
- Explicit rules for each state
Entire ALM is Managed Through TFS
- Real-time data. No emails
- Better metrics
- Team has data and time to make the right
decisions

12. Building on TFS – Bridging the Gap
- Limited and rigid reporting capability
in TFS
- Custom Tool to report and automate
manual tasks
- Features of this tool
• Self-serviced
• Real-time
• All data is from TFS

13. Benefits – Business Process Automation
- Automated creation of DOD tasks
- Code review reports
- Deployment reports
- Agility reports
- Story life cycle reports
- Cycle time between states
- Capacity utilization reports
- By developers
- Branch comparison reports
- Post-Release Issues

14. Improvements – Summary (1)
Activity Before After Result
 Code reviews  Inconsistent
 Not-verifiable
 100% feedback not possible
 Real time
 Verifiable
 Built into workflow
 < 5 min to generate
 Time Savings
 Eliminate waste
 Improved Quality
 Transparent
 Accessible to SM
 Creating Story
Definition of Done
Tasks
 Manual
 Inconsistent
 Managed by SM
 < 5mins to generate
 Time Savings
 Eliminate waste
 Standardization
 Consistency among teams
 TFS Customizations  Was not done due to lack of Test
Environment
 Lack of confidence
 TFS Test Environment
allowed to test, validate and
improve
 Build in-house expertise
 Rapid customizations
 Playground for teams to
experiment
 DLL Versioning through
TFS Build
 Basic Configuration is missing
 Troubleshooting Deployment
issues is not reliable
 Too many teams are involved in
troubleshooting
 Lacking in verification step for
deployment validation
 DLLs are versions based on
the release branch and
build number
 Easy to troubleshoot
deployment issues
 Visibility to version info on
servers helps to identify
potential issues

15. Improvements – Summary (2)
Activity Before After Result
 Basic Sprint
Reports
 Inconsistences among the teams v
 Consolidated reports are difficult to
create due to variations
 Standardized reports
from tool
 < 5mins to generate
 Consistency among teams
 Variations are eliminated
 Time Saving
 Utilization Reports  Inconsistences among the teams in
report creation
 Incomplete data
 Unable to complete due to data
limitations
 Standardization among
teams
 Real time
 < 5mins to generate
resulting in early
feedback
 Self-service reports. Any one
create reports
 Better data leads to learning
 Better resource utilization
usage
 Advanced Sprint
Commitment
Reports
 Difficulty in creating reports. Each report
can take up to 4hours to create manually
 Inconsistences among the teams in
report creation
 Standardization
 Real time reports
 < 5mins to create
advanced time
 Team members can create
these reports leaving Scrum
Master focus on helping
reams
 Reports are self-service. No
waiting
 QA Story Testing
Reports
 Tracked manually
 Incorrect and data is prone to errors
 Too much time to consolidate and report
 Tracked within TFS
 Real time report
 Transparency
 Better change control
documentation
 Waste eliminated

16. Improvements – Summary (3)
Activity Before After Result
 Impediments Tracking  Done Manually
 Lack of visibility
 Tracked within TFS  Quick removal of impediments
 Historical data is preserved for further
analysis
 Tracking Regression
Testing
 Done manually
 Lack of visibility
 People are waiting for
information
 Data is available in TFS  Wait time is removed
 Visibility to all stakeholders
 Release Scope Tracking  Done in TFS but not
reported across all teams
 Managed using Excel
 Reporting is now based
on all teams
 Better resource planning for Release
Candidate
 Better resource planning
 Redeployment
Requests
 Communicated via emails  Done via PULL system  Reduce the noise
 Data for further analysis
 Redeployment Report  Time consuming to create
 Take up to 4 hours to
create
 < 5 min to create
 Real time
 Identify stories which have gap in gap in
understanding. Teams can learn from it
and improve further
 Leading indicators for potential issues.
Team can take pro-active steps to
mitigate issues

17. Improvements – Summary (4)
Activity Before After Result
 Business Approvers
Testing and Approval
 Done manually via email
 Approval tracking is time
consuming
 Takes up to several days for
follow-up and tracking
 Tracked via TFS Dashboard
 < 5 min for status
 < 15 for follow-up
 Business Approvers use
PULL system for identifying
stories for testing
 Early visibility to resources needed
 Real time data
 Capturing state changes
 Conversation and tracking approval
 Tracking Post-Releases  Tracked in Excel and in
Emails
 Lack of correlation to actual
changes
 Tracked in TFS and
transparent
 < 5mins to generate the
post-release issues report
 Can be tied to actual changes
 Valuable information to learn from
and improve upon
 Branch Variance
Detection
 Done manually
 Done late
 Done daily using automated
process
 Avoiding late code merges
 Better code quality and prevent
regressions
 Single Source of Truth  Data is dispersed in too
many places
 Most of the data is in TFS  System behavior can be measure
and adjusted
 Visibility to the Process  Limited visibility to the
whole process
 Process is not transparent
 End to End Release Process
is transparent
 All other systems have to be
configured to use TFS as single
source of Truth

18. Improvements – Summary (5)
Activity Before After Result
 Alerts on State
Changes
 Not Available
 Done via emails
 Being piloted in small
teams
 Real time notifications
 Workflow for
Operational Teams
 Tracked in Excel  TFS Workflow being is
used
 Capacity Planning
 Transparency
 Time saving
 Portfolio and Program
Level Tracking
 Done in Manually  Tracked used TFS SAFe
capabilities
 Transparency
 Real time information
 Project Management
Tracking
 Done in MS Project  TFS is being pilot to
represent
 Saving in license costs
 Real time integration with work
being done
 Release Status
Updates
 Done in emails  No emails. Real time
information
 Tremendous saving in time. Lot of
noise eliminated
 Release Go / No Go
Decision
 Subjective  Based on data in TFS  Transparent
 Repeatable

19. Key Takeaways
Process alone is not sufficient for improvements
- Augment with tools to measure, learn and improve
TFS can be used
- As a process improvement tool. Beyond Version control and Agile PM
- By stakeholders. Beyond Dev Teams. Microsoft has freed licenses to be used
stakeholders
- Implement workflow. Business can easily be implemented
Invest in tools
- As a learning tool
- Standardize and remove variations
- Self-serving
- Scaling
Setup TFS Test Environment
- Learn, Fail-fast and Fail-Safe
Most Important - Put Right People, Connect Passion and Strengths
to Organization Goals, Empower, Trust and Support Them

20. Sample ALM Reports
Lean and Agile Learning Network - Chicago

21. Sprint Story Status Report
• Snapshot of the Stories in any Sprint for any team
• Story Status from different team’s perspective
• Used by Product Leads, Scrum Masters, Team Leaders and other stake holders
• Purpose: Bring visibility to stories that have potential for slippage or
not getting into Release
Sprint & Date Story ID and Description Story Points Tags/ Comment Story Owner Dev Story Owner QA Test Case Status Development Status QA Satus PO Story Signoff Story Delivery Date to QA
Story 1 8 N/A Developer 1 N/A N/A In Progress To Do To Do
Sprint 1 - 1/1/2015 Story 2 5 N/A Developer 2 QA 1 Done In Progress To Do N/A
Story 3 8 N/A Developer 3 QA 2 Done In Progress To Do To Do
Story 4 0 N/A N/A N/A N/A N/A N/A N/A

22. Committed vs Delivered Report
• Committed vs Delivered Report across all the teams
• Purpose: Ease of generation. One report covering all the
teams. Consistent way to determine what is “Done” across the
teams
Committed Vs Delivered Report
Team Iteration Path Committed Delivered Ratio: Committed Vs Delivered
Team A Release Path 50 50 100

23. Process Metrics Report
• Summary of # of Stories and Points Completed in a Sprint
Process Metrics Report
Team Sprint Story Committed Story Completed Story Point Committed Velocity Story Spike Story Stretch
Test A Sprint 10 33 9 101 31 2 2

24. Story Report by Task & Hours
Sprint High Level Analysis Report
Sprint# StoryId Title Story Points Story Committed
Story Completed Hours Estimated Hours Spent
Deployment (Hrs) Design (Hrs) Development (Hrs) Testing (Hrs) Requirements (Hrs) Documentation (Hrs)
Sprint 12
100 Story 1 5 Yes Yes 53 51 0 0 35 16 0 0
101 Story 2 1 Yes Yes 20 20 0 0 5 15 0 0
102 Story 3 1 Yes Yes 14 12 0 0 6 6 0 0
103 Story 4 2 Yes Yes 25 28 0 0 14 14 0 0
• Reporting Tasks and Hours for each
story
• Provides insight into story activities and
discipline in the delivery

25. Release Metrics
• Release Status By Team
• Deployment Reports
• Readiness by Team
• Completed by Team
• Business Owner Signoff Status

26. Testing Status Reports
• UAT Testing Status. More of pull
model
• Summary of Testing by Teams
• Similar Reports for Business Owner for
Sign-offs
• All data is captured