This document summarizes the challenges of adopting agile practices in testing at CollabNet and their efforts to address them. Key challenges included siloed testing within sprints, communication issues between distributed scrum teams, provisioning testing environments, and ensuring quality for early releases. To overcome these, CollabNet emphasized cross-team communication using collaboration tools, increasing automation to speed testing and reduce defects, training testers in development practices, and getting the entire team involved in quality. The goal was to continuously improve testing practices and metrics sprint over sprint to fully integrate testing into their agile processes.

1. 1
Agile Adoption in Testing
Case study of Agile Transformation at CollabNet
Janardhanam Venkat – Director Engineering
NGTC Conference (c)

2. 2
Overview
• Challenges
– Agile Team
– Communication
– Environment
– Release
• Defects Prevention
• Discussion
• Q & A

3. 3
Agile Team Challenges – Scrum Fall
• Mini waterfall within a sprint
– Testing during end of the sprint
• QA co-located, Dev box testing
• Estimation by Story Points – Dev / QA
• Someone within team as Scrum Master
NGTC Conference (c)
Dev
QA

4. 4
Communication Challenges
• QA could not share ideas between scrum teams
– Scrum of Scrum
– Team rooms during stabilization – last sprint
– End of release catch up
• Distributed scrum teams
– Same time zone, Dev & QA are co-located
– Same time zone, Dev & QA in one team are distributed
– Different time zone, Dev & QA in one team are distributed
NGTC Conference (c)
Business people and developers must work
together daily throughout the project

5. 5
Distributed across time zones/geographies
Distributed Teams
Scrum Master
Product Owner
USA / ArgentinaGermanyIndia
Shadow PO
USA
NGTC Conference (c)
Global QA Teams
Overlapping
time
standup
Offline
Standup /
UAT

6. 6
Communication Challenges
NGTC Conference (c)
• Information lost through phone conferences & chats
– Collaboration Tools
• Notification of changes – Epic, Stories, Code Reviews, Defects, Tasks
• Wiki, Discussion Board - shared mailing list
• Sprint review & meeting recording

7. 7
Environment Challenges
• Self provisioning of boxes
– Platform, OS & application configuration
– E.g.: Solaris, CentOS, Suse, RHEL, Windows in combination with 32/64
Bit, 2/4/8 core & Storage space
NGTC Conference (c)NGTC Conference (c)
Datacenter
Distributed Teams
SDLC Mgmt.
Customer
(Early release)
Product – Ops 1
Product – QA 1
Product – QA 2
Product – QA 3
Product – Dev 1
Give them the environment and support they need

8. 8
Release Challenges
• Early release to Sales & Customer
• Quality of code
– No P0 & P1 defect
– Demo to CAB
– To achieve this milestone – Whole team involved in automation
• Defect Prevention – Quality is everyone’s responsibility
– Make testing an integrated natural part of the agile process
– Beyond Agile
NGTC Conference (c)
Deliver Working Software very frequently

9. 9
QA Defect Tunnel – Prevent Defect State
Customer
Reported Issue
People
ToolsProcess
CSAT Story quality audit
 Usability quality audit
 Exploratory testing
 Customer specific testing
 Increase automation coverage
 Security testing
Metrics
 Improve Stage Testing  Refine triangle meeting
 Refine release signoff
 Invest Static Code analysis
 Invest in unit test coverage tool
Cost
 Customer Advocate
The team reflects on how
to become more effective, then tunes and adjusts
Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.

10. 10
Feedback loop
• Filling gaps for Dev - Invest in tools
– Security Vulnerability
– Code Coverage
– Static Code Analysis
– Broken Link Checker
• Training
– Basic JAVA, Automation
• QA writing unit test
• Scope Creep
– Accepted, re-estimate the effort & prioritize
– Fixed date variable scope release
NGTC Conference (c)
Welcome changing requirements, even late in
development.

11. 11
Discussion: Where this leads to?
• Thin line between Dev and QA
• Challenges Continue
– Global QA Team – Ratio between Dev and QA ?
– Documentation lags sprint after sprint
NGTC Conference (c)

12. 12
Summary
• Productivity & Agility
– Transform cumbersome, manual application testing & deployments into
fully governed, lean and efficient automated processes that take hours
instead of days.
• Right Metrics
– Help improve sprint over sprint
• Tools are critical, but they are not the only answer
– Necessary to have good guidelines in place
• Best Engineering practices
– Quality across agile
• Agile by the books
– Whatever works for the team at respective agile maturity level
NGTC Conference (c)

13. 13
venkatj@collab.net
www.collab.net
NGTC Conference (c)