Agile Metrics Boom or Bane

www.unicomlearning.com
India Agile Week-2014
13-June-2014, Mumbai
Agile Metrics: Boon or Bane?
Suchitra Vasudevan
General Manger
Atos India Pvt. Ltd.
www.agileinbusiness.com
Samir Umrikar
Associate Manager

Agile Metrics
UNICOM Presents

Agile Metrics - Agenda
UNICOM Presents
To Measure or Not to Measure?
Challenges faced
Case study of project
Industry best practices /
recommendations
Q& A

Agile Metrics
UNICOM Presents
Metric = “A measure or a combination of measures for quantitatively assessing, controlling or improving
a process, a product, a team”
Key Performance Indicator =“Quantifiable metrics which reflect the performance of an organization in
achieving its goals and objectives”
Diagnostic = “A metric used to diagnose and improve the processes that produce business value. What
it measures might not have a direct value to the customer.”

To Measure OR Not To Measure?
UNICOM Presents
Why - To Measure
√Feedback & Decision-making
For team performance
Continuous improvement
Motivate, Challenge
What - To Measure
√“Just enough” simple metrics
√“Business Value” to customer / Measure-Up
Outcomes
Trends, Ratios
How - To Measure
√“Good-enough” quality
Storypoints, DoD, Burndown charts
“Hard-to-game” metrics
When - To Measure
√Start / Stop
√Frequency, regular (iteration/sprint)
• Why - Not to Measure
√ For individual performance
√ Comparing projects against
√ Controlling teams
• What - Not to Measure
√ Outputs (efforts, variance from plan..)
√ Numbers (rather than trends)
√ Difficult/time-consuming
• How- Not to Measure
√ Organization level rather than project-level
√ Not set targets (use minimal)

Journey to Agile
UNICOM Presents
Client was completely following waterfall /
iterative model till 2011
Agile / Scrum methodology started
implementing in project in early 2012 –
closely linked to agile Global Delivery
Platform with customization specific to
client
Deliver highest business value with
real working software to customer in
the shortest time.
Status:
80% Java projects around 25 projects,
170 team members following Agile

Challenges faced / Solutions applied – 1/2
UNICOM Presents
▶ Team new to Agile
– Scrum Training for teams, 35 Certified Scrum Masters (CSM) & 2 PO
– Agile Club – Monthly meeting to share best practices (client, local)
– Agile Day – KXC for team with innovation games and lessons learned
Distributed delivery and roles spread across locations
– Scrum Masters in each location working with local scrum teams Sometimes
appointed 2 Scrum masters for a sprint
– Backup of key roles PO and SM present at offshore
– For large teams - daily Scrum meeting at each location using Lync (taking
time-zone constraints), “Scrum of scrum” meeting between Scrum Masters
across locations
▶ Common estimation model, standard definition of story points
– Complexity guideline template with guidelines at project-level to define
story points
– No standard estimation model defined

Improvements : Lessons Learned from Agile
UNICOM Presents
Good practices / Learnings
Experience & Knowledge
Customer Brainstorming QualityEmployees
• Agile Club – extended
with local experts
• Deputy of client Key roles
(PO, SM) in local
• Mandatory Demo with
PO in sprint
• Agile Day - KXC, Quiz
• Lessons Learned in
Retrospective meetings
• Customization of Agile
based on client context
⇒Pair-programming of
new and experienced
members
⇒Roles matched across
distributed teams
• Scrum training,
certification (35 Certified
Scrum Masters, 2 Product
Owner)
• Daily scrum meetings with
teams and “Scrum of
Scrum” weekly across
locations
• Definition of “Done”
standardized
• Continuous
Integration (SONAR
dashboard,
automated testing)
• Usage of Jira
dashboard

Case Study Project - Overview
UNICOM Presents
▶ Tools used to mange SCRUM / Agile Practices : JIRA + Green Hopper
Plug-in
▶ Scrum Team Size: 6 to 8
▶ Product Owner Role is performed by someone from client team. However
there are 2 product owners now in local team.
▶ Scrum Master Role is performed by local / client
▶ Sprint Size: 2 to 4 weeks
▶ Product Owner makes Release and Sprint backlog in JIRA.
▶ Before starting a sprint, local team provides the capacity to client team
considering public / personal holidays of local team members working on
the sprint.
▶ Local team receive requirements for a given sprint (ppt, doc). client
team also creates stories for each sprint in JIRA.

Case Study project -Agile Metrics Dashboard (1/3)
UNICOM Presents
Velocity Ratio :- Objective is to Keep track of the
velocity w.r.t. Baseline Velocity and analyse
improvement.
Target >=0
Indicators
• > 0 : Positive Velocity improvement trend
• = 0 : Velocity as per baseline
• < 0 : Velocity not as per expectations; further
analysis and actions needed
Story Completion Ratio:- Objective is to Measure
the budget and timeliness of deliverables of sprint
Target >= 70%
Indicators
• 100% -> Deliverables exceeding expectations
• > 70% and < 100% -> Deliverables meeting
expectations
• < 70% -> Deliverables NOT meeting expectations,
need analysis and actions

UNICOM Presents
Story Acceptance Ratio:- Objective is to
Measure Quality of deliverables of sprint
Target >= 90%
Indicators
• 100% -> Deliverables exceeding expectations >
90% and < 100% -> Deliverables meeting
expectations
• < 90% -> Deliverables NOT meeting expectations,
need analysis and actions
Average Cost Per Story Point:- Objective is to
Measure effort / Cost spent to complete a story
point. Ideally it needs to decrease over Sprints.
Target <= Agreed baseline
Indicators
• Reducing Trend is good indicator
• In case of increasing trend then analysis and
actions needed.

UNICOM Presents
Quality completeness by SPRINT:- Objective is to
Measure Quality of overall sprint. It is based on
• Product Backlog Stories Written (INVEST
Model- Independent, Negotiable, Valuable,
Estimable & Testable)
• Sprint Organised
• Sprint Content Proposed
• Sprint Planned
• Sprint Prepared
• Sprint Executed
• Sprint Reviewed
• Sprint Retrospective Completed
Phase wise Dashboard is also used to analyze
Quality with in and across SPRINT.
It is Quality check questionnaire based on various
phases of Sprint life cycle.

Case Study project - SONAR Dashboard
UNICOM Presents
Unit Test Success:- Objective is to Measure how
many test cases are passed. It is run by J Units and
it is delivered at the time of Sprint Review.
Unit Test Coverage:- Objective is to Measure how
much existing code is covered by the test cases.
The objective is to cover 50% of legacy code in
next one year.
Rule Compliance :- As client has its own
technology and platform. There are specific coding
rules that had been defined and continuously
updated by client and the developed code by local
teams is expected too follow at least 95% of the
rules.

Industry best practices / recommendations
UNICOM Presents
Here is a quick list that you may consider using:
• Velocity Ratio, Acceleration Ratio
• Resource Utilization, Burndown charts
• Average Cost
• Adherence to doneness/acceptance criteria, checklists
(checklist item burnup), standards (PCI Standards)
• Test coverage (Code Coverage, Branch Coverage),
Running tested features
• Rule Compliance (wrt. Customer standards), Team
member peer reviews
Category Metric
Quality Defect Count, Technical Debt, Faults-Slip-Through
Predictability Velocity, Running Automated Tests
Value Customer Satisfaction Survey, Business Value Delivered (ROI, EV
analysis)
Lean Lead Time, Work In Progress, Queues
Cost Average Cost Per Functions

Questions / Queries
UNICOM Presents

Agile Metrics - Agenda
UNICOM Presents
List of references
• http://www.slideshare.net
• http://www.scrumalliance.org/
• http://agile2009.agilealliance.org
• http://www.toolsjournal.com/agile-articles

Organized by
UNICOM Trainings & Seminars Pvt. Ltd.
contact@unicomlearning.com
Suchitra Vasudevan
Suchitra.vasudevan@atos.net
Samir Umrikar
Samir.umrikar@atos.net

Agile Metrics Boom or Bane

More Related Content

What's hot

Viewers also liked

Similar to Agile Metrics Boom or Bane

Agile Metrics Boom or Bane