Yin and Yang: Metrics within Agile and Traditional Lifecycles

W6
Special Topics
5/1/2013 11:30:00 AM

Yin and Yang: Metrics within Agile and
Traditional Lifecycles
Presented by:
Shaun Bradshaw, Zenergy Technologies, Inc.
Bob Galen, Rgalen Consulting

Brought to you by:

340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com

Shaun Bradshaw
Cofounder of Zenergy Technologies, Inc., a QA and agile solutions consulting firm
(zenergytechnologies.com), Shaun Bradshaw has spent the past fifteen years advising, teaching, and
mentoring clients to improve their QA and test processes with effective testing and test management
techniques. He is the coauthor, editor, and trainer of a suite of methodologies covering testing, test
management, and test metrics. A popular speaker at many of the major QA industry conferences, Shaun
is known for his presentations on test metrics, the S-Curve, and the Zero Bug Bounce.

Bob Galen
Bob Galen is an agile coach at RGalen Consulting and director of agile solutions at Zenergy
Technologies, a North Carolina-based firm specializing in agile testing and leading agile adoption
initiatives. Bob regularly speaks at international conferences and professional groups on topics related to
software development, project management, software testing, and team leadership. He is a Certified
Scrum Master Practicing (CSC), Certified Scrum Product Owner (CSPO), and an active member of the
Agile Alliance and Scrum Alliance. Bob published Scrum Product Ownership–Balancing Value from the
Inside Out, which addresses the gap in guidance toward effective agile product management. Contact Bob
at bob@rgalen.com or bob.galen@zenergytechnologies.com.

4/16/2013

The Yin & Yang
Title
of Software Metrics

STAREAST 2013
Presented By:
Bob Galen and Shaun Bradshaw

Slide 1

Software Peace of Mind in Action

???
Introductions
Bob Galen & Shaun Bradshaw

Director, Agile Solutions – Zenergy
Technologies
Determined agile adoption coach, trainer,
and consultant
10+ years of agile experience across
Software, QA/Test, and Project Management
Specialty in Agile at-Scale challenges
bob.galen@zenergytechnologies.com

Slide 2

VP of Consulting Solutions – Zenergy
Technologies
Experienced test manager, consultant,
trainer
IT Testing strategist
15 years of multi-domain experience
Agile experience; Scrum Master
shaun.bradshaw@zenergytechnologies.com


1

4/16/2013

???
Format For This Session
Focus for this session: QA & Testing Metrics
Panel format
15 minutes for Shaun
15 minutes for Bob

Write down your questions, ideas, metrics, etc. on 3x5
cards
30 minutes of Q&A interaction and debate
Slide 3


What is Measurement?

• “Measurement is the process by which numbers
or symbols are assigned to attributes of entities
in the real world in such a way as to
characterize them according to clearly defined
rules.”
• “Measurement is the empirical, objective
assignment of numbers, according to a rule
derived from a model or theory, to attributes of
objects or events with the intent of describing
them.”

2

4/16/2013

???
Software Metrics Traps
Karl Wiegers 1.
2.
3.
4.
5.
6.
7.
8.
9.
10.

http://www.processimpact.com/articles/mtraps.html

Lack of Management Commitment
Measuring Too Much, Too Soon
Measuring Too Little, Too Late
Measuring the Wrong Things
Imprecise Metrics Definitions
Using Metrics Data to Evaluate Individuals
Using Metrics to Motivate, Rather than to Understand
Collecting Data That is Not Used
Lack of Communication & Training
Misinterpreting Metrics Data

Slide 5


???
Case Study
Large, traditional financial / insurance organization
100 year company, traditionally staffed IT organization
Regulatory support
Large-scale project; joint with other consulting firms

Slide 6


3

4/16/2013

???
Waterfall is ‘Real’
Not every project can be broken down into 2-week
‘chunks’
Not every company wants to be agile
Scale of the project impacts everything
We’ll know we’re done when we’re done is NOT an
acceptable answer in many contexts
Waterfall and Agile

Slide 7


???
Monitoring Phases & Transitions
The overriding goal was to plan a series of maturation
phases
Testing would serve as the guide between phases
Qualifying progress
Verifying functionality
Measuring variation / defects
Guiding next steps

Slide 8


4

4/16/2013

???
Risk-Based Testing

All testing was focused towards what
Was delivered
Worked
Was planned
Was considered Higher Priority features
Was considered Higher Risk items

Slide 9


???
Key Metrics – “Exit Criteria”
Test Execution by priority
Have we tested enough? Did we cover our plans?

Overall pass rates
Is the software working? Where are we ‘Blocked’?
Are there any ‘Patterns’ to the failures

Percent of defects corrected by severity
Are we catching up on repairs?
Is the software maturing at a sufficient rate to support the plan?
Slide 10


5

4/16/2013

???
Customer Override…
Basically, the customer has the final authority to adjust
Metrics
Analysis
(Re)Actions
Goals
So, ’agility’ from a leadership & governance perspective

Slide 11

Slide 12



6

4/16/2013

Transition…

Slide 13


???
Agile Metrics
Agile methods are generally not metrics centered,
They’re much more team-centered and progress or
outcome centered
Rarely do you want to examine the individual; leave that to the
team
Exceptions – experienced coaches

Avoid “Metrics Dysfunction” metrics
Lines of code produced
Bugs found
Estimate vs. Actual
Slide 14


7

4/16/2013

???
Big 4 Agile Metrics
1.
2.
3.
4.

Value Delivered
Velocity & Throughput
Quality
Team Morale & Satisfaction

Slide 15


???
Value-Delivered
Prioritization
Always deliver the highest priority work first
Earned value – it’s “Done”

Customer Acceptance
User Story sign-off
Active customer demonstration; embrace changes

ROI – Usage
Sales & usage targets; confirm
Did we make the right decisions on value? Adjustments?
Slide 16


8

4/16/2013

???
Velocity & Throughput
Productivity matters in agile teams
First, the software is only accepted if it meets clear Done-Ness
criteria
Velocity – the number of Story Points produced per Sprint
Throughput – the number of ‘Days’ that a story was InProgress, relative to it’s size; also a risk play

Continuous improvement
Velocity – UP; Throughput - DOWN
Slide 17


???
Quality
Test Automation
Continuous Integration & Continuous Deployment
Every check-in…run automation; Stop-the-Line

Defect ‘Escapes’
Escaping your Done-Ness
Escaping your Sprint
Escaping into the wild…Customer Escapes

Pairing, inspections, RCA
Continuous Improvement – downward trending
Slide 18


9

4/16/2013

???
Team Morale & Satisfaction
Self-directed; Team autonomy; Listened to
Attrition is analyzed; New hire onboarding
Sustainable pace
Periodic surveys
Team overall health & satisfaction
Feedback on adjustments
Agile values

Make the survey and the actions Transparent
Make it clear—people are central to your success

Slide 19


???
That’s it…
Aligned with Goals
Sprint Goals & Release Goals
Organizational Goals & Company Goals

Attributes:
They’re all important…perhaps “People First”
There are no “fixed targets”
Keep them simple; everyone is aware of the goal

Measure as a team
Stop the individual metrics dysfunction
Slide 20


10

4/16/2013

Transition…

Slide 21


Now It’s YOUR
Turn
Questions?
Debate!

Slide 22


11

4/16/2013

???
Questions for your Consideration
What ‘healthy’ metrics have you personally experienced?
What about stories of dysfunctional metrics?
Challenges you’ve seen in moving from Waterfall to Agile
or vice versa?
If you only had 1-3 metrics to measure software in your
own company, what would they be?

Slide 23


???
Questions for your Consideration
Example of what we could discuss
Estimation vs. Actuals
Planning Poker & ‘Relative’ estimation

Anyone use GQM?

Slide 24


12

4/16/2013

Any other questions?

Thank you!

Slide 25


13

Yin and Yang: Metrics within Agile and Traditional Lifecycles

More Related Content

What's hot

Similar to Yin and Yang: Metrics within Agile and Traditional Lifecycles

More from TechWell

Recently uploaded

Yin and Yang: Metrics within Agile and Traditional Lifecycles