Yin and Yang: Metrics within Agile and Traditional Lifecycles


Published on

Metrics are powerful tools when used to effect positive change in a project or organization. However, the value and benefits of metrics are often dependent on the context. While certain metrics provide information and insight to drive decision making for a traditional development approach, they may not be useful in an agile landscape—and vice versa. QA and agile experts Shaun Bradshaw and Bob Galen delve into the value, pitfalls, pros, and cons of various metrics in agile and waterfall development environments. Hear their experiences as they discuss and explore a variety of project-level, software development, and software testing metrics through the lens of both traditional and agile development contexts. Although Bob and Shaun respect each other’s knowledge and skill, they don’t often agree on metrics. And in this showdown, you’ll see why! Be prepared to learn, be entertained, and be ready to get in on the action as these two metrics titans go head-to-head.

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Yin and Yang: Metrics within Agile and Traditional Lifecycles

  1. 1. W6 Special Topics 5/1/2013 11:30:00 AM Yin and Yang: Metrics within Agile and Traditional Lifecycles Presented by: Shaun Bradshaw, Zenergy Technologies, Inc. Bob Galen, Rgalen Consulting Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
  2. 2. Shaun Bradshaw Cofounder of Zenergy Technologies, Inc., a QA and agile solutions consulting firm (zenergytechnologies.com), Shaun Bradshaw has spent the past fifteen years advising, teaching, and mentoring clients to improve their QA and test processes with effective testing and test management techniques. He is the coauthor, editor, and trainer of a suite of methodologies covering testing, test management, and test metrics. A popular speaker at many of the major QA industry conferences, Shaun is known for his presentations on test metrics, the S-Curve, and the Zero Bug Bounce. Bob Galen Bob Galen is an agile coach at RGalen Consulting and director of agile solutions at Zenergy Technologies, a North Carolina-based firm specializing in agile testing and leading agile adoption initiatives. Bob regularly speaks at international conferences and professional groups on topics related to software development, project management, software testing, and team leadership. He is a Certified Scrum Master Practicing (CSC), Certified Scrum Product Owner (CSPO), and an active member of the Agile Alliance and Scrum Alliance. Bob published Scrum Product Ownership–Balancing Value from the Inside Out, which addresses the gap in guidance toward effective agile product management. Contact Bob at bob@rgalen.com or bob.galen@zenergytechnologies.com.
  3. 3. 4/16/2013 The Yin & Yang Title of Software Metrics STAREAST 2013 Presented By: Bob Galen and Shaun Bradshaw Slide 1 Software Peace of Mind in Action ??? Introductions Bob Galen & Shaun Bradshaw Director, Agile Solutions – Zenergy Technologies Determined agile adoption coach, trainer, and consultant 10+ years of agile experience across Software, QA/Test, and Project Management Specialty in Agile at-Scale challenges bob.galen@zenergytechnologies.com Slide 2 VP of Consulting Solutions – Zenergy Technologies Experienced test manager, consultant, trainer IT Testing strategist 15 years of multi-domain experience Agile experience; Scrum Master shaun.bradshaw@zenergytechnologies.com Software Peace of Mind in Action 1
  4. 4. 4/16/2013 ??? Format For This Session Focus for this session: QA & Testing Metrics Panel format 15 minutes for Shaun 15 minutes for Bob Write down your questions, ideas, metrics, etc. on 3x5 cards 30 minutes of Q&A interaction and debate Slide 3 Software Peace of Mind in Action What is Measurement? • “Measurement is the process by which numbers or symbols are assigned to attributes of entities in the real world in such a way as to characterize them according to clearly defined rules.” • “Measurement is the empirical, objective assignment of numbers, according to a rule derived from a model or theory, to attributes of objects or events with the intent of describing them.” 2
  5. 5. 4/16/2013 ??? Software Metrics Traps Karl Wiegers 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. http://www.processimpact.com/articles/mtraps.html Lack of Management Commitment Measuring Too Much, Too Soon Measuring Too Little, Too Late Measuring the Wrong Things Imprecise Metrics Definitions Using Metrics Data to Evaluate Individuals Using Metrics to Motivate, Rather than to Understand Collecting Data That is Not Used Lack of Communication & Training Misinterpreting Metrics Data Slide 5 Software Peace of Mind in Action ??? Case Study Large, traditional financial / insurance organization 100 year company, traditionally staffed IT organization Regulatory support Large-scale project; joint with other consulting firms Slide 6 Software Peace of Mind in Action 3
  6. 6. 4/16/2013 ??? Waterfall is ‘Real’ Not every project can be broken down into 2-week ‘chunks’ Not every company wants to be agile Scale of the project impacts everything We’ll know we’re done when we’re done is NOT an acceptable answer in many contexts Waterfall and Agile Slide 7 Software Peace of Mind in Action ??? Monitoring Phases & Transitions The overriding goal was to plan a series of maturation phases Testing would serve as the guide between phases Qualifying progress Verifying functionality Measuring variation / defects Guiding next steps Slide 8 Software Peace of Mind in Action 4
  7. 7. 4/16/2013 ??? Risk-Based Testing All testing was focused towards what Was delivered Worked Was planned Was considered Higher Priority features Was considered Higher Risk items Slide 9 Software Peace of Mind in Action ??? Key Metrics – “Exit Criteria” Test Execution by priority Have we tested enough? Did we cover our plans? Overall pass rates Is the software working? Where are we ‘Blocked’? Are there any ‘Patterns’ to the failures Percent of defects corrected by severity Are we catching up on repairs? Is the software maturing at a sufficient rate to support the plan? Slide 10 Software Peace of Mind in Action 5
  8. 8. 4/16/2013 ??? Customer Override… Basically, the customer has the final authority to adjust Metrics Analysis (Re)Actions Goals So, ’agility’ from a leadership & governance perspective Slide 11 Slide 12 Software Peace of Mind in Action Software Peace of Mind in Action 6
  9. 9. 4/16/2013 Transition… Slide 13 Software Peace of Mind in Action ??? Agile Metrics Agile methods are generally not metrics centered, They’re much more team-centered and progress or outcome centered Rarely do you want to examine the individual; leave that to the team Exceptions – experienced coaches Avoid “Metrics Dysfunction” metrics Lines of code produced Bugs found Estimate vs. Actual Slide 14 Software Peace of Mind in Action 7
  10. 10. 4/16/2013 ??? Big 4 Agile Metrics 1. 2. 3. 4. Value Delivered Velocity & Throughput Quality Team Morale & Satisfaction Slide 15 Software Peace of Mind in Action ??? Value-Delivered Prioritization Always deliver the highest priority work first Earned value – it’s “Done” Customer Acceptance User Story sign-off Active customer demonstration; embrace changes ROI – Usage Sales & usage targets; confirm Did we make the right decisions on value? Adjustments? Slide 16 Software Peace of Mind in Action 8
  11. 11. 4/16/2013 ??? Velocity & Throughput Productivity matters in agile teams First, the software is only accepted if it meets clear Done-Ness criteria Velocity – the number of Story Points produced per Sprint Throughput – the number of ‘Days’ that a story was InProgress, relative to it’s size; also a risk play Continuous improvement Velocity – UP; Throughput - DOWN Slide 17 Software Peace of Mind in Action ??? Quality Test Automation Continuous Integration & Continuous Deployment Every check-in…run automation; Stop-the-Line Defect ‘Escapes’ Escaping your Done-Ness Escaping your Sprint Escaping into the wild…Customer Escapes Pairing, inspections, RCA Continuous Improvement – downward trending Slide 18 Software Peace of Mind in Action 9
  12. 12. 4/16/2013 ??? Team Morale & Satisfaction Self-directed; Team autonomy; Listened to Attrition is analyzed; New hire onboarding Sustainable pace Periodic surveys Team overall health & satisfaction Feedback on adjustments Agile values Make the survey and the actions Transparent Make it clear—people are central to your success Slide 19 Software Peace of Mind in Action ??? That’s it… Aligned with Goals Sprint Goals & Release Goals Organizational Goals & Company Goals Attributes: They’re all important…perhaps “People First” There are no “fixed targets” Keep them simple; everyone is aware of the goal Measure as a team Stop the individual metrics dysfunction Slide 20 Software Peace of Mind in Action 10
  13. 13. 4/16/2013 Transition… Slide 21 Software Peace of Mind in Action Now It’s YOUR Turn Questions? Debate! Slide 22 Software Peace of Mind in Action 11
  14. 14. 4/16/2013 ??? Questions for your Consideration What ‘healthy’ metrics have you personally experienced? What about stories of dysfunctional metrics? Challenges you’ve seen in moving from Waterfall to Agile or vice versa? If you only had 1-3 metrics to measure software in your own company, what would they be? Slide 23 Software Peace of Mind in Action ??? Questions for your Consideration Example of what we could discuss Estimation vs. Actuals Planning Poker & ‘Relative’ estimation Anyone use GQM? Slide 24 Software Peace of Mind in Action 12
  15. 15. 4/16/2013 Any other questions? Thank you! Slide 25 Software Peace of Mind in Action 13