While traditional performance metrics often measure individual output or adherence to pre-defined plans, measuring performance in agile teams requires a different approach. Agile teams operate in iterative cycles, prioritizing adaptability and learning over rigid goals. So, why do organizations still measure their performance? By using the right metrics in the right way, organizations can empower their agile teams to thrive and deliver exceptional results.

1. Nesma webinar
2024-02

2. www.nesma.org

3. Nesma - Software Measurement Standards and Improvement
Nesma is a non-profit international software measurement organization, founded in 1989, focused on:
▪ Spreading knowledge about software measurement and software metrics;
▪ Act as a Body of Knowledge for the industry regarding the use of software metrics in all business areas;
▪ Remain independent, objective and not-for-profit;
▪ Research the applicability of software metrics in all business areas;
▪ Connect relevant organizations in the industry that Nesma feels are expert in one of the areas where
software measurement and metrics are important;
▪ Produce relevant guidelines, reports and other information products that are useful for the software
industry;
▪ Produce a platform where people can discuss issues they experience with software measurement and metrics
or where they can exchange ideas and/or knowledge.
Nesma is Gold Partner of ISBSG, partner in the ICEAA Software Special Interest Group and partner with the China
Software Process Improvement group.

4. Introducing me
Harold van Heeringen
▪ >25 years experience in IT, >20 years in software measurement and metrics.
▪ Ex-Sogeti – 17 years – metrics desk
▪ IDC Metri – 8 years – Principal Consultant and Practice Lead IT Intelligence services.
▪ ISBSG – Immediate Past President.
▪ NESMA – chairman.
▪ SIG ICEAA Software – Board member.
50 years young.
Living in Veendam, the Netherlands.
Married, 3 children.
Passions: playing speed chess, skiing, travelling, fitness, mountain biking, playing padel.

5. Topics for today
▪ Key benefits of agile.
▪ Why agile team performance measurement?
▪ Typical Agile / DevOps metrics used by teams.
▪ The CIO challenge.
▪ Measuring Business Value.
▪ High-performance and low-performance teams
▪ Q&A
This webinar is being
recorded! And
uploaded to the
Nesma YouTube
channel

6. Key Benefits of Agile
▪ Increased Speed and Responsiveness: By working
in short sprints, teams can deliver working software
frequently, adapting to changing requirements and
market demands quickly.
▪ Improved Quality: Continuous feedback loops and
rigorous testing practices within each sprint lead to
higher quality software with fewer defects.
▪ Enhanced Team Morale and Engagement:
Collaborative and empowered environments promote
ownership, foster creativity, and boost team morale.
▪ Greater Customer Satisfaction: By actively involving
customers in the development process, agile teams
ensure the delivered software meets their needs and
expectations.
▪ Reduced Risk and Waste: The iterative nature of
agile allows for early detection and correction of
issues, minimizing wasted effort and reducing project
risks.

7. Why Agile Team Performance Measurement?
• Uncover hidden insights: Gain a deeper
understanding of your team's strengths, weaknesses,
and areas for improvement.
• Drive continuous improvement: Use data to identify
opportunities for process optimization, skill
development, and overall team effectiveness.
• Make data-driven decisions: Allocate resources
effectively, prioritize initiatives, and measure the impact
of changes on team performance.
• Benchmark against industry standards: Compare
your team's performance to industry benchmarks to
identify areas for improvement and track progress over
time.
• Understand your bottlenecks: Does a team have a lot
of rework due to unclear requirements for instance, or
because of changing requirements, or because of bad
development practices. This is the starting point to
address the issues and to improve!

8. Metrics are to improve, not to punish!
▪ Metrics provide insights, which should result in decision-making.
▪ Decision-making may result in change, and change is considered ‘bad’ or
‘threatening’.
▪ Therefore, there is a lot of resistance against measuring team
performance in many teams.
▪ However, most people and teams do want to improve their performance!
▪ Good Team performance should therefore result in happy teams and
happy people, and also happy customers and happy users.
▪ Team Performance metrics should be:
▪ Value-driven: Aligned with the team's and organization's overall goals.
▪ Actionable: Providing insights that lead to concrete improvements.
▪ Collaborative: Enabling transparency and shared understanding
within the team.
▪ Lightweight and Sustainable: Easy to collect and analyze without
adding excessive burden.

9. Examples of Agile metrics
▪ Flow metrics: Cycle time, lead time, work in
progress (WIP), and throughput, to assess workflow
efficiency.
▪ Process Quality metrics: Defect escape rate, rework
rates, test coverage, and automated testing
effectiveness, to gauge software quality.
▪ Delivery metrics: Release frequency, on-time
delivery, velocity, and burndown charts, to track
progress and predict outcomes.
▪ Team collaboration and communication metrics:
Code review time, team meeting effectiveness, and
collaboration tools adoption, to assess team
dynamics.
▪ Product Quality Metrics: Security, Maintainability,
Performance Efficiency, Reliability.
▪ Functional Size metrics: Project Delivery Rate, Cost
Efficiency, Delivery Speed, Sprint Quality.

10. DORA metrics
The DevOps Research and Assessment (DORA) team has
identified four metrics that measure DevOps performance.
▪ Deployment Frequency: Measures how often your team
successfully deploys code changes or releases to production.
• Lead Time for Changes: Measures the time it takes from the
moment code is committed until it's running successfully in
production.
• Change Failure Rate: Tracks the percentage of deployments
that result in failures, rollbacks, or require hotfixes.
• Time to Restore Service: Tracks the time it takes to recover
from service degradations, outages, or other incidents.

11. DORA metrics and performance
Being an Elite DORA performer is great, but how much business value do you deliver?
Source: LinearB 2023 Software Engineering Benchmarks report

12. The CIO challenge

13. Business Value is key!

14. Business value
▪ What is Business Value? Business value is a multifaceted concept
encompassing all aspects that contribute to the long-term health and well-
being of a company. It goes beyond just financial metrics and incorporates
both tangible and intangible elements. Components:
• Financial Performance: This includes traditional metrics like revenue, profit,
and return on investment (ROI).
• Customer Value: This refers to the perceived value customers gain from
your products or services, often measured by customer satisfaction, loyalty,
and retention.
• Operational Efficiency: This encompasses the ability of the business to
operate effectively, minimizing waste and maximizing resource utilization.
• Innovation and Competitive Advantage: This includes the ability to develop
new and innovative products or services that give you an edge over
competitors.
• Brand Reputation and Market Share: A strong brand reputation and a
healthy market share contribute significantly to a company's overall value.
• Employee Satisfaction and Engagement: A happy and engaged workforce
leads to better productivity, innovation, and customer service, ultimately
impacting business value.

15. The goal is to provide business value!
IT Business perception
Business value perception

16. Can Functionality be a proxy for business value?
Functionality can be a good proxy for business value,
when it meets certain conditions:
▪ Well-prioritized by the Product Owner.
▪ Useful for the users/customers.
▪ Increases customer satisfaction.
▪ Increases brand recognition.
▪ Improves operational efficiency.
▪ Increases market share and company profitability.
Functionality is a proxy for business value!
▪ Then, measurement of business value delivered
by teams becomes possible!
▪ ISO standards for functional size measurement
can be used to objectively measure the
business value delivered by agile teams!

17. Why identify high-performing / low-performing teams?
Benefits of Identifying High-Performing Teams that deliver more business value, faster, better!
▪ Replication and Scalability: Analyze the practices and characteristics of high-performing teams to replicate
their success across the organization.
▪ Improved Resource Allocation: Allocate resources, such as training, budget, and tools, to support and further
empower high-performing teams.
▪ Retention and Motivation: Recognizing and celebrating high-performing teams boosts morale, fosters a
sense of accomplishment, and increases employee retention.
Benefits of Identifying Low-Performing Teams that deliver less business value, slower, worse.
▪ Early Intervention: Identify and address the root causes of underperformance before they escalate and
significantly impact productivity or morale.
▪ Targeted Support: Provide specific support and resources tailored to address the unique challenges faced by
low-performing teams.
▪ Improved Team Dynamics: Facilitate collaboration, communication, and problem-solving within low-
performing teams to help them improve their effectiveness.
But how to identify high- and low-performing teams? Measure the output delivered by teams!

18. The industry
Velocity:
▪ Measures the amount of work a team completes in a
sprint.
▪ Usually quantified in story points (relative estimates
of effort) or ideal days.
▪ Helps track trends over time and forecast future work.
▪ Limitations: Should not be used for comparisons
between teams or to pressure teams to overcommit.
Burndown Chart
▪ Visualizes the remaining work in a sprint (often in
story points) over time.
▪ Ideally, the line trends downward as work progresses.
▪ Helps identify potential issues and adjust the sprint
backlog if needed.
▪ Provides transparency and fosters team collaboration.
Source: softwaretestinghelp.com
Source: https://unichrone.com/blog/agile/sprint-burndown-chart/

19. Story Points guess effort, don’t measure value

20. Functional Size Measurement – Nesma ISO/IEC 24570:2018
The size of an information system, product or project is the basis for all business management activities such as
estimating system development, measuring team productivity, benchmarking and project control.
▪ Person independence: it should not matter who carries out the measurement;
▪ Repeatability: when the measurement is made, should not matter; the outcome must remain the same;
▪ Comparability: the size of different systems should be compared with each other;
▪ Independence of technology: it should not matter what technical environment the system is built. Technical
aspects shall not in any way have an influence on the size to be measured. The value and appreciation a user gives
an information system is often also determined without that the user has knowledge of the technical
backgrounds;
▪ Applicability in an early stage, the measurement should be carried out not only on delivery of the system. The
sooner you can measure in the system development process, the better you can monitor productivity and
budgets;
▪ Simplicity: the unit should be easy to understand
▪ Future-proof: also in the introduction of new technology, the data must be meaningful and directly applicable;
▪ Expression of user value: the unit says something about the functionality offered to the user;
▪ Expression complexity: the unit shows which (hidden) complexity processing the information contains.
▪ ISO/IEC standard.

21. Enhancement Function Points (EFP): FP add + modify + delete

22. Team Performance Metrics based on EFP
▪ Project Delivery Rate: effort hours/EFP
▪ Cost Efficiency: Cost/EFP
▪ Delivery Speed: EFP/month
▪ Sprint Quality: Defects/ 1000 EFP
Tracking the trend brings value, but how is
the performance against:
▪ other teams?
▪ the industry?

23. Comparing to the industry – ISBSG data
11800+ data points of new development and enhancement projects, releases and sprints.
253 columns with project data attributes.
PDR = hours per FP (inverse of universal concept of
Productive)

24. Calculating indices based on industry data
Select a Peer Data set using attributes like programming
language, team size, industry, application type,
development method.
After determining the industry average metrics for the data
set, it becomes possible to calculate the team performance
against the industry average:
▪ Productivity Index: PDR compared to industry
▪ Cost Index: Cost Efficiency compared to industry
▪ Delivery Speed Index: Delivery Speed compared to
industry
▪ Process Index: Sprint Quality compared to industry
By doing this for each team, understanding team
performance against the industry becomes possible,
identifying high-performing and low-performing teams!
Also, it allows the comparison of teams based on these
indices!

25. Compare teams based on indices
Industry average
(different for each team)

27. Balancing metrics

28. Elevate Your Expertise in Software Measurement & Estimation
Join the Global Community of Nesma Professionals!
• Access a network of experts:
• Learn from experienced practitioners.
• Collaborate on projects and initiatives.
• Gain valuable resources:
• Free Research publications, white papers, case studies.
• Free Standardized measurement methods and guidelines (FPA standard).
• Large discounts on ISBSG data!
• Boost your career:
• Professional development opportunities & certifications.
• Increased industry recognition and credibility.
• Access and contribute to the ICEAA Software SIG.
• Expand your network:
• Connect with potential employers and clients on our free annual physical networking event!
• Contribute to shaping the future of the field.
• Stay informed:
• Gain insights into industry trends and challenge.
• Sign up now: Membership (fee: 125 EUR per Year).

29. Thank you for attending this webinar!
▪
Haroldvanheeringen
Become a Nesma member now! Annual fee is only €125
https://nesma.org/members/registration-form/
Connect with the Nesma community, including
free downloads of all digital products +
free access to a physical member meeting/network event!
large discounts on ISBSG data!
Nesma: http:// www.nesma.org
Harold.van.Heeringen@nesma.org