This document discusses measuring outcomes using the Evidence Based Management framework. It begins by outlining some common pitfalls of measuring, then provides an overview of the EBM framework which focuses on continuously improving customer outcomes, organizational capabilities, and business results under uncertainty. The framework evaluates four key value areas - current value, unrealized value, time to market, and ability to innovate. The document provides examples of metrics for each value area and suggests starting with clear goals and custom measurements to inspect and adapt. It encourages attendees to take first steps to increase transparency on product value.

1. MEASURING OUTCOMES
WITH THE EVIDENCE BASED MANAGEMENT FRAMEWORK
Host: Sjoerd Kranendonk
Support: Kai Stevens
Twitter: @sjoerdly
Sjoerd@scrumfacilitators.nl
Feel free to connect on Linkedin!
KAI SJOERD

2. WHICH OF THESE PICTURES…
• …is the best match to how your team feels about
measurements?
A B C

3. WHICH OF THESE PICTURES…
• …is the best match to how your team feels about
measurements?
A B C
Nice! Let’s
see what
you’re
measuring
DANGER!
Gaming of
metrics
ahead!
Ouch! I
hope we
can help!

4. WHAT’S YOUR QUESTION?
What question would you like to get
answered about
Measuring Outcomes
and the
Evidence Based Management
framework??
Please share using the chat window.

5. WHAT WE WILL DISCUSS TODAY
Measuring Outcome with the Evidence Based
Management framework
• Pitfalls of measuring
• The concept – why is EBM a useful tool to Inspect & Adapt Product
Value?
• 3 Things you can do today – Getting started with EBM

6. PITFALLS OF MEASURING
Pitfall Bingo
• Top-down enforced metrics (often ‘best practice cookie cutter’)
• Comparing teams
• No support from wider organization / Teams have no access to
relevant data
• The measure becomes the goal
• The measure drives other unwanted behavior (law of unintended
consequences)
• Irrelevant or unclear metrics (for instance activity or output metrics)
• Set and forget
What to look out for and avoid

7. 5 METRICS TO BE WARY OF
Mostly harmless?
• Net Promoter Score
• School Grades
• Stock Prices
• Employee Engagement
• (Election) Polls
Learn more in the excellent book: Key Performance Illusions – Coen de Bruijn

8. WHAT ARE YOU CURRENTLY MEASURING?
Where do you measure?

9. WHAT ARE YOU CURRENTLY MEASURING?
Where do you measure?
We assume outputs lead
to certain outcomes!
Fact is, these pesky
humans act irrationally &
unpredictably.
Which is why it is
extremely important to
measure outcomes of
real use. And then
Inspect & Adapt to
maximize value.
For more, see LEAN UX (book) & various Scrum.org courses, like PSU, PSPO & PSPO-A

10. EVIDENCE BASED MANAGEMENT
Purpose: Provide a framework
that helps organizations to
continuously improve customer
outcomes, organizational
capabilities, and business results
under conditions of uncertainty
• There’s more than just paying attention
to four Key Value Areas (KVAs)
• A series of intermediate goals to reach
Strategic Goals
• Defining the Current State and desired
next target state (Intermediate Goal)
• Understanding what is valuable: focus on
Outcomes (also see Lean UX & PSU)
Purpose & overview of the framework
Learn more about EBM at: Scrum.org/EBM or download the guide in your language

11. EVIDENCE BASED MANAGEMENT
The value that the
product delivers today
1. How happy are users and
customers today? Is their
happiness improving or
declining?
2. How happy are your
employees today? Is their
happiness improving or
declining?
3. How happy are your investors
and other stakeholders
today? Is their happiness
improving or declining?
KVA: Current Value (CV)
Learn more about EBM at: Scrum.org/EBM or download the guide in your language

12. EVIDENCE BASED MANAGEMENT
The potential future value
that could be realized if the
organization met the needs
of all potential customers
or users
1. Can any additional value be
created by our organization in
this market or other markets?
2. Is it worth the effort and risk to
pursue these untapped
opportunities?
3. Should further investments be
made to capture additional
Unrealized Value?
KVA: Unrealized Value (UV)
Learn more about EBM at: Scrum.org/EBM or download the guide in your language

13. EVIDENCE BASED MANAGEMENT
The organization’s ability to
quickly deliver new
capabilities, services, or
products
1. How fast can the organization
learn from new experiments
and information?
2. How fast can you adapt based
on the information?
3. How fast can you test new ideas
with customers?
KVA: Time to Market (T2M)
Learn more about EBM at: Scrum.org/EBM or download the guide in your language

14. EVIDENCE BASED MANAGEMENT
The effectiveness of an
organization to deliver new
capabilities that might
better meet customer
needs
1. What prevents the organization
from delivering new value?
2. What prevents customers or
users from benefiting from that
innovation?
KVA: Ability to Innovate (A2I)
Learn more about EBM at: Scrum.org/EBM or download the guide in your language

15. APPENDIX: LOTS OF SUGGESTED METRICS
Key Value Measures (KVMs)
Learn more about EBM at:
Scrum.org/EBM
or download the guide in your language

16. MEASURING OUTCOME GAME
• Available in English, Dutch,
Chinese, Spanish &
Vietnamese
• Free download for self-
print
• Physical card decks
available (Profit for charity)
• Explore EBM and its KVMs
in an engaging way, discuss
improvement focus, or find
your own use
Explore EBM & metrics with your team(s)
Free download or buy a card deck via https://www.scrumfacilitators.nl/nl/products/
(many other Scrum Facilitators products available too)

17. SETTING CUSTOM MEASUREMENTS
1. Start with a clear Goal. What do
you want to achieve?
2. What to measure?
• Which behaviors do we want?
More, less or different?
3. How to measure, which indicator?
• Tried and tested (KVM examples)
or something custom?
4. Can we set limits or should we
not?
5. Use metrics, Inspect & Adapt
G.A.M.E = Goal, Actions, Metrics, Evaluation
Read more about the GAME approach to working with metrics

18. THE FIRST STEP TO IMPROVEMENT
... What will you do to increase
transparency on your Product Value?
Only have internal
(output/activity) measures?
Start by measuring Current
Value for new initiatives and
collaborate on this with teams
and Stakeholders
In our Slack community
practitioners share
questions, ideas &
struggles. Get advice
from certified trainer-
practitioners and the
wider community.
Have measurements as
targets? Refocus to make the
desired Outcome clear and
periodically evaluate if the
measures help drive us in the
right direction
Contact us if you
attended the webinar
but did not see/save the
Join link for the Slack
community!

19. JOIN A TRAINING FOR A DEEP DIVE
EBM & Measuring Outcomes are covered
in the following Scrum.org certified
trainings
• PSPO & PSPO-A
• PSMII
• PSU
• Training Schedule & info
https://www.scrumfacilitators.nl/professional-
scrum-training/
The Measuring Outcome game and other
free products can be found at:
• http://scrumfacilitators.nl/products
In our Slack community
practitioners share
questions, ideas &
struggles. Get advice
from certified trainer-
practitioners and the
wider community.
Contact us if you
attended the webinar
but did not see/save the
Join link for the Slack
community!

20. Your company is transitioning. In the global digital
transformation, it’s important to develop business agility
to be able to adjust to the ever-changing environment
with new or improved digital services.
Agile is a mindset with a collection of different values,
principles, frameworks and methodologies that can help
increase this organizational agility.
Scrum Facilitators works with highly experienced trainers
and coaches on different areas of agile. Everyone has a lot
of experience and works in the field with customers daily
to help them develop skills and experience around Scrum
and business agility.
HOW WE CAN HELP

21. Scrum Facilitators is a group of independent
professional trainers and coaches that work
with agile frameworks and practices every
day. They are experiences Product Owner,
Scrum Master, team or organizational coach.
We offer all available Scrum.org training as
official partner of the Scrum.org
Professional Training Network. We provide in
company and public training with certified
trainers.
We also combine LEGO® SERIOUS PLAY®
with Scrum: LEGO® SERIOUS Scrum with
certified LEGO® SERIOUS PLAY® facilitators.
WE STAND FOR
WIN WIN
WORKING WITH TWO TRAINERS
TRAINERS WITH EVERYDAY EXPERIENCE
COMMUNITIES
CHEE HONG HSIA JASPER ALBLAS KAI STEVENS SJOERD KRANENDONK ZIRYAN SALAYI
INSPECT AND ADAPT
THE TEAM

22. JASPER ALBLAS KAI STEVENS SJOERD KRANENDONK ZIRYAN SALAYI
CHEE HONG HSIA