How to measure the product value
Agile Business Day
September 14th 2019, Venezia
Ilaria Mauric
Head of design
@ilariamauric
Giovanni Puliti
Agile coach and Trainer
@giovannipuliti
489 €
~650 €
It’s undone
when it’s done.
It’s undone
when it’s done.
It’s undone
when it’s done.
It’s done when it’s done.
(aka “it works for me”)
Extended lifecycle
BUSINESSPROFITS
Development Introduction Growth Maturity Decline
TIME
from
backlog compiler
from “it’s done
when it’s done”
to
product strategist
to “create something
useful and valuable”
We need to measure
in a different way.
Direct external measurements
Indirect internal measurements
Indirect internal
measurements
Indirect internal measurement
Maintenance VS Dev
Indirect internal measurement
Lead time, cycle time, etc.
NUMBEROFACTIVITIES
DAYS, WEEKS, SPRINTS
Backlog
Requirements gathering
Development
Test
Deploy
No… I gave up by now…. When I call I’m kept on
hold, no one ever answers.
I don’t find you in the list…..
Did you call to reschedule your appointment?
We are here to answer every morning.
I don’t doubt it… but I hold indefinitely and
sometimes I got dropped.
You tell me if there’s another service
available every morning for 4 hourse….
I don’t think that a metric….
Direct external
measurements.
Getting out of the
building.
B2 C
B2 B2 C
B2 Health Service (C)2 U
More a KPI than a User?
‣ Patients per room
‣ Staff-to-patient ratio
‣ Bed or room turnover
‣ Patient Wait Times By Process Step
‣ Time Between Symptom Onset & Hospitalisation
‣ Average Insurance Claim Processing Time & Cost
‣ …
Source: https://www.clearpointstrategy.com/25-healthcare-metrics-kpis/
Define which data you want to use.
Define how to use them.
Be open to review them.
Qualitative design research
Through live observation, face-to face interviews and stories,
it aims to understand user’s behaviours, motivations, needs, pains,
emotions and real contexts.
Quantitative design research
Through numbers and graphs,
it gives a picture of the magnitudo of the “nitty-gritty” facts,
their distribution, their frequency, their tendency.
Qualitative
design research
Quantitative
design research
What do we need?
exploratory
WHY
innovation
evaluative
HOW
maintenance
SolutionProblem
We believe that improving the product water resistancy
will help patients using it when swimming or training.
KPI: product usage over summertime
today: -40% - goal: same as winter !
We believe that letting users editing carbos 

will improve ratings or reviews in the store
KPI: ratings and reviews on the App Store
today: 1,8 - goal: 2 !
Measuring UX in a scrum team, by Mr. David Travis at BAD Conference 2019
Takeaways
#1
Product is not making a project.
We need to ask ourselves (first with the
vision and the need profiling, but then
with measurement) if what we’re building
is useful for anybody
#2: PO Perspective
‣ PO is not just backlog compiler, or request decoder.
‣ A PO is a promoter, an investigator, an interviewer, a
strategist.
‣ When it’s done it is still undone (understand a change of
perspective is needed).
‣ Design during the setup phase (budgeting, design, staffing,.. ),
resources to define the metrics and then apply them
BACKLOG
ITERATIVE 

PRODUCT 

DEVELOPMENT
1
FEEDBACK
BACKLOG
ITERATIVE 

PRODUCT 

DEVELOPMENT
1
FEEDBACK
BACKLOG
ITERATIVE 

PRODUCT 

DEVELOPMENT
1
FEEDBACK
VISION BACKLOG
ITERATIVE 

PRODUCT 

DEVELOPMENT
2
FEEDBACK
VISION BACKLOG
ITERATIVE 

PRODUCT 

DEVELOPMENT
2
USER NEEDS
FEEDBACK
VISION BACKLOG
ITERATIVE 

PRODUCT 

DEVELOPMENT
USER NEEDS
2
FEEDBACK
VISION BACKLOG
ITERATIVE 

PRODUCT 

DEVELOPMENT
USER NEEDS
2
USER NEEDS
FEEDBACK
VISION BACKLOG
ITERATIVE 

PRODUCT 

DEVELOPMENT
3
DELIVERED

VALUE
MEASURE
#3: UX perspective
‣ be crystal clear about the business model
‣ declare the personas of the product and their role
(customers, users, stakeholders), but don’t ignore
edge cases
‣ declare your KPIs (business, product and team)
‣ declare what are the principles that you’ll put in the
product, regardless the KPIs
#3: UX perspective
‣ get out of the building, but don’t delegate: 

bring data inside and use them
‣ define how to measure the internalisation of the
research within your process, measure it like you
measure your agility
‣ never forget that you are not the user of the product
you’re working on.
References
[ITA] Product ownership e misurazione del valore del
prodotto
http://bit.ly/ABD19POUX
Quantitative versus Qualitativa Usability Testing
https://www.nngroup.com/articles/quant-vs-qual/
[BOOK] Observing the User Experience (2012)
https://www.amazon.it/Observing-User-Experience-
Practitioners-Research/dp/0123848695
Thank you! " #

How to measure the product value