How to Measure What Matters: What is a KPI and what makes a good one? Who should be involved in data driven decision making in your business? What tools do you need to start being data-driven? What should you measure? Next Steps & Best Practices

1. How to
Measure
What Matters
Shawna Tregunna
Director of Digital Services
Data Driven Business
January2020
$
$
$

2. Content
Data, not just for data scientists!
Tools of the data trade.
What should you measure?
Now what?
Next Steps & Best Practices
What is a KPI?
01
02
03
KPI stands for Key Performance Indicator,
but what is that and what makes a good
one?
Who in your organization should be
involved in being data driven and how?
What tools, processes and
foundations do you need to start being
data driven?
Just because you CAN measure something
doesn’t mean it’s important –and how do you
measure people?
So much data, so little time! How do we know
what to do based on the data, or now that we’re
data-driven how do we make decisions if we don’t
have data?
A quick checklist to see where you are in
becoming a data driven person, department
and company!
04
05
06
2

3. What is a
KPI?
3

4. Key Performance
Indicators (KPI)
A measurable value which
indicates how a company is
performing as related to
defined, measurable, business
goals.
Vanity Metrics
Measurable values which
are not tied to overarching
business goals.
4

5. Hard KPIs
Hard KPIs are generally
KPIs that are wholly
controlled within the system
or business.
Soft KPIs
Generally have an external or
subjective element; any KPI
that has an element of
“experience” “satisfaction”
“perception” “sentiment”
scores are soft KPIs.
5

6. Who should
be involved?
6

7. 7
Team (Together)
Individuals
Leadership
Management
Everyone

8. ASK YOURSELF
Does your entire company know what
the company goals are?
Questions
8

9. ASK YOURSELF
Does everyone in the company know
how they contribute to the company
goals?
Questions
9

10.  Show Your Work “Data
Democratization”
Transparency and access reinforce the value of
the metrics, if you hide some metrics or manager
metrics or financial metrics you devalue all
KPIs.
 Magic Number Three
No one should be responsible for more
than three metrics / KPIs. The fewer the
better.
 Money Talks
If you want people to pay attention to KPIs, tie their
compensation to them.
TeamStats
10
 Data Governance
Create policies on how to use data, how to keep it
safe, how do you track changes to data, how and
when to audit models and reporting methods, etc

11. If you have leaders that regularly (and publicly) make
decisions on gut feels, intuition or that go against the
data they have, without good rationalizations.
(Especially if those decisions don’t always turn out
well.) Your team wont value data either – or even try
to.
BUY
IN
Leaders should set
the example
11

12. Reviews
Even if your
organization isn’t
data driven – you can
and should be, for
your own sake.
When it comes to reviews, promotions,
job hunts and interviews if you can
show unequivocally how your efforts
moved the needle in relevant ways you
can truly articulate your value to an
organization.
12

13. Where do you
start?
13

14. Figure out what is important, what 1-3
metrics can you measure, that are not
vanity and mostly hard KPIs? Define
them; what’s the number and how do
you get it?
How are you performing now (or
historically), don’t base against the
ideal, base your actual
performance over time or projects,
etc.
The
Beginning
14
Define KPIs Capture Baseline

15. A benchmark is an external
metric you are comparing
against
A baseline is an internal metric
you are comparing against
Source
Don’t you mean
benchmark?
15

16.  Can use Benchmark
Not ideal because rarely do the
same conditions exist everywhere
 Absolute or Variable Target?
May be unrealistic and can
Shortchange yourself if you can
exceed, consider incremental %
change, x over x.
GoalSetting
 Change Performance Target
Shift baseline to new level (new
threshold for expected
performance)
 Performance Standard
Define the quality of the work
contributing to the KPI
16

17. Time for
planning
Investment
17
Time for analysis
& reporting
Time & budget for
training
If you’re sitting there saying this is a lot, I hardly have time for my own job… now
advocate for time (or staff) & budget.
Budget for
tools

18. Tracking
18

19. OneTruth
…that there is “one point
of truth” that everyone
agrees is the truth.
IT IS
IMPORTANT
19

20. WHAT DATA CAN’T DO
Data can’t make decisions for you.
People need to interpret the data then
decide on steps forward.
NowWhat
20

21. Go Big or … Go Small
When making decisions based on data you can go big – but you can also
go small; try small changes and iterate depending on the change
Fight Fear With Numbers
Humans are notorious for fighting against change but they usually can’t
argue with numbers, use your data to support change
Decisions
21

22. Change
Please
Being data driven will
rarely keep you at status-
quo.
BEWARE
22

23. What should
we measure?
23

24. I see so many companies focus data
efforts in one area (usually front line)
eg call centres all on call stats (but no
management, finance, satisfaction
scores, etc), Ecommerce (marketing,
sales but not production, logistics,
development, management),
Remember your KPIs on the other
side of the funnel; everyone seems
to be in support of marketing and to
some extent sales bearing the
burden of being data driven but what
about after you convert that sale?
Decide
24
Divergent Data Funnel to Hour Glass

25. Working capital, Operating cash
flow, Return on equity, Debt to
equity ratio, Inventory turnover,
Accounts receivable turnover,
Gross profit margin
Finance
Healthcare
Inpatient mortality rate, Bed
turnover, Readmission rate,
Average length of stay, Patient
satisfaction, Total operating
margin, Average cost per
discharge, Cash receipt to bad
debt, Claims denial rate
Product
Product related tickets
Customer satisfaction
Usage statistics (SaaS
products)
Customer acquisition cost,
Conversion rate of a particular
channel, Percentage of leads
generated from a particular channel,
Customer Churn, Dormant
customers, Average spend per
customer
Marketing
Retail
Gross margin (as a percentage of
selling price), Inventory turnover,
Sell-through percentage, Average
sales per transaction, Percentage
of total stock not displayed
R&D
Number of bugs, Length of
development cycle, App
usage
Examples
25

26. Bad
Examples Loans
1 2Netflix
A loan broker’s biggest product is
selling loans. It has a great team of
salesmen and they get bonuses
based on how many loans they sell
(Volume). The sales staff is highly
motivated to sell loans to anyone
and they will, in turn, sell many bad
loans that the bank will not collect,
and this will hurt profitability. –
Alejandro Martinez
Netflix awarded a $1 million prize to
a developer team in 2009 for an
algorithm that increased the
accuracy of the company's
recommendation engine by 10
percent. But it doesn't use the
million-dollar code and has no plans
to implement it in the future – the
cost to implement was too high and
didn’t align with all goals.
26

27. What if
there is no
data?
27

28. “ Count what is countable,
measure what is
measurable, and what is not
measurable, make
measurable.
Galileo Galilei
Wise
Words
28

29. Decision
Matrix
29

30. 30
Analyze data, decide what to do - small
changes
Identify your data – baseline & one source of
truth
Confirm your goals (company /
individual)Define your KPIs (max 3)
NextSteps
Use data to “sell” and manage change
Set audit schedule (goals, data, models,

31. Question
s?
31
Shawna Tregunna
Director of Digital Services
@ShawnaActually
Shawna@Iversoft.ca