Defining KPIs for use in Enterprise Services Planning and with Kanban systems. Understanding the difference between KPIs, Improvement Guides, and General Health Indicators. Understanding how KPIs drive behavior such as establishing multiple classes of service. Relating KPIs to evolutionary change. KPIs are Fitness Criteria Metrics with defined threashold values
Call now : 9892124323 Nalasopara Beautiful Call Girls Vasai virar Best Call G...
Defining Key Performance Indicators and Fitness Criteria for Purpose
1. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Learn to care about
what the customer
cares about
KPIs should shape
improvements to
service delivery
Enterprise Services
Planning
Defining Key Performance Indicators
Presenter
David J. Anderson
Swift Kanban
Webinar
29 April 2015
2. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Lean Kanban North America 2015
conference
• “Back to our roots”
• Implementing Kanban
• Looking to the future…
• Enterprise Services Planning
3. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
• 2 Days of Learning
Sessions
• Choose Your ½ Day
Workshops
• Risk Profiling – David J Anderson
• Cost of Delay – Don Reinertsen
• Project Management with Kanban
• Cynefin 101 for Portfolio Kanban
• Simple Probabilistic Forecasting
…and more!
• Becoming Data-Driven
• Objective Retrospectives
• Forecasting
• Enterprise Kanban & Lean Startup
• Scrumban
• Kanban Coaching
• Blockers for Improvement
• RBS Project Sizing
• Kanban Academic Research …
4. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Kanban experience reports
Ultimate
Software
• BazaarVoice
• web recommendations
app and mobile development
Including…
8. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Which system is fitter?
We don’t know!
System B is faster but without understanding customer
expectations, both may be fit enough
0
2
4
6
8
10
12
14
Lead Time (Days)
System A
Frequency
0
5
10
15
20
25
30
5 10 15 20 25 30 More
Lead Time in Days
System B
Frequency
Mean 17 days Mean 12 days
9. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Measuring delivery against expectation
0
2
4
6
8
10
12
14
Lead Time (Days)
System A
Frequency
0
2
4
6
8
10
12
Lead Time Expectation Spread (Days)
System A
Frequency
0
5
10
15
20
25
30
5 10 15 20 25 30 More
Lead Time in Days
System B
Frequency
0
5
10
15
20
25
30
35
40
45
-15 -10 -5 0 5 10 15 20 More
Lead Time Expectation Spread (Days)
System B
Frequency
Mean 17 days Mean 12 days
System B is clearly fitter!
System B delivers 5/7 within expectations
System A only delivers 3/7 within expectations
10. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
What makes a pizza delivery service
“fit for purpose” ?
Fitness criteria are metrics that
measure things customers
value when selecting a service
again & again
Delivery time
Quality
Predictability
Safety (or conformance to
regulatory requirements)
11. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Meet Neeta - a project manager
Neeta’s team are working late
(again)
Neeta needs to feed them with
pizza
What attributes do her team
care about in a pizza delivery
service?
• Delivery time =
approximately 1 hour
• Non-functional quality =
tasty & hot
• Functional quality (order accuracy) =
doesn’t matter if small mistakes are
made, geeks will eat any flavor of pizza
• Predictability =
+/- 30 minutes is acceptable
• Safety =
so long as health & safety in food
preparation is good, it’s fine
12. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Neeta is also a working mom!
Neeta gets home late.
Her kids are really
hungry and even though
she shouldn’t she
decides to order pizza for
them
What makes a pizza
delivery service
acceptable to her kids
age 4, 6, 9 & 11 years?
• Delivery time =
20 minutes
• Non-functional quality =
doesn’t matter too much, it’s pizza!!!
• Functional quality (order accuracy) =
it must be cheese pizza! No other flavor is
acceptable! (even if you take the pepperoni
off)
• Predictability =
+/- 5 minutes maximum!!!
• Safety =
only mommy worries about that stuff!
13. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
To be “fit for purpose” there is a
product component & a service
delivery component
We need to offer a selection of
different recipes which are tasty
& popular. However, we must
also deliver with speed &
predictability
Lesson
14. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Modern creative & knowledge
worker businesses often obsess
with product definition & strategy
Operational excellence and service
delivery excellence are often
overlooked or treated as inferior
management skills
Lesson
15. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Neeta has 2 identities –
Mother and Project Manager
Each of Neeta’s identities
represents a different market
segment for the pizza delivery
service
Traditional demographic &
income group segmentation does
not accurately capture the
context to understand
“fit for purpose”
Nor, for that matter, do
personas. As Neeta represents
two segments not just one
persona
Lesson
16. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Exercise – Understanding Fitness for Purpose
Pick a service with which you are
familiar
Consider what makes it “fit for
purpose?”
Which attributes make you select
the service, again and again?
What are your expectations for
each attribute?
Why?
What (business) risks drive your
expectations?
18. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Fitness criteria are metrics that measure
observable external outcomes
Fitness criteria are metrics that
measure things customers or
other external stakeholders value
Delivery time
Quality
Predictability
Safety (conformance to regulatory
requirements)
or metrics that qualitatively
assess actual outcomes such as
customer satisfaction
employee satisfaction
19. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Evolutionary change has no defined end point
Evolving
Process
Roll
forward
Roll
back
Initial
Process
Future process is
emergent
Evaluate
Fitness
Evaluate
Fitness
Evaluate
Fitness
Evaluate
Fitness
Evalua
Fitnes
We don’t know the
end-point but we do
know our emergent
process is fitter!
20. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Business Risks, Fitness Criteria & Classes of
Service should all align
Business risks are things which are uncertain that affect
the performance of our business such as nature of
demand, consistency of supply, delivery predictability,
seasonal windows of opportunity, time value of money
Classes of service offered should align with business risks
Metrics used to evaluate service delivery capability
should be fitness criteria that are derived from specific
business risks
For example, opportunity cost of delay requires us to
measure lead time and understand sensitivity to
schedule uncertainty
21. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Select Key Performance Indicators Carefully!
KPIs should be fitness criteria metrics with threshold
values that represent “good enough” – the level where
the service delivery is “fit for purpose”
KPIs should assess service delivery capability and
indicate fitness for purpose. In doing so, a KPI
indicates your likelihood of success – of surviving and
thriving - by adequately satisfying your customers?
KPIs should be recognizable by your customers as
something meaningful!
If your customer doesn’t recognize the metric it isn’t a
“key” “performance” indicator, it is some other kind of
metric
22. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Other Useful Metrics
Some other metrics are useful
Those which guide improvements
Those which indicate general health
Is your metric evaluating and guiding a specific
change to improve fitness of your business such as
an initiative to improve vendor response times?
Or, is it a general business health indicator such as
liquidity?
If neither of these, then it is a metric that you almost
certainly don’t need! It should be removed!
24. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Market Adoption Lifecycle Segmentation
Enthusiasts Early
Adopters
Early
Majority
Late
Majority
Laggards
Rate
Of
Market
Adoption
time
Moore’s
Chasm
Little
Chasm
Hip
Cool
Buggy
Community
development
Niche
Market
Features
Good func quality
Adequate non-func
quality
Permission
Giving
Early adopter
Exceptional
func and non-
func quality
Cost
Effective
Broad Features
Exceptional
func and non-
func quality
Low Cost
Easy Access
Forced adoption
Viewed as
taxation
Fit for purpose Fit for purposeChanges over time
25. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Customer Storytelling & Clustering
Tell stories about real
customers, their
motivation, what they
buy and why. Cluster
similar stories
Give each cluster a “nickname”
e.g.
• “All ins”
• “Aspirationals”
• “Bet hedgers”
• “Boy scouts”
26. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
You can’t just ask!
Neeta, how fast would
you like your pizza
delivered? How
predictable do you need
us to be with our delivery
estimate?
Customers will tend to tell
you they need better service
and more features than they
really need!
Would you pay more for
the things you say you
need and want?
No, probably not!
Believe what customers
actually do, do not believe
what they say they’ll do!
Actually behavior will vary
from declared intent!
27. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Who knows your customers?
Front line staff
Those who take and those
who deliver orders
Those who provide
“customer care”
Often the lowest paid staff
in a business
Often the highest turnover,
shortest tenured positions
And yet, they have the vital
information that enables
the business to survive,
thrive and compete
28. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Pizza boy knows Neeta’s Story!
Staff who meet customers
can be trained to learn
what matters to them
and why
Create ways to capture
customer stories or directly
involve customer facing staff
when defining customer
segments fitness criteria
29. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
GT car manufacturer story
A well known manufacturer
of GT cars determined
customers were prepared to
wait 21 months to take
delivery
They learned this by letting
delivery time slip to 27 months
and receiving cancellations and
customers switching to a rival
manufacturer
Determining fitness
criteria thresholds by
reducing service levels until
customer complaints rise
to dangerous levels isn’t a
“safe to fail” approach!
Damaging your brand,
your reputation and your
profitability is a strange
way to discover how to
be…
“fit for purpose!”
30. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Retarding customer service until
customers complain vehemently or
take their business elsewhere could
be damaging
• Undermines brand
• Damages reputation
• Loss of market share
• Loss of revenue
Probing for threshold values by
reducing service quality isn’t “safe to
fail”
Is it “safe to fail”?
We need general guidance
that allows us to probe
for fitness criteria
threshold values that is
“safe to fail”
If we can’t ask, and we can’t
allow service to decline until
complaints make the
threshold evident, what can
we do?
32. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Look for clusters or
patterns of demand, or
patterns of similar
expectations, or new
sources of demand that
may represent an
emerging segment
Probe with classes of service
Create a class of service to
respond to the believed new
segment
• Set service levels at or close to
anticipated threshold levels
33. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Probe with classes of service
Observe take up of class of service
• Is it over-used? (or abused?) If so
tighten qualification criteria
• Is it under-used? Consider removing
it
• Is it used but you fail to deliver to
expectations? Do people complain? If
no then consider removing it. You
are over promising
Fixed delivery date class of
service emerged this way.
Initially abused by
marketing, eligibility
criteria were tightened
up.
34. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Telecom Equipment Example
A platform maintenance
department at a telecom
equipment manufacturer
receives demand only from
internal application
departments…
… Each request is tagged with the
originating telco operator for
whom the request is being
implemented. Each operator is
given a lane on the kanban board
35. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Telecom Equipment Example
Imagine 3 American telco operators
with different strategic positions…
• Verizon value quality most
• Sprint value time-to-market
• Voicestream/T-Mobile USA values low
cost
Now design and offer 3
classes of service…
• High quality, tight “done”
criteria for each step
• Short lead time – pull priority,
looser “done” criteria
• Low cost – junior staff, lowest
priority compared to other
work
36. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Different lanes, different risks
Done
F
E
I
Engin-
eering
Ready
Deploy-
ment
Ready
G
D
GY
PB MN
10 ∞
P1
AB
Ongoing
Development Testing
Done Verification Acceptance
10 10
Verizon 10
10
Sprint
T-Mobile
10
DE
DA
Each lane represents a different
source of demand but also
different fitness criteria and
threshold values
Different classes of service and
different pull criteria policies are
defined for each lane providing
service levels tuned to the “fitness
for purpose” expectations of each
customer
37. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
To have confidence you are
offering a service that is “fit for
purpose”, you must offer different
classes of service
To serve more than one market
segment adequately, you must
offer a selection of classes of
service
Lesson
38. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Classes of service should align to
market segments and fitness
criteria (or stakeholders needs)
Lesson
KPIs cannot be general! They need to
be tied to customer expectations.
Different segments have different
expectations. Hence, different
threshold levels of the KPI
40. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
85% at
10 days
Mean
5 days
98% at
25 days
ChangeRequests
ProductionDefects
85% at
60 days
Mean
50 days
98% at
150 days
Median
45 days
Lead Time Distribution is a KPI
41. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Blocker Cluster Data Guides Improvements
Harvested blocker tickets over a
1 month period
Cluster blockers based on the
stories behind the delay
Each cluster represents a risk
Identify Likelihood, Total Impact
& Average Impact
Identify whether occurs in the
tail of the lead time distribution
Define risk reduction &
mitigation actions
http://www.klausleopold.com/2013/09/blocker-clusters-problems-are-not.html
42. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Avg. Lead Time
Avg. Delivery RateWIP
Pool
of
Ideas
Ready
To
Deliver
Cumulative Flow is a General Health Indicator
43. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Test
Ready
Flow Efficiency is a General Health Indicator
F
E
I
G
D
GY
PB
DE MN
P1
AB
Customer Lead Time
Waiting Waiting WaitingWorking
Ideas
Dev
Ready
5
Ongoing
Development Testing
Done
3 35
UAT
Release
Ready
∞ ∞
Flow efficiency % = Work Time x 100%
Lead Time
Working WaitingWorking
44. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
Liquidity is a General Health Indicator
The volume of pull transactions in a kanban system defines its liquidity
47. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
ESP Training Modules
Module 1 Portfolio Management (Day 1 & 2)
Strategy, fitness for purpose, KPIs, Cost of Delay, Scheduling,
Sequencing, Portfolio risk, risk hedging, risk profiling, aligning
strategy & capability, strategy review
Module 2 Enterprise Services (Day 3)
Understanding kanban systems, real options, upstream Kanban,
commitment & replenishment, lead time, chance vs assignable
cause variation
Module 3 Project & Demand Management (Day 4)
Demand analysis, demand shaping, capacity planning, project
forecasting, risk review, labor pool liquidity, workflow liquidity
Module 4 Real Options, Portfolios, Programs,
Dependencies & Scaling(Day 5)
Scaling Kanban, dependencies, visualizing dependencies, portfolio
Kanban, stand ups, service delivery reviews, ops reviews
48. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
ESP Training
ESP training is delivered on-premises with clients
around the world
To order an ESP training class, contact Wes Harris,
Commercial Director of David J. Anderson &
Associates, wes@djaa.com
50. sales@leankanban.com @leankanbanu Copyright Lean Kanban Inc.
About
David Anderson is an innovator in
management for 21st Century
professional services businesses. He
leads a training, consulting,
publishing and event planning
company dedicated to developing,
promoting and implementing new
management thinking & methods…
David has 30+ years experience in the high technology
industry starting with computer games in the early 1980’s.
He has led software organizations delivering superior
productivity and quality using innovative methods at large
companies such as Sprint and Motorola.
David defined Enterprise Services Planning and originated
the Kanban Method an adaptive approach to improved
service delivery. His latest book, published in June 2012, is,
Lessons in Agile Management – On the Road to Kanban.
David is also Chairman of Lean Kanban Inc., a business
operating globally, dedicated to providing quality training &
events that bring Kanban and Enterprise Services Planning to
a broad audience of professionals around the world.
Fitness criteria are metrics that measure things customer or other external stakeholders value such as delivery time, quality, predictability, conformance to regulatory requirements or metrics that value actual outcomes such as customer satisfaction or employee satisfaction
Fitness criteria are metrics that measure things customer or other external stakeholders value such as delivery time, quality, predictability, conformance to regulatory requirements or metrics that value actual outcomes such as customer satisfaction or employee satisfaction
Fitness criteria are metrics that measure things customer or other external stakeholders value such as delivery time, quality, predictability, conformance to regulatory requirements or metrics that value actual outcomes such as customer satisfaction or employee satisfaction
Fitness criteria are metrics that measure things customer or other external stakeholders value such as delivery time, quality, predictability, conformance to regulatory requirements or metrics that value actual outcomes such as customer satisfaction or employee satisfaction
Traditional change is an A to B process. A is where you are now. B is a destination. B is either defined (from a methodology definition) or designed (by tailoring a framework).
To get from A to B, a change agency* will guide a transition initiative to install destination B into the organization.
*either an internal SEPG or external consultants
Median is always less than the mean and lies between the mode and the mean. Median is less sensitive to the tail on distribution and hence less variable in the presence of assignable/special cause variation causing a long tail. However, it is the mean that is used in Little’s Law and therefore we do care about risks that affect the tail in the distribution when using Little’s Law to forecast.