Enterprise Agility Europe 2019. Presentation from Shuman Ip from LoyaltyOne and Airmiles.

1. Enterprise Agility Europe 2019
How “Agile” Are We?
June 11, 2019

2. Who am I

3. Shuman Ip @ LoyaltyOne (Canada)
KMP, ESP
Associate Director, Agility and Lean
& Services Delivery Manager
Past Experience:
- Agile Coach
- Scrum Master
- Product Owner
- Developer
- Software Test Engineer

4. Agenda
• Business Agility
• Measure How “Agile”
• Metrics 1
• Metrics 2
• Metrics 3
• Discussion

5. Business
Agility

6. Question: How Agile are our businesses?
1. How important is Agility to the business?
2. How Agile is your business currently?

7. How Agile are our businesses per Deloitte and McKinsey?
More than 90% of senior executives
give high priority to becoming agile
While less than 10% see their firm as
currently highly agile
Sources:
https://www.forbes.com/sites/stevedenning/2019/05/23/understanding-fake-agile/amp/?fbclid=IwAR3Dzi69RuBNNopaR5qEcCgldaL0W9PjiqTvZogT2gUZAhO0tSfiOVossVE
https://www.mckinsey.com/business-functions/organization/our-insights/how-to-create-an-agile-organization
https://www2.deloitte.com/content/dam/Deloitte/lu/Documents/human-capital/lu-hc-2017-global-human-capital-trends-gx.pdf

8. Question: What does it mean to be Agile as a Business?

9. What does it mean to be Agile as a Business?
Business Agility is the ability of an
organization to rapidly adapt to
market and environmental changes

10. What are the indicators for Agility?

11. What are the indicators for Agility?

12. What are the indicators for Agility?
Team Outputs?

13. What are the indicators for Agility?
Team Outputs?

14. What are the indicators for Agility?
Team Outputs?

15. What are the criteria for the appropriate indicators for Agility?

16. What are the criteria for the appropriate indicators for Agility?
1. Customer Lens

17. What are the criteria for the appropriate indicators for Agility?
1. Customer Lens
2. Measure End-to-End Service Delivery

18. What are the criteria for the appropriate indicators for Agility?
1. Customer Lens
2. Measure End-to-End Service Delivery
3. Different Services have Different
Agility Expectations and Capability

19. Measuring
How “Agile
Are We

20. Exercise: Different Group has different identity

21. Feature: Fitness CriteriaWhich service are we inspecting?
There is no one size fit all expectations
and delivery capability
Each service has a different service
level expectation and their respective
delivery capability

22. Feature: Fitness CriteriaWho is our customer for this service?
In case you don’t know who your customer is…
Identify the group who is yelling the loudest and fastest
when the delivery of a service is not up to expectation

23. Feature: Fitness Criteria3 Metrics for Service Delivery Agility
1. Replenishment Frequency
2. Release Frequency
3. Lead Time

24. Feature: Fitness Criteria3 Metrics for Service Delivery Agility
1. Replenishment Frequency
2. Release Frequency
3. Lead Time

25. Replenishment
Frequency

26. Feature: Fitness CriteriaReplenishment Frequency

27. Feature: Fitness CriteriaReplenishment Frequency
Replenishment happens
when the delivery system
pulls in customer request(s)
as commitment(s)
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Replenishment Frequency

28. Payment
(Commit to fulfill Request)
Replenishment
Moonbucks Coffee
Queue
(Optional)

29. Feature: Fitness CriteriaReplenishment Frequency
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Replenishment Frequency
Replenishment vs. Business Agility
It shows how often we can adapt to
changes introduced by our customers
Agile does not mean delivering faster,
it means adapting to changes quick
enough

30. Feature: Fitness CriteriaReplenishment Frequency
How often do customers
expect us to react (take in
their request of this
particular work item type)?
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency

31. Feature: Fitness CriteriaReplenishment Frequency
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency
How often do we currently
take in customer request of
this particular work item type?

32. Feature: Fitness CriteriaReplenishment Frequency
Replenishment Frequency can
signal the customers to prepare
for their request in a ready-to-
consume state
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency

33. Feature: Fitness CriteriaLoyaltyOne Replenishment Example – Feature Work
Censored

34. Feature: Fitness CriteriaLoyaltyOne Replenishment Example – Feature Work
Category Notes
Goal Setting (What)
50% of new POS Loyalty Program
enrollments come from Digital Channels
Strategic Objective (Why)
Validate hypothesis: Our potential POS
customer-base is moving towards Digital
Workflow
See attached for current paper-based
workflow and ideal digital workflow
Business Scope Trim Divided Digital Enrollment into 3 Phases
Business Dependencies Client Services, Partner, Marketing
Analytics
User flow (look into drop off)
Enrollment Location (for future campaign)
Technical Feasibility Yes
Effort (hrs, days, wks, mths) Days
Technical Dependencies Dani’s team to work on front end tweaks
Technical Scope Trim N/A

35. Feature: Fitness CriteriaLoyaltyOne Replenishment Example – Feature Work
With an established Replenishment
Cadence, customers know when to
get the Feature Request form filled
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency

36. Feature: Fitness CriteriaExercise: Replenishment Frequency
Avg. Replenishment Rate = Number of Days
Number of Replenishment
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency
What is the expected and
current Replenishment
Frequency of your system?

37. Day Replenishment? Release?
1 √
2
3
4
5
6 √ √
7
8
9
10 √ √
11
12
13
14
15 √ √
16
17
18
19
20 √ √
21
22
23
24
25
26 √ √
27
28
29
30 √ √
Day Replenishment? Release?
31
32
33
34
35 √ √
36
37
38
39
40 √ √
41
42
43
44
45 √
46 √
47
48
49
50 √
51 √
52
53
54
55 √ √
56
57
58
59
60 √ √
1 2 3 4 5 6 7 8 9 10
Lead Time

38. Feature: Fitness CriteriaExercise: Replenishment Frequency
Avg. Replenishment Rate = Number of Days
Number of Replenishment
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency
What is the expected and
current Replenishment
Frequency of your system?

39. Feature: Fitness CriteriaReplenishment Frequency
Replenishment vs. Business Agility
It shows how often we can adapt to
changes introduced by our customers
Agile does not mean delivering faster,
it means adapting to changes quick
enough
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency

40. Feature: Fitness Criteria3 Metrics for Service Delivery Agility
1. Replenishment Frequency
2. Release Frequency
3. Lead Time

41. Release
Frequency

42. Feature: Fitness CriteriaRelease Frequency

43. Feature: Fitness CriteriaRelease Frequency
Release happens when the
customer has received the
service (product) as requested
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Release Frequency

44. Moonbucks Coffee
Medium Diablo
Coffee for David
&
Two Small Angel
Coffees for Ann Yep I am David.
Thanks!
Oh I am Ann.
Coming now
Coffee is Made
Customer has not received
Coffee Received
(Outcome realized)
Release

45. Feature: Fitness CriteriaRelease Frequency
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Release Frequency
Release vs. Business Agility?
It shows how often stakeholders
need to do release-related tasks, and
It shows how often customer can
adapt to changes introduced by us
Collaborate with Customer to
achieve outcomes instead of
pushing outputs to Customer

46. Feature: Fitness CriteriaRelease Frequency
How often do customers
expect to receive fulfilled
requests of this particular
work item type from us?
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Release Frequency

47. Feature: Fitness CriteriaRelease Frequency
How often do we currently
deliver completed request(s)
of this particular work item
type to customers?
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Release Frequency

48. Feature: Fitness CriteriaLoyaltyOne Release Example – Feature Work
Release Frequency can
signal customer and
stakeholders to prepare for
release-related tasks
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Release Frequency

49. Feature: Fitness CriteriaLoyaltyOne Release Example – Feature Work
Release Tasks Customer Internal Stakeholders
Deploy to Production √ (Ops)
Communication √ √ (Internal, Exec)
Training √ √
User Acceptance Test √ √ (User Research)
Add to Starter Kit √ (Dev)

50. Feature: Fitness CriteriaExercise: Release Frequency
What is the expected and
current Release Frequency of
your system?
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Release Frequency
Avg. Release Rate = Number of Days
Number of Release

51. Day Replenishment? Release?
1 √
2
3
4
5
6 √ √
7
8
9
10 √ √
11
12
13
14
15 √ √
16
17
18
19
20 √ √
21
22
23
24
25
26 √ √
27
28
29
30 √ √
Day Replenishment? Release?
31
32
33
34
35 √ √
36
37
38
39
40 √ √
41
42
43
44
45 √
46 √
47
48
49
50 √
51 √
52
53
54
55 √ √
56
57
58
59
60 √ √
1 2 3 4 5 6 7 8 9 10
Lead Time

52. Feature: Fitness CriteriaExercise: Release Frequency
What is the expected and
current Release Frequency of
your system?
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Release Frequency
Avg. Release Rate = Number of Days
Number of Release

53. Feature: Fitness CriteriaRelease Frequency
Release vs. Business Agility?
It shows how often stakeholders
need to do release-related tasks, and
It shows how often customer can
adapt to changes introduced by us
Collaborate with Customer to
achieve outcomes instead of
pushing outputs to Customer
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Release Frequency

54. Feature: Fitness CriteriaReplenishment and Release Frequencies
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Release Frequency
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency

55. Feature: Fitness CriteriaReplenishment and Release Frequencies
Replenishment and Release
have Transaction Costs
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Release Frequency
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency

56. Feature: Fitness Criteria3 Metrics for Service Delivery Agility
1. Replenishment Frequency
2. Release Frequency
3. Lead Time

57. Lead Time

58. Feature: Fitness CriteriaLead Time

59. Feature: Fitness CriteriaLead Time
Lead Time is the elapsed time between
Replenishment and Release
Slow
Fast
How fast?
Low
High
How Predictable?

60. Payment
(Commit to fulfill Request)
Replenishment
Moonbucks Coffee
Coffee Received
(Outcome realized)
Release
Lead Time
How much
sugar/ milk?
Which
coffee
bean?

61. Feature: Fitness CriteriaLead Time
Lead Time is not a mere number,
it is a Probabilistic Distribution
Slow
Fast
How fast?
Low
High
How Predictable?

62. Feature: Fitness CriteriaLead Time
Slow
Fast
How fast?
Low
High
How Predictable?
Lead Time vs. Business Agility
It answers the question “when will this be done?”
based on historical data
• End-to-end delivery
• Customer-recognizable service

63. Feature: Fitness CriteriaLead Time
Slow
Fast
How fast?
90 Days
Low
High
How Predictable?
75%
How fast do customers expect our delivery to be?
How predictable do customers expect our delivery to be?
E.g.
Customer expects 90 Days
delivery 75% of the time

64. Feature: Fitness CriteriaLead Time
How fast is our current delivery to customers’ expected confidence level?
How predictable is our current delivery to customers’ expected duration?
E.g.
We deliver to 90 days 57% of the time
We deliver to 106 days 75% of the time
Slow
Fast
How fast?
90 Days
106
Low
High
How Predictable?
75%
57%

65. Feature: Fitness CriteriaLoyaltyOne Release Example – Feature Work
Knowing the Lead Time allows customer to select
the appropriate class of service for each work item
Slow
Fast
How fast?
90 Days 85
Low
High
How Predictable?
75%82%
1
0
1 1
0 0
4
1 1
0
1
0 0 0 0 0 0
1 1
0
14 22 30 38 46 54 62 70 78 86 94 102 110 118 126 134 142 150 158 166
Itemcount(higherequalsmoreoften)
Cycle time value (calendar days from start to complete)

66. Feature: Fitness CriteriaLoyaltyOne Release Example – Feature Work
p actual estimated rounded days
5% 21 30.730 31
10% 27 38.397 38
15% 33 43.908 44
20% 43 48.434 48
25% 56 52.396 52
30% 56 56.000 56
35% 56 59.369 59
40% 58 62.584 63
45% 59 65.706 66
50% 60 68.783 69
55% 63 71.862 72
60% 67 74.988 75
65% 74 78.213 78
70% 82 81.600 82
75% 90 85.240 85
80% 123 89.269 89
85% 147 93.930 94
90% 152 99.732 100
95% 160 108.194 108
100% 170 170.000 170
Slow
Fast
How fast?
90 Days 85
Low
High
How Predictable?
75%82%

67. Feature: Fitness CriteriaExercise: Lead Time
What is the expected and current lead time of your system?
Low
High
How Predictable?
75%
Slow
Fast
How fast?
6 Days

68. Feature: Fitness CriteriaExercise: Lead Time
What is the expected and current lead time of your system?
1. Histogram (X-axis = Range of duration; Y-axis = # of work done in a particular duration)
Low
High
How Predictable?
75%
1 2 3 4 5 6 7 8 9 10 11 12 …
Lead Time (Days)
Slow
Fast
How fast?
6 Days

69. Feature: Fitness CriteriaExercise: Lead Time
Based on our historical data:
0% chance we can deliver in less than 3 days
100% chance we can deliver within 12 days
1 2 3 4 5 6 7 8 9 10 11 12 …
Lead Time (Days)

70. Feature: Fitness CriteriaExercise: Lead Time
What is the expected and current lead time of your system?
1. Histogram (X-axis = Range of duration; Y-axis = # of work done in a particular duration)
Low
High
How Predictable?
75%
Slow
Fast
How fast?
6 Days
1 2 3 4 5 6 7 8 9 10 11 12 …
Lead Time (Days)

71. Feature: Fitness CriteriaExercise: Lead Time
What is the expected and current lead time of your system?
1. Histogram (X-axis = Range of duration; Y-axis = # of work done in a particular duration)
2. Calculate Lead Time Duration per customer’s expected predictability (e.g. 75%)
Slow
Fast
How fast?
6 Days
Low
High
How Predictable?
75%
8.5
1 2 3 4 5 6 7 8 9 10 11 12 …
Lead Time (Days)

72. Feature: Fitness CriteriaExercise: Lead Time
What is the expected and current lead time of your system?
1. Histogram (X-axis = Range of duration; Y-axis = # of work done in a particular duration)
2. Calculate Lead Time Duration per customer’s expected Predictability (e.g. 75%)
3. Calculate Lead Time Predictability per customer’s expected Duration (e.g. 6 Days)
Slow
Fast
How fast?
6 Days
Low
High
How Predictable?
75%
8.5
65%
1 2 3 4 5 6 7 8 9 10 11 12 …
Lead Time (Days)

73. Feature: Fitness CriteriaExercise: Lead Time
What is the expected and current lead time of your system?
1. Troy Magennis’s http://focusedobjective.com/free-tools-resources/
2. Excel: =PERCENTILE.INC(array, desired percentile)
3. Excel: =PERCENTRANK.INC(array, desired duration)
Slow
Fast
How fast?
6 Days
Low
High
How Predictable?
75%
8.5
65%
1 2 3 4 5 6 7 8 9 10 11 12 …
Lead Time (Days)

74. Feature: Fitness CriteriaLead Time
Lead Time vs. Business Agility
It answers the question “when will this be done?”
based on historical data
• End-to-end delivery
• Customer-recognizable service
Slow
Fast
How fast?
90 Days
85
Low
High
How Predictable?
75%82%

75. Feature: Fitness CriteriaHow “Agile” is our Cost-Saving Feature Service in LoyaltyOne?
Slow
Fast
How fast?
90 Days
85
Low
High
How Predictable?
75%82%
Go-to-market Strategies
Current State
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Release Frequency
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency

76. Feature: Fitness CriteriaWhen do we use the Service Delivery Dashboard?
Customer Meetings
Service Delivery Review
Operations Review
Improvement Initiatives… etc.

77. Service Delivery Agility

78. Feature: Fitness CriteriaPresentation: What does your dashboard say?
Can you guess the
identity of your system?

79. What are the criteria for the appropriate indicators for Agility?
1. Unit of measure is “customer recognizable”
2. Represent the capability of an End-to-End delivery pipeline
3. One dashboard per service

80. Discussion

81. Feature: Fitness CriteriaDiscussion: How useful is this dashboard?
For Customers?
For the Service Delivery Group?
For Executives?
Any other groups?

82. Feature: Fitness CriteriaDiscussion: How to close the gaps in dashboard?
How do we close the gap between
customer expectation and delivery
system capability?

83. Feature: Fitness CriteriaHow to close the gaps in dashboard?
1. Turn off a customer segment

84. Feature: Fitness CriteriaBank Monthly Financial Report Dashboard
Slow
Fast
How fast?
30 Days
34
Low
High
How Predictable?
100%
73%
Go-to-market Strategies
Current State
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Release Frequency
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency

85. Feature: Fitness CriteriaHow to close the gaps in dashboard?
1. Turn off a customer segment
2. Manage Customer Demand and Expectations

86. Feature: Fitness CriteriaGaming Startup Feature Request Dashboard
Slow
Fast
How fast?
5 Days
6
Low
High
How Predictable?
75%
70%
Go-to-market Strategies
Current State
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Weekly
Release Frequency
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Daily
Replenishment Frequency

87. Feature: Fitness CriteriaHow to close the gaps in dashboard?
1. Turn off a customer segment
2. Manage Customer Demand and Expectations
3. Increase Delivery Capability

88. Feature: Fitness CriteriaHow to close the gaps in dashboard?
1. Analyze the Histogram tail

89. Feature: Fitness CriteriaAnalyze the Histogram Tail
Why were these
tickets delayed?
1 2 3 4 5 6 7 8 9 10 11 12 …
Lead Time

90. Feature: Fitness CriteriaAnalyze the Histogram Tail

91. Feature: Fitness CriteriaAnalyze the Histogram Tail
Special cause variation –
outside of normal fluctuation
+3 standard deviation
-3 standard deviation

92. Feature: Fitness CriteriaHow to close the gaps in dashboard?
1. Analyze the Histogram tail
2. Visualize Workflow

93. Feature: Fitness CriteriaVisualize Workflow

94. Feature: Fitness CriteriaVisualize Workflow
Do we have too much going
on in Design?
How long are the tickets in
Design for?
What is the flow efficiency?
Do we have too little going on
in UAT?

95. Feature: Fitness CriteriaHow to close the gaps in dashboard?
1. Analyze the Histogram tail
2. Visualize Workflow
3. Limit Work In Progress

96. Feature: Fitness CriteriaLimit Work In Progress
Avg. Lead Time =
Work In Progress
Avg. Delivery Rate
Little’s Law
Limiting WIP puts a constraint to the
system to lower the Avg. Lead Time

97. Feature: Fitness CriteriaHow to close the gaps in dashboard?
1. Analyze the Histogram tail
2. Visualize Workflow
3. Limit Work In Progress
4. Deferred Commitment

98. Feature: Fitness CriteriaDeferred Commitment
Option vs. Commitment
Are you treating everything
as Commitment?

99. Feature: Fitness CriteriaHow to close the gaps in dashboard?
1. Analyze the Histogram tail
2. Visualize Workflow
3. Limit Work In Progress
4. Deferred Commitment
5. Classes of Service

100. Feature: Fitness CriteriaClasses of Service
A service might need to offer
different classes of service to
satisfy different “urgency”

101. Feature: Fitness CriteriaHow to close the gaps in dashboard?
1. Analyze the Histogram tail
2. Visualize Workflow
3. Limit Work In Progress
4. Deferred Commitment
5. Classes of Service
6. Work Item Types

102. Feature: Fitness CriteriaWork Item Types
Demand Analysis:
Are we dividing our services too granularly?
Do we need to further split our work item types?

103. Feature: Fitness CriteriaHow to close the gaps in dashboard?
1. Analyze the Histogram tail
2. Visualize Workflow
3. Limit Work In Progress
4. Deferred Commitment
5. Classes of Service
6. Work Item Types
7. Staffing

104. Feature: Fitness CriteriaHypothesis
If we <the change / experiment>
we expect to see <impact to the context for change>
seen through <measurement and observations>

105. Feature: Fitness CriteriaHypothesis
Example:
If we decrease WIP Limit from 10 to 7
we expect to see a increase in lead time predictability
seen through the shape of our lead time histogram

106. Feature: Fitness Criteria
Slow
Fast
How fast?
90 Days
85
Low
High
How Predictable?
75%82%
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Release Frequency
Yearly
Quarterly
Monthly Weekly
Daily
Hourly
Monthly
Replenishment Frequency
106
57%
Go-to-market Strategies
Current State
Previous State
NEW System vs Old System

107. Feature: Fitness CriteriaAll components are interrelated
All components are interrelated
Intending to improve one area (e.g. Release Rate)
can have an impact on the other areas

108. Feature: Fitness CriteriaDiscussion: What about Throughput?
Why doesn’t this dashboard include
Throughput as a metrics?