Continuous Delivery can be a hard sell to product managers, even in small, healthy organisations. The benefits sound implausible, the costs high, and it’s really hard to understand all that it enables. However, CD has been hugely beneficial at the leading edge of product management. As deployment pipelines have taken root, there has also been a shift in how we see and measure user value. Take these trends together, and there’s been a giant leap forwards in how to prioritise work and judge its success. When our teams stopped throwing features into a black hole, and instead closed the loop with feedback, it had surprising knock-on effects, bad and good! Why did we choose it ? Because continuous delivery is not only about delivering fast with a good quality, that's also about delivering the right thing. Welcome in the continuous delivery of value.

1. Elizabeth Ayer
Closing the Loop
Product Management for
Continuous Delivery

2. Le Louvre?

5. @ElizAyer #FlowCon19
Kirkaldy stopped
bridges falling down

7. @ElizAyer #FlowCon19
Kirkaldy stopped
trains derailing

9. @ElizAyer #FlowCon19
by testing materials
and components

11. <image> Machine

12. <image> Machine

13. @ElizAyer #FlowCon19

14. @ElizAyer #FlowCon19
Could tell whether
something had flaws in
weeks rather than years

15. @ElizAyer #FlowCon19
This led to faster
delivery, lower cost of
maintenance, and safety

16. @ElizAyer #FlowCon19
Continuous Delivery

17. @ElizAyer #FlowCon19
This led to faster
delivery, lower cost of
maintenance, and safety

18. @ElizAyer #FlowCon19
Continuous Delivery:
Get useful software to
users quickly, safely,
and sustainably

19. @ElizAyer #FlowCon19
Version
Control
Continuous
Integration
Release
Management Production
Design, Dev,
Test, Review
Performance
User
Acceptance
Pre-production

20. @ElizAyer #FlowCon19
Version
Control
Continuous
Integration
Release
Management Production
Design, Dev,
Test, Review
Performance
User
Acceptance
Pre-production

21. @ElizAyer #FlowCon19
CD presents opportunities to
Product Managers

22. @ElizAyer #FlowCon19
PRODU

23. @ElizAyer #FlowCon19
From the Big Bang…

25. https://www.slideshare.net/indrisrozas/example-requirements-specification

26. @ElizAyer #FlowCon19
Then we got Agile!
Backlog
Software
Team Product Owner
Icons by freepik at flaticon.com

27. @ElizAyer #FlowCon19
PRODU

28. @ElizAyer #FlowCon19
… to Black Holes?
Backlog
Software
Team Product Owner
Real Users
Icons by freepik at flaticon.com

29. @ElizAyer #FlowCon19
… to Black Holes

30. @ElizAyer #FlowCon19
Signs of Black Hole Agile
• Done means coded/tested/released (not validated)
• Releases celebrated more than user success
• No feature ever fails
• Planning starts with feature ideas

31. @ElizAyer #FlowCon19
What lies
beyond the
black hole?

32. @ElizAyer #FlowCon19
Closing the Loop

33. @ElizAyer #FlowCon19
Version
Control
Continuous
Integration
Release
Management Production
Design, Dev,
Test, Review
Performance
User
Acceptance
Pre-production

34. @ElizAyer #FlowCon19
Closing the loop is easy
1. Describe the outcome
2. Ship software
3. Measure

35. @ElizAyer #FlowCon19

36. @ElizAyer #FlowCon19

37. @ElizAyer #FlowCon19
Objective-setting
frameworks
Objectives and Key Results (OKRs)
Also, 4 Disciplines of Execution (4DX), Key
Performance Indicators (KPIs), SMART goals,
Performance Measurement Process (PuMP),
Big Small Quick (BSQ), North Star
Framework…

38. @ElizAyer #FlowCon19
OKR example: Conference
From previous years, they’ve established categories that they believe correlate
with attendee satisfaction, and hence drive recommendation and repeat
attendance: Registration, Speaker Quality, and Lunch
Objective Keep user satisfaction with registration high
Key result 1 Get more than 90% of registrants into the
hall before first session.

39. @ElizAyer #FlowCon19
Product Managing for CD
#1
Align at the level of intent

40. @ElizAyer #FlowCon19
Product Managing for CD
#1
Align at the level of intent,
check and change course

41. @ElizAyer #FlowCon19
But it’s probably OK if
we don’t measure, right?

42. @ElizAyer #FlowCon19
But it’s probably OK if
we don’t measure, right?
NO

43. @ElizAyer #FlowCon19 http://ai.stanford.edu/~ronnyk/ExPThinkWeek2009Public.pdf

44. @ElizAyer #FlowCon19
Obviously value can go
down as well as up when
a “feature” is added

45. @ElizAyer #FlowCon19
https://www.slideshare.net/EhudUdiWaizer/ui-origin
https://www.slideshare.net/EhudUdiWaizer/ui-origin

46. @ElizAyer #FlowCon19
“We have a lot of
metrics, but it isn’t
changing anything.”

47. @ElizAyer #FlowCon19
The difference is
culture

48. @ElizAyer #FlowCon19
CULTURE is “a pattern of
shared basic assumptions
that the group learned …
that has worked well
enough to be considered
valid and, therefore, to be
taught to new members as
the correct way”
Edgar Schein
Organizational Culture and
Leadership

49. @ElizAyer #FlowCon19
But it’s probably OK if
we don’t measure, right?

50. @ElizAyer #FlowCon19
Product wins when we extend
the CD (DevOps) culture

51. @ElizAyer #FlowCon19
Feedback
Autom
ation
Build quality in
Shared
responsibility
https://www.martinfowler.com/bliki/DevOpsCulture.html
Autonomous
teams
No silos
Organizational Culture
Team Culture
DevOps Culture

52. @ElizAyer #FlowCon19
Feedback
Autom
ation
Build quality in
Shared
responsibility
Autonomous
teams
No silos
Organizational Culture
Team Culture
DevOps Culture
Product CultureValidation

53. @ElizAyer #FlowCon19

54. @ElizAyer #FlowCon19
Primary mechanisms
to embed culture (Schein)
• Attention
• Reactions to big events
• Resource & reward allocation
• Role modelling, teaching, coaching

55. @ElizAyer #FlowCon19
GOAL: KEEP THE GOAL IN MIND

56. @ElizAyer #FlowCon19

57. @ElizAyer #FlowCon19
Promoting a close-
the-loop culture
• Make objective and current results visible
• Normalise working in small batches
• Make time to collect and analyse results
• Act on what you learn

58. @ElizAyer #FlowCon19
CD presents challenges to
Product Managers

59. @ElizAyer #FlowCon19
Version
Control
Continuous
Integration
Release
Management Production
Design, Dev,
Test, Review
Performance
User
Acceptance
Pre-production

60. @ElizAyer #FlowCon19
Continuous Delivery
feedback cycles
Dev CI Production
Dave Farley, Private Communication

61. @ElizAyer #FlowCon19
Continuous Delivery
feedback cycles
Dev CI Production

62. @ElizAyer #FlowCon19

63. @ElizAyer #FlowCon19
Despite better dev practices,
teams were feeling disengaged,
and decision-making was slow.

64. @ElizAyer #FlowCon19
Product Managing for CD
#2
Empower your teams to avoid
being bottlenecked on product
decisions

65. @ElizAyer #FlowCon19
How we broke and fixed
team autonomy
The 'A' Word
Elizabeth Ayer
@ElizAyer

66. @ElizAyer #FlowCon19

67. @ElizAyer #FlowCon19 https://www.youtube.com/watch?v=OqmdLcyES_Q
Train in
product:
scope,
ordering,
timing,
thinking
commercially

68. @ElizAyer #FlowCon19
Feedback
Autom
ation
Build quality in
Shared
responsibility
Autonomous
teams
No silos
Organizational Culture
Team Culture
DevOps Culture
Product CultureValidation Empowerment

69. @ElizAyer #FlowCon19
Validation culture and
empowerment are not easy

70. @ElizAyer #FlowCon19
… but we can recognise the
challenges for what they are
and make progress together!

72. @ElizAyer #FlowCon19
Continuous Delivery

73. @ElizAyer #FlowCon19
Goal-setting

74. @ElizAyer #FlowCon19
Culture and Empowerment

76. Thank you!