You know that adopting Continuous Delivery is key to a high-performance company. You’ve read the books and are ready to build microservices in the cloud. Great! Let’s go back to the principles and see how to apply them in a cloud native environment. What used to be about shipping code to static servers, is now about quickly creating decoupled pipelines for new services that are readily wired up into the platform, and everything is driven by code. This talk will give concrete guidance for a world where autonomous teams continuously deploy many independent services and containers into an infrastructure that is dynamically created via APIs. Release without staging environment, but with confidence. Learn how to apply cloud native concepts to the delivery pipeline itself and how the concepts of continuous delivery need to be adjusted for microservices in the cloud. Hear real world examples, including how to QA in production.

1. Cloud native
Continuous Delivery
Christian Deger, christian@deger.eu, @cdeger
Continuous Lifecycle London, 17.05.2018

2. Cloud native Continuous Delivery
• Cloud Native Computing Foundation
• Container packaged
• Dynamically managed
• Microservices oriented
• My opinion
• Serverless, FaaS
• Managed services

3. Cloud native Continuous Delivery
• Bring changes into production
• Fast
• Reliable
• Repeatable
• Traceable
• In order to
• Get fast feedback
• Lower risk

4. Microservices

5. Microservices
Speed

6. Microservices
Speed
Scale the organization

7. Microservices
Speed
Fast local decisions
Scale the organization

8. Microservices
Speed
Fast local decisionsAutonomous teams
Scale the organization

9. Microservices
Speed
Fast local decisionsAutonomous teams
Scale the organization
Loosely coupled

10. Microservices
Speed
Fast local decisionsAutonomous teams
Strong boundaries
Scale the organization
Loosely coupled

11. Microservices
Speed
Fast local decisionsAutonomous teams
Strong boundaries
Scale the organization
Independent deployable
Loosely coupled

12. Microservices
Speed
Fast local decisionsAutonomous teams
Strong boundaries
Technology diversity
Scale the organization
Independent deployable
Loosely coupled

13. Development
“Change”
Dev

14. Development
“Change”
Operations
”Stability”
Ops

15. Development
“Change”
Operations
”Stability”
Dev and Ops silos

16. Development
“Change”
Operations
”Stability”
Dev and Ops silos

17. Cross-functional teams

18. Science and Continuous Delivery
Forsgren, Nicole and Humble, Jez, The Role of Continuous Delivery in IT and Organizational Performance (October 27, 2015). Forsgren, N., J. Humble
(2016). "The Role of Continuous Delivery in IT and Organizational Performance." In the Proceedings of the Western Decision Sciences Institute (WDSI)
2016, Las Vegas, NV. . Available at SSRN: http://ssrn.com/abstract=2681909 or http://dx.doi.org/10.2139/ssrn.2681909

19. Deployment Pipeline

20. Simplified deployment pipeline
Application code in
one repository per
service.

21. Simplified deployment pipeline
Application code in
one repository per
service.
CI
Deployment package
as artifact.

22. Simplified deployment pipeline
Application code in
one repository per
service.
CI
Deployment package
as artifact.
CD
Deliver package to
production

23. Code hosting is commodity
Application code in
one repository per
service.
CI
Deployment package
as artifact.
CD
Deliver package to
production

24. • GitHub, GitLab, Bitbucket, …
• Increased developer productivity
• Ecosystem: Apps and integrations
• Security
• Account Management, SSO, MFA
• git-secrets
• Backup using clone/fetch
Code hosting: Managed over self-hosted

25. One tool?
CI
CI/CD tool
with support for
deployment pipelines
CD
• Simpler
• Better overview

26. Two tools?
CI
CI tool
CD
CD tool
Artifact as
trigger/ handover
• Best tool for the job
• More complex

27. • Travis CI, CircleCI, GitLab CI, …
• Deploy agent needs access to production
• Use separate tools for CI and CD
• AWS Code*
• Definitely for OSS
Managed deployment pipelines?

28. • CD infrastructure should be the first task in a new project
• CD should not become a snowflake itself
• For disaster recovery you will need your CD infrastructure
• Aim for “CD as a service”
Automate CD infrastructure

29. • Containerized
• Isolated builds – bring your own agent
• Elastic agents
• Container as artifact
• Pipeline as code
• Declarative in service repository
• Fast and simple bootstrapping of new pipelines
• Avoid single, shared CI instance
New CI/CD practices

30. • Everything that used to be good practices
• No CI theatre
• Embrace deployment pipelines
• No smarts in the CI tool
Old CI/CD practices – Recap

32. Pets?

33. Cattle, not pets

34. Burgers, not cattle

35. Cloud native deployment pipeline
Application code and
infrastructure
specification in one
repository per service.

36. Cloud native deployment pipeline
Application code and
infrastructure
specification in one
repository per service.
CI
Deployment package
and infrastructure
declaration as
artifact.

37. Cloud native deployment pipeline
Application code and
infrastructure
specification in one
repository per service.
CI
Deployment package
and infrastructure
declaration as
artifact.
CD
1. Create or update
service infrastructure.

38. Cloud native deployment pipeline
Application code and
infrastructure
specification in one
repository per service.
CI
Deployment package
and infrastructure
declaration as
artifact.
CD
1. Create or update
service infrastructure.
2. New instances pull
down package and
start application.

39. No infrastructure monolith

40. • Follow microservices boundaries
• At least one stack per microservice
Decompose into Micro-Infrastructures

41. • Macro stack(s)
• Outputs parameters exported
• Keep it small, only things that don’t
change often
• No services
Macro-Infrastructure
• Network
• Security
• Bastion Host

42. • Services share macro stack
• Service stacks import parameters
• Service teams own service stack
• All services are in service stacks
Shared stack and service stacks

43. • Services have dependencies
• CD infrastructure
• Macro stack
• Base images (AMI, container)
• …
• But avoid explicit pipeline dependencies
• Try to reference pinned dependencies
Isolate deployment pipelines

44. Deployment

45. You build it,
you run it

46. How many environments?
V2V3
V6 V5
V4
V7
V5
V8
Enginee
r
CI Dev Staging
V1
V4
Prod

47. • Integrate in production
• Consumer-driven contract testing
• Reduce impact of failures
• MTTR over MTBF
• Monitoring
• Canary releases
• Rollbacks
• Semantic monitoring
No staging environment

48. • Separate code deployment from feature release
• Trunk-based development
• No long lived feature branches
Feature toggles

49. Feature toggles – release and experiment
• Product is in charge of releasing a feature
• Canary releases
• A/B testing

50. Immutable deployment patterns

51. Function as a Service - FaaS

52. Done 
Function as a Service - FaaS

53. Lifecycle of immutable servers/containers
Created
V3

54. Lifecycle of immutable servers/containers
Created
V3
Healthcheck
ok
V3

55. Lifecycle of immutable servers/containers
Created
V3
Healthcheck
ok
V3
Traffic from
load balancer
V3

56. Lifecycle of immutable servers/containers
V3
Created
V3
Healthcheck
ok
V3
Traffic from
load balancer
V3
Connections
drained

57. Lifecycle of immutable servers/containers
V3
Created
V3
Healthcheck
ok
V3
Traffic from
load balancer
V3
Terminated
V3
Connections
drained

58. Lifecycle of immutable servers/containers
V3
Created
V3
Healthcheck
ok
V3
Traffic from
load balancer
V3
Terminated
V3
Connections
drained
• No need for configuration management tools: Chef, Puppet, Ansible
• Patches/Security? Alert on base image age
• Simpler with stateless services

59. Rolling update
V3 V3 V3

60. Rolling update
V3 V3 V3 V4

61. Rolling update
V3 V3 V3 V4

62. Rolling update
V3 V3 V3 V4

63. Rolling update
V3 V3 V4V3

64. Rolling update
V3 V3 V4V3

65. Rolling update
V3 V3 V4

66. Rolling update
V3 V3 V4 V4

67. Rolling update
V3 V4 V4

68. Rolling update
V3 V4 V4 V4

69. Rolling update
V4 V4 V4

70. Rolling update
V4 V4 V4
• Only few additional resources required during deployment
• Takes some time

71. Blue/green
V3 V3 V3

72. Blue/green
V3 V3 V3 V4 V4 V4

73. Blue/green
V3 V3 V3 V4 V4 V4

74. Blue/green
V4 V4 V4V3 V3 V3

75. Blue/green
V4 V4 V4V3V3V3

76. Blue/green
V4 V4 V4V3V3V3
• Can keep drained instances for faster rollback

77. Blue/green
V4 V4 V4V3V3V3

78. Blue/green
V4 V4 V4
• Twice the resources required during deployment
• Faster deployment

79. Canary analysis
V3 V3 V3

80. Canary analysis
V3 V3 V3 V4
• Make explicit, automated canary analysis
• Error rate
• Latency
• Load
• Alternative: Feature toggle based canaries

81. • Existing service in production
Dark launches
Service
Client

82. • New service to be launched
Dark launches
Old New
Client

83. • Fork real traffic to new service and
discard response
• Monitor new service under real load
• Compare responses
• Fork on server or client side
Dark launches
Old New
Client

84. Wrapping it up

85. • Build isolation
• Independent pipelines
• Elasticity
• Everything as code
• Pipelines owned by teams
Recommendations for deployment pipeline

86. • Time from commit to production – delivery lead time
• Time to bootstrap a new service including the deployment pipeline
• Deployment frequency
• Mean time to recover – MTTR
Metrics

87. • “You build it, you deploy it, you run it”
• Embrace immutability
• Infrastructure follows microservices architecture
• Failures happen
• Reduce impact
• Fast detection
• Fast recovery
Important

88. Thank You!

89. • Regent's Rowing 8 by Jmf3333 [CC BY 3.0]
https://en.wikipedia.org/wiki/File:Regents_rowing.JPG
• Aquapark Aquacolors by Pantharei.2017 (Own work) [CC BY-SA 4.0]
https://commons.wikimedia.org/wiki/File:Aquapark_Aquacolors.jpg
• The Key of a Chamberlain by Niklitov [CC BY-SA 4.0]
https://commons.wikimedia.org/wiki/File:The_Key_of_a_Chamberlain_at_Kingdom_of_P
russia_Kalinigrad_Blindage_museum.JPG
• Beziers Fonseranes by Dedounet [CC BY-SA 1.0]
https://commons.wikimedia.org/wiki/File:Beziers_Fonseranes.jpg
Image attribution