Mettles journey towards throw away clusters (Cloud Native Nordics)
•
0 likes•138 views
Mettles journey towards throw-away-cluster describes Mettle's transition to using GitOps and ephemeral infrastructure. They moved from manually deploying to environments via Concourse to using Flux for automated deployments through Git. This provides rollbacks if failures occur and ensures deployments are consistent. Reliability is ensured through testing, validation, and monitoring successful production deploys which numbered 218 last month without any failures. While progress has been made, further work is needed on progressive delivery and failure handling to fully achieve their goal of a reliable, consistent platform.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Mettles journey towards throw away clusters (Cloud Native Nordics)
Similar to Mettles journey towards throw away clusters (Cloud Native Nordics) (20)
Recently uploaded
Recently uploaded (20)
Mettles journey towards throw away clusters (Cloud Native Nordics)
- 1. Mettles journey towards throw-away-cluster Steve Wade Platform Lead - Mettle @swade1987 | steven@stevenwade.co.uk
- 2. STEVE WADE ( @swade1987 ) Independent Consultant & Trainer Currently Platform Lead at Mettle Flux / Istio / CoreDNS contributor Blog: https://medium.com/@swade1987
- 6. Public cloud Open by default closed by exception Open source by default Kubernetes centric
- 9. ● Personal platform sandboxes ● Platform sandbox ● Development ● Staging ● Bug Bounty ● Production ● Tools … destined for some more!
- 10. A little history lesson (what Mettle’s platform used to look like) Our journey thus far What is left to do? Q & A Agenda
- 11. A look back at Mettle’s existing platform
- 14. Development
- 15. Staging
- 16. Production
- 21. Concourse
- 25. Concourse StagingDevelopment Production helm upgrade <microservice chart> --env-values.yaml
- 28. The journey …
- 30. The platform teams mission statement
- 31. The platform teams mission statement To provide an easily extensible, config driven, ephemeral platform with clear ownership and most importantly reliable and consistent.
- 32. Key Terraform - version controlled infrastructure Flux - provides us “rise of the phoenix” style deployments Kubebuilder - an SDK to create custom Kubernetes operators
- 33. The platform teams mission statement To provide an easily extensible, config driven, ephemeral platform with clear ownership and most importantly reliable and consistent.
- 34. AWS Setup
- 37. Infrastructure ● A dedicated repository for all our Terraform modules ● A number of dedicated Terraform roots (consuming modules at specific git SHAs) ● Every environment is built using a set of roots executed in a specific order.
- 43. The platform teams mission statement To provide an easily extensible, config driven, ephemeral platform with clear ownership and most importantly reliable and consistent.
- 44. How do we make the platform ephemeral?
- 45. GitOps!
- 46. kubectl apply -f
- 47. kubectl apply -f kubectl set image
- 48. kubectl apply -f kubectl set image helm upgrade
- 49. kubectl apply -f kubectl set image helm upgrade
- 50. kubectl apply -f kubectl set image git push helm upgrade
- 51. Flux: The GitOps Operator
- 54. HelmRelease
- 55. HelmRelease : Update Policy
- 56. HelmRelease : Update Policy
- 57. Flux: Secrets
- 58. Flux: Platform Engineering Resources
- 60. Let’s take a closer look!
- 67. How do we deploy through environments?
- 68. 68
- 69. 69 docker build
- 70. 70 dev- commit SHA docker build
- 71. docker build 71 dev- commit SHA docker push
- 72. docker build 72 dev- commit SHA docker push
- 73. docker build 73 dev- commit SHA docker push new image with tag dev-1234
- 76. 76 patches HelmRelease new image tag dev-1234
- 77. 77 patches HelmRelease kubectl apply -f new image tag dev-1234
- 78. 78 new image tag dev-1234 patches HelmRelease kubectl apply -f commits changes to git
- 81. 81 wait for update testing!! docker build / push stg- commit SHA
- 82. 82
- 83. 83
- 87. How does the cluster come up?
- 88. AWS
- 91. Kubernetes resources - --git-url=git@github.com:eeveebank/kubernetes-resources - --git-branch=master - --git-path=environments/dev - --git-poll-interval=30s - --git-sync-tag=flux-sync-dev
- 92. Kubernetes resources - --git-url=git@github.com:eeveebank/kubernetes-resources - --git-branch=master - --git-path=environments/dev - --git-poll-interval=30s - --git-sync-tag=flux-sync-dev
- 99. The platform teams mission statement To provide an easily extensible, config driven, ephemeral platform with clear ownership and most importantly reliable and consistent.
- 100. How do we make sure it’s all reliable?
- 103. The platform teams mission statement To provide an easily extensible, config driven, ephemeral platform with clear ownership and most importantly reliable and consistent.
- 104. What has GitOps given us?
- 105. Number of production deploys last month:
- 106. Number of production deploys last month: 218
- 107. Successful production deploys last month:
- 108. Successful production deploys last month: 218
- 110. What is left to do?
- 111. Progressive delivery
- 112. Summary …
- 113. Summary … Having a consistent naming convention is key!
- 114. Summary … Having a consistent naming convention is key! Always think pull-based rather than push
- 115. Summary … Having a consistent naming convention is key! Always think pull-based rather than push Failure will happen we all need to be ready!
- 116. Useful Links - https://github.com/kubernetes-sigs/kustomize - https://github.com/fluxcd/flux - https://github.com/fluxcd/helm-operator - https://github.com/swade1987/deprek8ion - https://github.com/swade1987/gitops-with-kustomize (example repo) - https://medium.com/@swade1987 (lots more detail in these)
- 118. Thank You Happy to take questions …