Self Healing blue/green Deployments with Dynatrace and Keptn

1. Self healing blue/green deployments with
keptn and Dynatrace

2. 2Confidential
• Move tocloud-nativearchitecturestomove faster
• Use agile development torelease moreoften
• Give developers moreoperationalresponsibility
• Build processesthatarebuilt oncollaboration
• Automationresultsin costefficiency
Observationsinourindustry

3. 3Confidential
•24xmorelikely tomakemonitoring andalertingconfigurationbyteams
•23xmorelikely toreuse deploymentpatterns
•44xmorelikely toreuse testing patterns
•44xmorelikely tocontributetootherteams’stooling
• 27xmorelikely to useconfigurationmanagementtools
WhatHigh-Performing CloudNative& DevOps Companiesdo well!
https://puppet.com/resources/whitepaper/state-of-devops-report

4. 4Confidential
• More releases and more complex systems are harder to
manage
• Agile delivery is not enough, if operations is manual
• Developers have responsibility but information
workflows are not in place (aaSthinking)
• People-centric processesare slow
Challenges

5. Confidential 5
How manyapps canyou manage?
Size
Unmanageablesize
Traditional operations
NoOps
Automation
AI
Effort

6. 6Confidential
• If you run 2 week sprints – do your release 25 times a year
• If you automate do you have a dev:ops relationship of
200:1
• Is your remediation as fast your deployment speed
• How long from a change untilthe data arrives at the
developer?
How maturearewe?

7. 7Confidential
• Release are held back from production
• Lock downperiods are introduced during critical
periods
• Team are overworkedand important projects
cancelled or delayed (unmanageable size)
• Developers don’t feel really responsible
Impactof lackofmaturity

8. 8Confidential
• Move towards massive automation (build the
enablement not the ”actualthing”)
• Monitoring part of continuous deployment
• Build“unbreakability” into the process
• Move from manual runbooks to “operations as code”
Thenew way

9. Autonomous Cloud Management with
keptn (0.1)

10. 10Confidential
• http://socks.weave.works
• OpenSource microservice app
• Polyglot
• Java
• Go
• Node.js
• RabbitMQ
• Database (Mongo, MySQL)
SockshopApplication

11. 11Confidential
SockshopArchitecture

12. 12Confidential
DemoSetup
Jenkins
GKE cluster
Developer
Docker Registry
GitHub
GitHub repositories
- catalogue
- carts
- front-end
- jmeter-tests
- ...
Sockshop App

13. 13Confidential
Workflow I

14. 14Confidential
Workflow II

15. Production Deployments

16. 16Confidential
ProductionDeployments
• From idea to demo-able feature

17. 17Confidential
ProductionDeployments
• From idea to demo-able feature production

18. 18Confidential
ProductionDeployments
• From idea to demo-able feature production adoption
• Deployment strategies
• Recreate
• Ramped
• Blue/Green
• Canary
• A/B testing
• Shadow
• Database changescan summondemons youdon’t need or want

19. 19Confidential
Deployment Strategies
https://thenewstack.io/deployment-strategies/

20. 20Confidential
DeploymentStrategy:Recreate
• Shutdown version A
• Deploy version B after Ais turnedoff
• Pros
• Easy to do
• Application state entirely renewed
• Cons
• Downtime – high impact on the user
https://thenewstack.io/deployment-strategies/

21. 21Confidential
DeploymentStrategy:Ramped
• aka Rolling upgrade
• Replace instances with new version one by one
• Pros
• Easy to setup
• New version is slowly released
• Stateful applications can rebalance data
• Cons
• No control over traffic

22. 22Confidential
DeploymentStrategy:Blue/Green
• Version B (green) is deployed alongside version A(blue)
• Traffic is switched to green at load balancer
• Blue is not immediately deleted
• Pros
• Instant rollout androllback
• Avoiding version conflicts
• Cons
• Difficult for stateful applications
• Long running transactions

23. 23Confidential
DeploymentStrategy:Canary
• Gradually shifting traffic from versionA to B
• E.g.: 90% A - 10%B for10 mins,then80% A - 20% B ...
• Controlled rollout witheasyrollback
• Criteriafor traffic distribution
• Pros
• Newversiononly releasedtoa subsetofusers
• Tryoutsunderproductionconditions
• Fast andeasyrollback
• Cons
• Difficultforstatefulapplications

24. 24Confidential
DeploymentStrategy:A/Btesting
• No deployment strategy but related
• Testing conversion ratesof features
• Criteriafor traffic distribution
• Pros
• Run several version in parallel
• Full trafficcontrol
• Cons
• Requires intelligent loadbalancer (L7)
• Hardtotroubleshoot

25. 25Confidential
DeploymentStrategy:Shadow
• Deploy versionB alongside versionA
• Mirrortrafficto versionB
• Go live whenquality requirementsaremet
• Pros
• TestingB underrealconditions
• Nouserimpact
• Controlledrollout
• Cons
• Expensive –twicethe resources
• Notatrueusertest
• May requiremocking services

26. DeploymentStrategies Overview

27. 27Confidential
Istio
• Open Source project initiated by Google andIBM
• Open services platform to manageservice interactions
• Service Mesh for cross cutting concerns
• Traffic routing
• Intelligent loadbalancing
• Security (encryption, authentication, access control)
• Circuit breaking
• Fault injection

28. 28Confidential
Istio -Architecture
• Pilot
• Controlplane toconfigure andpush service
communication policies
• Routing andforwardingpolicies
• Mixer
• Policy enforcement with aflexible plugin
model for providers fora policy
• Citadel
• Service-to-service authentication and
encryption using mutual TLS with built-in
identify andcredential management

29. 29Confidential
Istio Components
Gateway
VirtualService
DestinationRule
ServiceEntry

30. 30Confidential
Istio Components
Gateway
VirtualService
DestinationRule
ServiceEntry

31. 31Confidential
Istio Components
Gateway
VirtualService
DestinationRule
ServiceEntry

32. 32Confidential
Istio Components
Gateway
VirtualService
DestinationRule
ServiceEntry

33. 33Confidential
100%Trafficv1

34. 34Confidential
100%Trafficv1

35. 35Confidential
100%Trafficv1

36. 36Confidential
50/50TrafficDistribution v1 &v2

37. 37Confidential
Keptn.sh
• Feel free to tryit out at https://keptn.sh/
• Currentrelease: v0.1with “hardcoded” demo (sockshop)
• Release v0.2 is about to happen
• Strong focus on GitOps based CD
• Mainfeature: onboarding ofyourown apps
• Release v0.3:
• Pluggability, will enable you towrite yourown Services that can reactto certain events that occur in your CDpipeline