Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous Deployment To The Cloud @DevoxxPL 2017

721 views

Published on

“I have stopped counting how many times I’ve done this from scratch” - was one of the responses to the tweet about starting the project called Spring Cloud Pipelines. Every company sets up a pipeline to take code from your source control, through unit testing and integration testing, to production from scratch. Every company creates some sort of automation to deploy its applications to servers. Enough is enough - time to automate that and focus on delivering business value.

In this presentation we’ll go through the contents of the Spring Cloud Pipelines project. We’ll start a new project for which we’ll have a deployment pipeline set up in no time. We’ll deploy to Cloud Foundry and check if our application is backward compatible so that we can roll it back on production.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

Continuous Deployment To The Cloud @DevoxxPL 2017

  1. 1. @mgrzejszczak Continuous Deployment to the Cloud Marcin Grzejszczak, @mgrzejszczak
  2. 2. 2 Spring Cloud developer at Pivotal Working mostly on ● Spring Cloud Sleuth ● Spring Cloud Contract ● Spring Cloud Pipelines About me Twitter: @mgrzejszczak Blog: http://toomuchcoding.com
  3. 3. 3 What problem are we trying to solve?
  4. 4. 4 The necessity to create a deployment pipeline from scratch for each new project
  5. 5. 5 Spring Cloud Pipelines • opinionated template of a deployment pipeline • based on good practices from different projects • we believe in the “Infrastructure as Code” approach • in case of automation server going down you can recover everything in no time • the automation server setup should be automated too! • you can even write tests for your pipelines
  6. 6. 6 Spring Cloud Pipelines • we support following automation servers out of the box • Concourse • Jenkins using Jenkins Job DSL plugin • Jenkins using Jenkinsfile with Blue Ocean • logic of all steps done via Bash scripts • you can convert the internals to suit your needs • you can use whatever automation server you want • supports Maven & Gradle
  7. 7. 7 Spring Cloud Pipelines - Concourse
  8. 8. 8 Concourse
  9. 9. 9 Spring Cloud Pipelines - Jenkins Job DSL
  10. 10. 10 Jenkins Job DSL
  11. 11. 11 Spring Cloud Pipelines - Jenkinsfile & Blue Ocean
  12. 12. 12 Jenkinsfile
  13. 13. 13 SPRING CLOUD PIPELINES REPOSITORY & AUTOMATION SERVER DEMO
  14. 14. 14 WHAT ARE WE GOING TO WORK WITH?
  15. 15. 15 Demo - flow GITHUB-WEBHOOK GITHUB-ANALYTICS AMQPHOOK
  16. 16. 16 Definitions • Build environment is a machine where the building of the application takes place. It’s a CI / CD tool worker • Test is an environment where you can deploy an application to test it. It doesn’t resemble production • Stage is an environment where you can deploy an application to test it. It does resemble production. Typically shared by multiple teams • Prod is a production environment where we want our tested applications to be deployed for our customers
  17. 17. 17 Why do we deploy software? • We’re paid for delivering business value • Features are done when they are deployed to production • It’s better to deploy frequently for faster feedback • It’s better to fail the deployment pipeline as fast as possible • Deployments should take place in a standardised, automated fashion • Your deployment pipeline should test rollback • That way you can perform A/B or zero downtime deployment to production
  18. 18. 18 Spring Cloud Pipelines • We’re paid for delivering business value • Features are done when they are deployed to production • It’s better to deploy frequently for faster feedback • It’s better to fail the deployment pipeline as fast as possible • Deployments should take place in a standardised, automated fashion • Your deployment pipeline should test rollback • That way you can perform A/B or zero downtime deployment to production
  19. 19. 19 Spring Cloud Pipelines
  20. 20. 20 Problem - slow feedback • Nobody wants to wait until the end of the pipeline to see that something is not working fine • We want to fail fast - even during build time • If integration is faulty • If our API is backward incompatible • There should be no need to wait until end to end tests are executed
  21. 21. 21 Solution - fail fast • We’re running unit and integration tests during build time • To test faulty integration we use Spring Cloud Contract for HTTP / messaging • Producer defines a contract • From the contract o tests are generated to see if the producer is not lying o stubs are generated for consumers to use • Consumer can reuse it without running the producer • Fail at build time if integration fails (fail fast!) • All stubs reside in Nexus / Artifactory (thus can be reused)
  22. 22. 22 CONTRACT TESTS DEMO (BREAKING CONSUMER)
  23. 23. 23 Initial contract (producer) Body contains username and repository
  24. 24. 24 Generated test (producer) Body contains username and repository
  25. 25. 25 Expected model (consumer)
  26. 26. 26 Passing contract test (consumer)
  27. 27. 27 Breaking contract (producer) Was username and repository and we’ve made a breaking change by converting those to user and repo
  28. 28. 28 New, generated test (producer) Was username and repository and we’ve made a breaking change by converting those to user and repo
  29. 29. 29 Installing broken stubs locally (producer) New stubs with backward incompatible changes installed in Maven local
  30. 30. 30 Broken contract test locally (consumer) Expected repository and username but got repo and user
  31. 31. 31 CONTRACT TESTS DEMO (BREAKING PRODUCER)
  32. 32. 32 Contract for deletion of issues
  33. 33. 33 Demo - backward incompatible API change GITHUB-ANALYTICS V1 with DELETE @ /issues Contracts V1 with DELETE @ /issues GITHUB-ANALYTICS V2 removed DELETE @ /issues Contracts V2 removed DELETE @ /issues GITHUB-ANALYTICS V2 removed DELETE @ /issues Contracts V1 with DELETE @ /issues
  34. 34. 34 Backward incompatible changes of the API We delete the contract and the DELETE endpoint
  35. 35. 35 Broken build Current implementation does not support old contracts
  36. 36. 36 Broken build
  37. 37. 37 Broken build
  38. 38. 38 Spring Cloud Pipelines • We’re paid for delivering business value • Features are done when they are deployed to production • It’s better to deploy frequently for faster feedback • It’s better to fail the deployment pipeline as fast as possible • Deployments should take place in a standardised, automated fashion • Your deployment pipeline should test rollback • That way you can perform A/B or zero downtime deployment to production
  39. 39. 39 Spring Cloud Pipelines
  40. 40. 40 STANDARDISATION GIVES LOWER SUPPORT COSTS AND MORE CHANCES OF PEOPLE ROTATION
  41. 41. 41 Solution - PaaS & tooling • Use a PaaS to standardise the way you deploy and monitor your software • Spring Cloud Pipelines uses Cloud Foundry [1] as a PaaS implementation • For the demo purposes we’re using PCF Dev [2] • Cloud Foundry abstracts the application governance from underlying infrastructure • you can deploy, scale, manage applications in the same way if CF is running on your laptop, Amazon, Google, Azure etc. • Database schema upgrade is done via tools like Flyway [3] or Liquibase [4] [1] https://www.cloudfoundry.org [2] https://pivotal.io/pcf-dev [3] https://flywaydb.org/ [4] http://www.liquibase.org/
  42. 42. 42 Deploying apps with CF https://www.cloudfoundry.org https://pivotal.io/pcf-dev $ cf push
  43. 43. 43 CF DEMO
  44. 44. 44 Pivotal Cloud Foundry Apps Manager Different spaces for different environments
  45. 45. 45 Pivotal Cloud Foundry Apps Manager Running application in production space
  46. 46. 46 Pivotal Cloud Foundry Apps Manager Audit events of what happened with your instances State of your instances Number of instance you want to run Limits for memory and disk
  47. 47. 47 Pivotal Cloud Foundry Apps Manager Bound services to the application. Credentials get injected automatically
  48. 48. 48 Solution - schema upgrade standardisation (Flyway example)
  49. 49. 49 DEMO OF CONVENTIONS IN THE CODE
  50. 50. 50 Spring Cloud Pipelines After the application got deployed to test environment • The database schema gets updated upon application startup • We run a handful of smoke tests to see if crucial parts of our application are working fine • We want to test if the app is properly packaged • The application is surrounded by stubs - no real integrations take place • Spring Cloud Contract Stub Runner Boot app is responsible for serving stubs
  51. 51. 51 Problem - rollback DB • Deployment pipelines should test whether the application can be rolled back • Rolling back database changes is extremely difficult • Especially if you deploy once every 6 months (the number of changes is gigantic) • How can you roll back a deletion of a column?
  52. 52. 52 Solution - application rollback • The easiest solution is… NOT TO DB ROLLBACK • Perform only backward compatible changes (always add data) • Or perform backward incompatible changes in a backward compatible way [1] • Roll back the application only (the JAR) • The application and DB changes need to be backward compatible • That way you can ensure that two applications (old / new versions) can run simultaneously at the same time [1] https://spring.io/blog/2016/05/31/zero-downtime-deployment-with-a-database
  53. 53. 53 Demo - backward incompatible DB change GITHUB-ANALYTICS V1 with repository DB V1 with repository GITHUB-ANALYTICS V2 with repo DB V2 with repo GITHUB-ANALYTICS V1 with repository DB V2 with repo
  54. 54. 54 BACKWARD INCOMPATIBLE DB CHANGE DEMO
  55. 55. 55 Demo - initial DB state
  56. 56. 56 Demo - initial state
  57. 57. 57 Demo - backward incompatible DB change
  58. 58. 58 Demo - backward incompatible DB change
  59. 59. 59 ROLLBACK DEMO
  60. 60. 60 Demo - first run
  61. 61. 61 Demo - deploying backward incompatible DB change
  62. 62. 62 Demo - errors in the app logs Old version can’t insert data to a missing DB column
  63. 63. 63 Spring Cloud Pipelines
  64. 64. 64
  65. 65. 65 Problem - end to end tests • It takes ages to run end to end tests • They are slow and brittle • QA department writes an E2E for every feature we have • E2E environment setup • one environment shared between all applications? • one environment per application? • Surrounding apps should be deployed in • production versions? • development versions?
  66. 66. 66 Solution - don’t do E2E? • Regardless of the time spent on QA / UAT you can still have bugs on production • Assuming that you ... • embrace failure • introduce monitoring of business KPIs • introduce alerting over the metrics • ensure that you can rollback on production • … you could stop doing any end to end tests
  67. 67. 67 Spring Cloud Pipelines • The whole step is optional • it’s left there cause the “no e2e tests” approach is controversial • Deploy to stage and running e2e tests are manual steps • you have to wait for your turn for the env • some manual work has to be done to purge stale data etc.
  68. 68. 68 Spring Cloud Pipelines • We’re paid for delivering business value • Features are done when they are deployed to production • It’s better to deploy frequently for faster feedback • It’s better to fail the deployment pipeline as fast as possible • Deployments should take place in a standardised, automated fashion • Your deployment pipeline should test rollback • That way you can perform A/B or zero downtime deployment to production
  69. 69. 69 Spring Cloud Pipelines
  70. 70. 70 Problem - deployment to production • We don’t want to deploy the application to production at night • We want to treat a production deployment like any other deployment • We’d like to be able to perform A/B testing and zero downtime deployment • We’d like to easily rollback when sth goes wrong
  71. 71. 71 Solution - CF + SC Pipelines • Our application has KPI monitoring in place • Alerting are set for KPIs • It has been tested that the application can be easily rolled back • Cloud Foundry can take care of zero downtime deployment
  72. 72. 72 Spring Cloud Pipelines • Deploy to prod deploys the pipeline version of the app to production next to the current production version • Complete switch over allows to delete the old instance and leave only the new one • Once deployed we tag the repo with prod/VERSION_NUMBER
  73. 73. 73 A/B ON CF DEMO
  74. 74. 74 Two versions running at the same time Two versions registered under same hostname Old instance New instance
  75. 75. 75 After complete switch over One app remains
  76. 76. 76 Demo - alerts GITHUB-WEBHOOK GITHUB-ANALYTICS AMQPPOST POLL
  77. 77. 77 Demo - alerts GITHUB-ANALYTICS DELETE POLL PUSH
  78. 78. 78 METRICS & ALERTS DEMO
  79. 79. 79 Insert some data to the service
  80. 80. 80 Grafana Satisfactory number of issues Threshold below which alerts will be sent Metric configuration
  81. 81. 81 Slack notifications
  82. 82. 82 Delete all issues
  83. 83. 83 Grafana - alert The metric went down to 0
  84. 84. 84 Slack notifications
  85. 85. 85 Spring Cloud Pipelines
  86. 86. 86 Customizing Spring Cloud Pipelines
  87. 87. 87 Customizing Spring Cloud Pipelines $ curl -LOk https://github.com/spring-cloud/spring-cloud-pipelines/archive/v1.0.0.M5.zip $ unzip v1.0.0.M5.zip $ cd spring-cloud-pipelines-v1.0.0.M5 $ git init $ # modify the pipelines to suit your needs $ git add . $ git commit -m "Initial commit" $ git remote add origin ${YOUR_REPOSITORY_URL} $ git push origin master
  88. 88. 88 Customizing Spring Cloud Pipelines
  89. 89. 89 Summary • Continuous Deployment allows you to continuously deliver business value • Spring Cloud Pipelines gives you OOB tooling to test your software via • unit and integration testing • contract testing • rollback testing • Spring Cloud Pipelines allows you to easily adjust the deployment pipeline to suit your company’s needs • Thanks to Cloud Foundry you can easily do A/B & zero downtime deployment
  90. 90. 90
  91. 91. 91 ▪ Github Analytics: https://github.com/spring-cloud-samples/github-analytics ▪ Github Webhook: https://github.com/spring-cloud-samples/github-webhook ▪ SC-Pipelines documentation: https://cloud.spring.io/spring-cloud-pipelines/ ▪ No end to end tests • https://testing.googleblog.com/2015/04/just-say-no-to-more-end-to-end-tests.html • http://www.alwaysagileconsulting.com/articles/end-to-end-testing-considered-harmful/ ▪ Prometheus on CF https://github.com/mkuratczyk/prometheus-on-PCF ▪ Prometheus for PCF Dev (Docker compose) https://github.com/vegasbrianc/prometheus ▪ Pivotal Web Services trial : https://run.pivotal.io/ ▪ PCF Dev (CF on your laptop) : https://docs.pivotal.io/pcf-dev/ Links
  92. 92. 92 Learn More. Stay Connected. ▪ Read the docs http://cloud.spring.io/spring-cloud-pipelines/ ▪ Talk to us on Gitter https://gitter.im/spring-cloud/spring-cloud-pipelines Twitter: twitter.com/springcentral YouTube: spring.io/video LinkedIn: spring.io/linkedin Google Plus: spring.io/gplus
  93. 93. 93 mgrzejszczak

×