Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Continuous Deployment to the cloud

256 views

Published on

SpringOne Platform 2017
Marcin Grzejszczak, Pivotal; Cora Iberkleid, Pivotal

"“I have stopped counting how many times I’ve done this from scratch” - was one of the responses to the tweet about starting the project called Spring Cloud Pipelines. Every company sets up a pipeline to take code from your source control, through unit testing and integration testing, to production from scratch. Every company creates some sort of automation to deploy its applications to servers. Enough is enough - time to automate that and focus on delivering business value.

In this presentation we’ll go through the contents of the Spring Cloud Pipelines project. We’ll start a new project for which we’ll have a deployment pipeline set up in no time. We’ll deploy to Cloud Foundry and check if our application is backwards compatible so that we can roll it back on production."

Published in: Technology
  • Be the first to comment

Continuous Deployment to the cloud

  1. 1. Continuous Deployment of your Application By Cora Iberkleid @ciberkleid 1 By Marcin Grzejszczak @mgrzejszczak
  2. 2. Senior Platform Architect at Pivotal Working on ● Pivotal Cloud Foundry ● Cloud Native Architecture ● Dev Practices Modernization Twitter: @ciberkleid About us
  3. 3. Spring Cloud developer at Pivotal Working on ● Spring Cloud Sleuth ● Spring Cloud Contract ● Spring Cloud Pipelines About us Twitter: @mgrzejszczak Blog: http://toomuchcoding.com
  4. 4. What problem are we trying to solve?
  5. 5. The necessity to create a deployment pipeline from scratch for each new project
  6. 6. Spring Cloud Pipelines • opinionated template of a deployment pipeline • based on good practices from different projects • we believe in the “Infrastructure as Code” approach • in case of automation server going down you can recover everything in no time • the automation server setup should be automated too! • you can even write tests for your pipelines
  7. 7. Spring Cloud Pipelines • we support following automation servers out of the box • Concourse • Jenkins using Jenkins Job DSL plugin • Jenkins using Jenkinsfile with Blue Ocean • logic of all steps done via Bash scripts • you can convert the internals to suit your needs • you can use whatever automation server you want • supports Maven & Gradle • Over 150 Bash tests (using https://github.com/sstephenson/bats)
  8. 8. Spring Cloud Pipelines
  9. 9. Spring Cloud Pipelines
  10. 10. Spring Cloud Pipelines
  11. 11. Spring Cloud Pipelines
  12. 12. Spring Cloud Pipelines
  13. 13. Spring Cloud Pipelines - PaaS Types
  14. 14. Spring Cloud Pipelines - CF example
  15. 15. Spring Cloud Pipelines - Concourse
  16. 16. Concourse
  17. 17. Spring Cloud Pipelines - Jenkins Job DSL
  18. 18. Jenkins Job DSL
  19. 19. Spring Cloud Pipelines - Jenkinsfile & Blue Ocean
  20. 20. Jenkinsfile
  21. 21. WHAT ARE WE GOING TO WORK WITH?
  22. 22. Demo - flow GITHUB-WEBHOOK GITHUB-ANALYTICS AMQPHOOK
  23. 23. Why do we deploy software? • We’re paid for delivering business value • Features are done when they are deployed to production • It’s better to deploy frequently for faster feedback • It’s better to fail the deployment pipeline as fast as possible • Deployments should take place in a standardised, automated fashion • Your deployment pipeline should test rollback • That way you can perform A/B or zero downtime deployment to production
  24. 24. Spring Cloud Pipelines • We’re paid for delivering business value • Features are done when they are deployed to production • It’s better to deploy frequently for faster feedback • It’s better to fail the deployment pipeline as fast as possible • Deployments should take place in a standardised, automated fashion • Your deployment pipeline should test rollback • That way you can perform A/B or zero downtime deployment to production
  25. 25. Spring Cloud Pipelines
  26. 26. Problem - slow feedback • Nobody wants to wait until the end of the pipeline to see that something is not working fine • We want to fail fast - even during build time • If integration is faulty • If our API is backward incompatible • There should be no need to wait until end to end tests are executed
  27. 27. Solution - fail fast • We’re running unit and integration tests during build time • To test faulty integration we use Spring Cloud Contract for HTTP / messaging • Producer defines a contract • From the contract o tests are generated to see if the producer is not lying o stubs are generated for consumers to use • Consumer can reuse it without running the producer • Fail at build time if integration fails (fail fast!) • All stubs reside in Nexus / Artifactory (thus can be reused)
  28. 28. CONTRACT TESTS DEMO (BREAKING CONSUMER)
  29. 29. Initial contract (producer) Body contains username and repository
  30. 30. Generated test (producer) Body contains username and repository
  31. 31. Expected model (consumer)
  32. 32. Passing contract test (consumer)
  33. 33. Breaking contract (producer) Was username and repository and we’ve made a breaking change by converting those to user and repo
  34. 34. Installing broken stubs locally (producer) New stubs with backward incompatible changes installed in Maven local
  35. 35. Broken build (consumer)
  36. 36. Broken contract test (consumer)
  37. 37. Broken contract test(consumer) Expected repository and username but got repo and user
  38. 38. CONTRACT TESTS DEMO (BREAKING PRODUCER)
  39. 39. Demo - backward incompatible API change GITHUB-ANALYTICS V1 with DELETE @ /issues Contracts V1 with DELETE @ /issues Generated test sends DELETE to /issues
  40. 40. Contract for deletion of issues
  41. 41. Generated test for deleting issues
  42. 42. Demo - backward incompatible API change GITHUB-ANALYTICS V1 with DELETE @ /issues Contracts V1 with DELETE @ /issues Generated test sends DELETE to /issues GITHUB-ANALYTICS V2 removed DELETE @ /issues Contracts V2 removed DELETE @ /issues No generated test for this endpoint
  43. 43. Backward incompatible changes of the API We delete the contract and the DELETE endpoint
  44. 44. Demo - backward incompatible API change GITHUB-ANALYTICS V1 with DELETE @ /issues Contracts V1 with DELETE @ /issues Generated test sends DELETE to /issues GITHUB-ANALYTICS V2 removed DELETE @ /issues Contracts V2 removed DELETE @ /issues No generated test for this endpoint GITHUB-ANALYTICS V2 removed DELETE @ /issues Contracts V1 with DELETE @ /issues Generated test sends DELETE to /issues
  45. 45. Broken build Current implementation (V2) does not support old contracts (V1)
  46. 46. Broken - generated test from old contracts (V1)
  47. 47. Broken - generated test from old contracts (V1)
  48. 48. Spring Cloud Pipelines • We’re paid for delivering business value • Features are done when they are deployed to production • It’s better to deploy frequently for faster feedback • It’s better to fail the deployment pipeline as fast as possible • Deployments should take place in a standardised, automated fashion • Your deployment pipeline should test rollback • That way you can perform A/B or zero downtime deployment to production
  49. 49. Spring Cloud Pipelines
  50. 50. STANDARDISATION GIVES LOWER SUPPORT COSTS AND MORE CHANCES OF PEOPLE ROTATION
  51. 51. Solution - PaaS & tooling • Use a PaaS to standardise the way you deploy and monitor your software • Spring Cloud Pipelines uses Cloud Foundry [1] or Kubernetes [2] as a PaaS implementation • For the demo purposes we’re using PCF Dev [3] or Minikube [4] • PaaS abstracts the application governance from underlying infrastructure • you can deploy, scale, manage applications in the same way if PaaS was running on your laptop, Amazon, Google, Azure etc. • Database schema upgrade is done via tools like Flyway [5] or Liquibase [6] [1] https://www.cloudfoundry.org, [2] https://kubernetes.io/ [3] https://pivotal.io/pcf-dev, [4] https://github.com/kubernetes/minikube [5] https://flywaydb.org/, [6] http://www.liquibase.org/
  52. 52. Spring Cloud Pipelines
  53. 53. Spring Cloud Pipelines After the application got deployed to test environment • The database schema gets updated upon application startup • We run a handful of smoke tests to see if crucial parts of our application are working fine • We want to test if the app is properly packaged • The application is surrounded by stubs - no real integrations take place • Spring Cloud Contract Stub Runner Boot app is responsible for serving stubs
  54. 54. Problem - rollback DB • Deployment pipelines should test whether the application can be rolled back • Rolling back database changes is extremely difficult • Especially if you deploy once every 6 months (the number of changes is gigantic) • How can you roll back a deletion of a column?
  55. 55. Spring Cloud Pipelines
  56. 56. Solution - application rollback • The easiest solution is… NOT TO DB ROLLBACK • Perform only backward compatible changes (always add data) • Or perform backward incompatible changes in a backward compatible way [1] • Roll back the application only (the JAR) • The application and DB changes need to be backward compatible • That way you can ensure that two applications (old / new versions) can run simultaneously at the same time [1] https://spring.io/blog/2016/05/31/zero-downtime-deployment-with-a-database
  57. 57. BACKWARD INCOMPATIBLE DB CHANGE DEMO
  58. 58. Demo - backward incompatible DB change GITHUB-ANALYTICS V1 CODE with repository DB V1 with repository
  59. 59. Demo - initial DB state (Flyway)
  60. 60. Demo - first run
  61. 61. Demo - initial state
  62. 62. Demo - backward incompatible DB change GITHUB-ANALYTICS V1 CODE with repository DB V1 with repository GITHUB-ANALYTICS V2 CODE with repo DB V2 with repo
  63. 63. Demo - backward incompatible DB change
  64. 64. Demo - backward incompatible DB change
  65. 65. Demo - backward incompatible DB change GITHUB-ANALYTICS V1 CODE with repository DB V1 with repository GITHUB-ANALYTICS V2 CODE with repo DB V2 with repo GITHUB-ANALYTICS V1 CODE with repository DB V2 with repo
  66. 66. Demo - backward incompatible DB change New application (V2) works against new database (V2)
  67. 67. Demo - backward incompatible DB change Old application (V1) can’t work with new database (V2) Deploy (V1) and connect to the new database (V2)
  68. 68. Demo - errors in the app logs Old version can’t insert data to a missing DB column
  69. 69. Spring Cloud Pipelines
  70. 70. Problem - end to end tests • End to end tests are slow and brittle • QA department writes an E2E for every feature we have • E2E environment setup • one environment shared between all applications? • one environment per application? • Surrounding apps should be deployed in • production versions? • development versions?
  71. 71. Solution - don’t do E2E? • Regardless of the time spent on QA / UAT you can still have bugs on production • Assuming that you ... • embrace failure • introduce monitoring of business key performance indicators (KPIs) • introduce alerting over the metrics • ensure that you can rollback on production • … you could stop doing any end to end tests
  72. 72. Spring Cloud Pipelines • Deploy to stage and running e2e tests are manual steps • you have to wait for your turn for the env • some manual work has to be done to purge stale data etc.
  73. 73. Spring Cloud Pipelines • We’re paid for delivering business value • Features are done when they are deployed to production • It’s better to deploy frequently for faster feedback • It’s better to fail the deployment pipeline as fast as possible • Deployments should take place in a standardised, automated fashion • Your deployment pipeline should test rollback • That way you can perform A/B or zero downtime deployment to production
  74. 74. Spring Cloud Pipelines
  75. 75. Problem - deployment to production • We don’t want to deploy the application to production at night • We want to treat a production deployment like any other deployment • We’d like to be able to perform A/B testing and zero downtime deployment • We’d like to easily rollback when sth goes wrong
  76. 76. Solution - PaaS + SC Pipelines • Our application has KPI monitoring in place • Alerting are set for KPIs • It has been tested that the application can be easily rolled back • PaaS (CF or K8S) can take care of zero downtime deployment
  77. 77. Spring Cloud Pipelines • prod-deploy deploys the pipeline version of the app to production next to the current production version • Once deployed we tag the repo with prod/VERSION_NUMBER • prod-complete allows to stop the old instance and leave only the new one running • prod-rollback deletes the new instance, removes the prod/VERSION_NUMBER tag and leaves only the old one running
  78. 78. A/B ON CF DEMO
  79. 79. Two versions running at the same time Two versions registered under same hostname Old instance New instance
  80. 80. MIGRATING TO SPRING CLOUD PIPELINES DEMO
  81. 81. Application Refactoring for Spring Cloud Pipelines A Story in Three Acts
  82. 82. Premise
  83. 83. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Main Characters 84 Greeting UI Fortune Service Fortune DB Simon
  84. 84. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Supporting Characters 85 The SCS Triplets and their pet Rabbit Greeting UI Fortune Service Fortune DB Simon
  85. 85. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ The Setting
  86. 86. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 87 Greeting UI Simon
  87. 87. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 88 Greeting UI Simon Hey Greeting!
  88. 88. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 89 Greeting UI The SCS Triplets and their pet Rabbit Simon Hey Greeting!
  89. 89. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 90 Greeting UI The SCS Triplets and their pet Rabbit ??? Simon Hey Greeting!
  90. 90. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 91 Greeting UI The SCS Triplets and their pet Rabbit !!! Fortune ServiceSimon Hey Greeting!
  91. 91. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 92 Greeting UI The SCS Triplets and their pet Rabbit Fortune ServiceSimon Hey Greeting!
  92. 92. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 93 Greeting UI The SCS Triplets and their pet Rabbit Fortune Service Fortune DB Simon Hey Greeting!
  93. 93. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 94 Greeting UI The SCS Triplets and their pet Rabbit Fortune Service Fortune DB Simon Hey Greeting!
  94. 94. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 95 Greeting UI The SCS Triplets and their pet Rabbit Fortune Service Fortune DB Simon Hey Greeting!
  95. 95. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 96 Greeting UI “Do What Works” Simon
  96. 96. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 97 Greeting UI Simon Great advice! Who said it?
  97. 97. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 98 Greeting UI Fortune ServiceSimon Great advice! Who said it?
  98. 98. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 99 Greeting UI Fortune ServiceSimon Great advice! Who said it?
  99. 99. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 100 Greeting UI Fortune Service Fortune DB Simon Great advice! Who said it?
  100. 100. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 101 Greeting UI Fortune Service Fortune DB Simon Great advice! Who said it?
  101. 101. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 102 Greeting UI Fortune Service Fortune DB Simon It’s not that hard a question...
  102. 102. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 103 Greeting UI Fortune Service Fortune DB Simon It’s not that hard a question... The SCS Triplets and their pet Rabbit
  103. 103. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 104 Greeting UI Fortune Service Fortune DB Simon I just want to know... The SCS Triplets and their pet Rabbit
  104. 104. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Story Line 105 Greeting UI Fortune Service Fortune DB Simon PIPELINE ALL THE THINGS!!! The SCS Triplets and their pet Rabbit
  105. 105. Act I The Scaffold
  106. 106. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Act I: The Scaffold 107 ● Objective: ○ Automate the deployment of greeting-ui and fortune-service a.k.a. Make the pipelines run green ○ Establish environments (PCF spaces) in accordance with SCP recommendations ● Approach: ○ Minimum changes to apps (scaffolding, not code) ○ Create a pipeline for each ○ Create PCF spaces ○ Run the pipelines
  107. 107. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Greeting and Fortune ● Spring Boot apps ● Source code on GitHub ● Maven ● No tests ● Use 5 services ○ MySQL database (for Fortune) ○ Spring Cloud Services: Eureka, Hystrix/Turbine, Config Server ○ Rabbit - for Config Server refresh trigger propagation
  108. 108. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ ● Add 2 new branches to the app repos App Scaffold 109
  109. 109. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ ● Add 2 new branches to the app repos ● Add a Maven wrapper ○ mvn -N io.takari:maven:wrapper App Scaffold 110
  110. 110. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ ● Add 2 new branches to the app repos ● Add a Maven wrapper ○ mvn -N io.takari:maven:wrapper ● Add artifact repo info to app pom.xml (and ~/.m2/settings.xml) ○ distributionManagement.repository.id ○ distributionManagement.repository.url App Scaffold 111
  111. 111. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ ● Add 2 new branches to the app repos ● Add a Maven wrapper ○ mvn -N io.takari:maven:wrapper ● Add artifact repo info to app pom.xml ○ distributionManagement.repository.id ○ distributionManagement.repository.url ● git push! App Scaffold 112
  112. 112. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Pipeline Configuration - Git and Mvn 113 ● Create a credentials file for each app (copy sample in SCP) ○ Git url and branch of your app ○ Git url and branch of the SCP code base (cf or v1.0.0.M8, coming soon!) ○ Your git name, email, and private key ○ Your maven repo info and credentials
  113. 113. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Build - Success! 114 ● At this point, the Build jobs should all succeed ● App builds, jar uploads to maven repo, git tagged as “dev/<version>” ● The api compatibility job runs “mvn clean verify -P apicompatibility”, but the profile does not exist yet, so it really isn’t doing much ● Takes place on Concourse worker
  114. 114. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ ● Create app manifest.yml ● Create a new manifest for SCP (e.g. sc-pipelines.yml) ○ Provide info for SCP to provision services on CF (test* & stage) ● Git push! App Scaffold 115 test: services: - name: fortune-db type: broker broker: cleardb plan: spark - name: config-server type: broker broker: p-config-server plan: standard params: git: uri: https://github.com/ciberkleid/app-config useExisting: true ...
  115. 115. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Pipeline Configuration - CF 116 ● Add general CF info to the credentials file ○ paas-type: cf ○ pipeline-descriptor (the sc-pipelines manifest) ○ paas-hostname-uuid (to avoid route collision) ● Add target & login info for Test, Stage and Prod ○ For Test, you specify a prefix. SCP appends the app name
  116. 116. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Cloud Foundry Setup: Test, Stage, Prod 117 ● Create the Test, Stage, and Prod spaces ● Manually provision services in Prod ○ And maybe Stage ○ But not Test
  117. 117. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Full Pipeline - Success! 118 ● App deployed to PCF through Prod ● Git tagged as prod/<version> ● Ability to trigger rollback through Concourse
  118. 118. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 119
  119. 119. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 120
  120. 120. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Act I: The Scaffold 121 ● Objective: ○ Automate the deployment of greeting-ui and fortune-service a.k.a. Make the pipelines run green ○ Establish environments (PCF spaces) in accordance with SCP recommendations ● Approach: ○ Minimum changes to apps (scaffolding, not code) ○ Create a pipeline for each ○ Create PCF spaces ○ Run the pipelines
  121. 121. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Is That Enough? 122
  122. 122. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Nope! 123 Same for apicompatibility and e2e
  123. 123. Act II: The Test
  124. 124. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Act II: The Test 125 ● Objective: ○ Increase effectiveness of and confidence in the pipelines ○ Standardize approach to testing ● Approach: ○ Use mvn profiles to control test executions ○ Add/organize tests in agreement with SCP framework ○ Enable DB backward compatibility testing
  125. 125. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Test Control - Maven Profiles 126 ● Add profiles to the app pom.xml ○ default ○ apicompatibility ○ smoke ○ e2e <profile> <id>default</id> <activation> <activeByDefault>true</activeByDefault> </activation> <build> <plugins> <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <configuration> <includes> <include>**/*Tests.java</include> <include>**/*Test.java</include> </includes> <excludes> <exclude>**/smoke/**</exclude> <exclude>**/e2e/**</exclude> </excludes> </configuration> </plugin> <plugin> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-maven- plugin</artifactId> </plugin> </plugins> </build> </profile>
  126. 126. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Test Control - Organize Tests 127 ● Organize tests into packages matching the profiles ● Add tests for each profile
  127. 127. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Test Coverage - Fortune Smoke 128 fortune-service fortune-db
  128. 128. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Test Coverage - Fortune E2E 129 greeting-ui fortune-db fortune-service Assuming same domain...
  129. 129. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Test Coverage - Greeting Smoke 130 greeting-ui circuit breaker fallback
  130. 130. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Test Coverage - Greeting E2E 131 greeting-ui fortune-service
  131. 131. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Enable Database Rollback Testing 132 ● Flyway ○ OSS DB migration tool ○ For relational databases ○ Convention over configuration ○ Database versioning ○ Migration on app startup (good for CD!)
  132. 132. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ DB Schema Creation With Flyway 133 ● Disable JPA auto-generated schemas; let flyway do it ● Follow flyway conventions for SQL file name ● Tada! Schema is now versioned!
  133. 133. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ DB Schema Population With Flyway 134 ● Add INSERT statements to V1__init.sql ● Data is populated upon app startup, but only once per DB version
  134. 134. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Act II: The Test 135 ● Objective: ○ Increase effectiveness of and confidence in the pipelines ○ Standardize approach to testing ● Approach: ○ Use mvn profiles to control test executions ○ Add/organize tests in agreement with SCP framework ○ Enable DB backward compatibility testing
  135. 135. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Is THAT Enough? 136 ● Good: ○ You are in a great position to include well-organized, robust testing and derive a high level of confidence from your pipelines ○ You can ensure safe rollbacks in case of database schema changes ● Better: ○ Implementing a contract-driven approach will have additional benefits to your general development practice ○ Inter-team communication will be simpler ○ Contract tests will catch breaking API changes in build rather than stage ○ Troubleshooting will be easier ○ Failure and feedback will be faster
  136. 136. Act III: The Contract
  137. 137. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Act III: The Contract 138 ● Objective: ○ Increase reliability - strive to maximize pipeline benefits ○ Encourage contract-based programming practices ● Approach: ○ Add contracts, stubs, and a stubrunner ○ Enable API backward compatibility testing
  138. 138. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ The Greeting/Fortune Contract 139 import org.springframework.cloud.contract.spec.Contract Contract.make { description(""" should return a fortune string """) request { method GET() url "/" } response { status 200 body "foo fortune" } }
  139. 139. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Contract Basics 140 ● Add the Spring Cloud Contract Maven Plugin to your pom.xml (Fortune) ○ Plugin requires a base class to set up the context and stub out the service that satisfies the API call (e.g. the DB) public class BaseClass { @Before public void setup() { FortuneService service = BDDMockito.mock(FortuneService.class); BDDMockito.given(service.getFortune()).willReturn("foo fortune"); RestAssuredMockMvc.standaloneSetup(new FortuneController(service)); } }
  140. 140. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Spring Cloud Contract Basics 141 ● Add the Spring Cloud Contract Maven Plugin to your pom.xml (Fortune) ○ Plugin will auto-generate tests and stubs (and a stub.jar) ○ Stub jar is uploaded to the maven repo, along with the app jar
  141. 141. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ For Back-Compatibility, Use Prod Contracts 142 <profile> <id>apicompatibility</id> <plugin> <configuration> <contractsRepositoryUrl>${repo.with.binaries}</contractsRepositoryUrl> <contractDependency> <groupId>${project.groupId}</groupId> <artifactId>${project.artifactId}</artifactId> <classifier>stubs</classifier> <version>${latest.production.version}</version> </contractDependency> ... In this case, we want the contracts plugin to generate tests based on contracts outside of the project (the ones from the latest prod jar on our maven repo). Pipeline injects values in red dynamically
  142. 142. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Greeting: Build with Stubrunner greeting-ui circuit breaker fallback or manually configured WireMock greeting-ui stubs without stubrunner with stubrunner ● Greeting starts an in-process stubrunner that automatically configures WireMock
  143. 143. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Greeting: Integration Tests Aligned with Stubs 144 For in-process stubrunner, the server-side Wiremock port For in-process stubrunner, the client-side endpoint config
  144. 144. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Greeting: Test with Stubrunner (standalone) greeting-ui circuit breaker fallback greeting-ui stubrunner without stubrunner with stubrunner Note: Update the greeting smoke tests accordingly
  145. 145. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Home Stretch: Standalone Stubrunner Config ● You need a stubrunner app! ○ Clone spring-cloud-samples/cloudfoundry-stub-runner-boot ○ Build it, and upload the jar to your maven repo ● Your stubrunner app needs a manifest! ○ Put it in your app repo (e.g. greeting-ui repo) ● You need to tell SCP to deploy the stubrunner app! ○ Do it through the pipeline descriptor (e.g. sc-pipelines.yml)
  146. 146. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Home Stretch: Standalone Stubrunner Config ● Specify the server side (stubrunner) ports <properties> <stubrunner.ids>io.pivotal:fortune-service:_latest:stubs:10000</stubrunner.ids> </properties> ● Disable Eureka lookup for smoke tests ● Provide URLs to non-8080 stubrunner ports explicitly
  147. 147. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Act III: The Contract 148 ● Objective: ○ Increase reliability - strive to maximize pipeline benefits ○ Encourage contract-based programming practices ● Approach: ○ Add contracts, stubs, and a stubrunner ○ Enable API back compatibility testing
  148. 148. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ So what have we achieved 149 ● The Scaffold: ○ Automate the deployment of greeting-ui and fortune-service a.k.a. Make the pipelines run green ○ Establish environments (PCF spaces) in accordance with SCP recommendations ● The Test: ○ Increase effectiveness of and confidence in the pipelines ○ Standardize approach to testing ● The Contract ○ Increase reliability - strive to maximize pipeline benefits ○ Encourage contract-based programming practices
  149. 149. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Finally...
  150. 150. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ One More Thing...
  151. 151. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Just Kidding :-)
  152. 152. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ You’re Done!
  153. 153. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Though Really, You’re Just Beginning... 154 Because now… ● You can evolve your app faster ● With (hopefully) better code ● Consistent testing practices ● Fast failure and feedback ● Contract-based API definitions ● API and DB rollback assurance ● And…. instant pipelines from source to production!
  154. 154. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ What About Poor Simon? 155 Greeting UI Fortune Service Fortune DB Simon WHO SAID Do What Works!?
  155. 155. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ What About Poor Simon? 156 Greeting UI Fortune Service Fortune DB Simon WHO SAID Do What Works!? Contract.make{“”” should return author name “””) ...
  156. 156. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ What About Poor Simon? 157 Greeting UI Fortune Service Fortune DB Simon WHO SAID Do What Works!? Fortune Service Stub
  157. 157. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ What About Poor Simon? 158 Greeting UI Fortune Service Fortune DB Simon WHO SAID Do What Works!? V2: ALTER TABLE fortune CHANGE COLUMN text TO fortune, ADD author CHAR (50);
  158. 158. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ What About Poor Simon? 159 Greeting UI Fortune Service Fortune DB Simon WHO SAID Do What Works!?
  159. 159. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ What About Poor Simon? 160 Greeting UI Fortune Service Fortune DB Simon Pivotal says it all the time!
  160. 160. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Simon Spring Cloud Pipelines 161 Greeting UI Fortune Service Fortune DB Oh, right! I knew that.
  161. 161. Unless otherwise indicated, these slides are © 2013-2017 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Sample Code... 162 ● Github repos showing the end-state of each “act” ● Step-by-step instructions will be published soon and made accessible through the repo Readmes
  162. 162. Back to Marcin
  163. 163. Spring Cloud Pipelines
  164. 164. Customizing Spring Cloud Pipelines
  165. 165. Customizing Spring Cloud Pipelines $ curl -LOk https://github.com/spring-cloud/spring-cloud- pipelines/archive/v1.0.0.M8.zip $ unzip v1.0.0.M8.zip $ cd spring-cloud-pipelines-v1.0.0.M8 $ git init $ # modify the pipelines to suit your needs $ ./gradlew customize $ … $ git add . $ git commit -a -m “Initial commit” $ git remote add origin ${YOUR_REPOSITORY_URL} $ git push origin master
  166. 166. Customizing Spring Cloud Pipelines
  167. 167. Summary • Continuous Deployment allows you to continuously deliver business value • Spring Cloud Pipelines gives you OOB tooling to test your software via • unit and integration testing • contract testing • rollback testing • You can gradually migrate your applications to start using SC Pipelines • SC Pipelines allows you to easily adjust the deployment pipeline to suit your company’s needs • Thanks to PaaS you can easily do A/B & zero downtime deployment
  168. 168. ● Github Analytics: https://github.com/spring-cloud-samples/github-analytics ● Github Webhook: https://github.com/spring-cloud-samples/github-webhook ● Fortune Service: https://github.com/ciberkleid/fortune-service ● Greeting UI: https://github.com/ciberkleid/greeting-ui/ ● SC-Pipelines documentation: https://cloud.spring.io/spring-cloud-pipelines/ ● Pivotal Web Services trial : https://run.pivotal.io/ ● PCF Dev (CF on your laptop) : https://docs.pivotal.io/pcf-dev/ Links
  169. 169. Learn More. Stay Connected. ▪ Read the docs http://cloud.spring.io/spring-cloud-pipelines/ ▪ Talk to us on Gitter https://gitter.im/spring-cloud/spring-cloud-pipelines Twitter: twitter.com/springcentral YouTube: spring.io/video LinkedIn: spring.io/linkedin Google Plus: spring.io/gplus
  170. 170. ciberkleid mgrzejszczak

×