SpringOne Tour Denver - Continuous Delivery of your Application

Supportive Cast
Marcin Grzejszczak
Spring Cloud developer at Pivotal
Twitter: @mgrzejszczak
Blog: http://toomuchcoding.com
Cora Iberkleid
Senior Platform Architect at Pivotal
Twitter: @ciberkleid

Continuously Delivering Business Value
https://www.youtube.com/watch?v=7APZD0me1nU by Onsi Fakhouri

The Circle of Code...
•  Depicts an ideal, iterative manner of delivering
business value to production
•  Enables the business and developers to test ideas
against reality quickly
•  Aims to improve software quality and
responsiveness to changing customer requirements

Spring Cloud Pipelines is the answer :-)
But what is the question?

How can we avoid creating a deployment
pipeline from scratch for each new project?

WHOSE DEPLOYMENT PIPELINE
LOOKS LIKE THIS?

15 HOURS OF E2E TESTS PLACE
HERE

What is Spring Cloud Pipelines?
•  opinionated template of a deployment pipeline
•  based on good practices from different projects
•  uses the “Infrastructure as Code” approach
•  if automation server goes down you can recover everything quickly
•  the automation server setup should be automated too!
•  you can even write tests for your pipelines

Spring Cloud Pipelines
•  Supports the following automation servers out of the box:
•  Concourse
•  Jenkins using Jenkins Job DSL plugin
•  Jenkins using Jenkinsfile with Blue Ocean
•  Logic of all steps done via Bash scripts
•  you can convert the internals to suit your needs
•  you can use whatever automation server you want
•  supports Maven & Gradle
•  Includes 150 Bash tests to validate your customizations of SCP
•  using https://github.com/sstephenson/bats

Spring Cloud Pipelines – some opinionated defaults
•  Four environments (Build, Test, Stage, Prod)
•  Types of tests
•  e.g. Unit, API compatibility, Smoke, Integration, Rollback, End to end
•  Use cloud-native apps
•  Use contracts for integration testing
•  PaaS deployment platform

Spring Cloud Pipelines - Concourse

Spring Cloud Pipelines – The big picture on Concourse

Spring Cloud Pipelines - Jenkins Job DSL

Spring Cloud Pipelines - Jenkinsfile & Blue Ocean

Why do we deploy software?
•  We’re paid for delivering business value
•  Features are done when they are deployed to production
•  It’s better to deploy frequently for faster feedback
•  It’s better to fail the deployment pipeline as fast as possible
•  Deployments should take place in a standardized, automated fashion
•  Your deployment pipeline should test rollback
•  That way you can perform A/B or zero downtime deployment to production

Problem - slow feedback
•  Nobody wants to wait until the end of the pipeline to see that
something is not working fine
•  We want to fail fast - even during build time
•  If integration is faulty
•  If our API is backward incompatible
•  There should be no need to wait until end to end tests are executed

Solution - fail fast
•  We’re running unit and integration tests during build time
•  To test faulty integration we use Spring Cloud Contract for HTTP / messaging
•  Producer defines a contract
•  From the contract
o  tests are generated to see if the producer is not lying
o  stubs are generated for consumers to use
•  Consumer can reuse it without running the producer
•  Fail at build time if integration fails (fail fast!)
•  All stubs reside in Bintray / Nexus / Artifactory (thus can be reused)

Broken build Current implementation (V2)
does not support old contracts
(V1)

STANDARDIZATION GIVES LOWER SUPPORT
COSTS AND MORE CHANCES OF PEOPLE
ROTATION

Solution - PaaS & tooling
•  Use a PaaS to standardize the way you deploy and monitor your software
•  Spring Cloud Pipelines uses Cloud Foundry [1] or Kubernetes [2] as a PaaS
implementation
•  You can also use PCF Dev [3] or Minikube [4]
•  PaaS abstracts the application governance from underlying infrastructure
•  you can deploy, scale, manage applications in the same way if PaaS was running on your
laptop, Amazon, Google, Azure etc.
•  Database schema upgrade is done via tools like Flyway [5] or Liquibase [6]
[1] https://www.cloudfoundry.org, [2] https://kubernetes.io/
[3] https://pivotal.io/pcf-dev, [4] https://github.com/kubernetes/minikube
[5] https://flywaydb.org/, [6] http://www.liquibase.org/

After the application got deployed to test environment
•  The database schema gets updated upon application startup
•  We run a handful of smoke tests to see if crucial parts of our application are
working fine
•  We want to test if the app is properly packaged
•  The application is surrounded by stubs - no real integrations take place
•  Spring Cloud Contract Stub Runner Boot app is responsible for serving stubs

Origin of term “Smoke Test”?

Problem - rollback DB
•  Deployment pipelines should test whether the application can be rolled back
•  Rolling back database changes is extremely difficult
•  Especially if you deploy once every 6 months (the number of changes is gigantic)
•  How can you roll back a deletion of a column?

Solution - application rollback
•  The easiest solution is… NOT TO DB ROLLBACK
•  Perform only backward compatible changes (always add data)
•  Or perform backward incompatible changes in a backward compatible way [1]
•  Roll back the application only (the JAR)
•  The application and DB changes need to be backward compatible
•  That way you can ensure that two applications (old / new versions) can run
at the same time

[1] https://spring.io/blog/2016/05/31/zero-downtime-deployment-with-a-database

Backward incompatible DB change
Old application (V1) can’t work
with new database (V2)
Deploy (V1) and connect to the
new database (V2)

Problem - end to end tests
•  End to end tests are slow and brittle
•  QA department writes an E2E for every feature we have
•  E2E environment setup
•  one environment shared between all applications?
•  one environment per application?
•  Surrounding apps should be deployed in
•  production versions?
•  development versions?

Solution - don’t do E2E?
•  Regardless of the time spent on QA / UAT you can still have bugs
on production
•  Assuming that you ...
•  embrace failure
•  introduce monitoring of business key performance indicators (KPIs)
•  introduce alerting over the metrics
•  ensure that you can rollback on production
•  … you could stop doing any end to end tests

•  Deploy to stage and running e2e tests are manual steps
•  you have to wait for your turn for the env
•  some manual work has to be done to purge stale data etc.

Problem - deployment to production
•  We don’t want to deploy the application to production at night
•  We want to treat a production deployment like any other deployment
•  We’d like to be able to perform A/B testing and zero downtime deployment
•  We’d like to easily rollback when something goes wrong

Solution - PaaS + SC Pipelines
•  Our application has KPI monitoring in place
•  Alerting are set for KPIs
•  It has been tested that the application can be easily rolled back
•  PaaS (CF or K8S) can take care of zero downtime deployment

•  prod-deploy deploys the pipeline version of the app to production
next to the current production version
•  Once deployed we tag the repo with prod/VERSION_NUMBER
•  prod-complete stops the old instance, leaving only the new one
running
•  prod-rollback deletes the new instance, removes the prod/
VERSION_NUMBER tag and leaves only the old one running

Two versions running at the same time
Two versions registered
under same hostname
Old instance
New instance

MIGRATING TO SPRING CLOUD PIPELINES DEMO

based upon the
Step-by-step Cloud Foundry migration
tutorial by @ciberkleid

Application Refactoring for
A Story in Three Acts

Main Characters
52
Greeting UI Fortune Service Fortune DB
Simon

Supporting Characters
53
The SCS Triplets
and their pet Rabbit
Simon

Story Line
55
Greeting UI
Simon

Story Line
56
Greeting UI
Simon
Hey Greeting!

Story Line
57
Greeting UI
The SCS Triplets
Simon
Hey Greeting!

Story Line
58
Greeting UI
The SCS Triplets
???
Simon
Hey Greeting!

Story Line
59
Greeting UI
The SCS Triplets
!!!
Fortune Service
Simon
Hey Greeting!

Story Line
60
Greeting UI
The SCS Triplets
Fortune Service
Simon
Hey Greeting!

Story Line
61
Greeting UI
The SCS Triplets
Fortune Service Fortune DB
Simon
Hey Greeting!

Story Line
62
Greeting UI
The SCS Triplets
Simon
Hey Greeting!

Story Line
63
Greeting UI
The SCS Triplets
Simon
Hey Greeting!

Story Line
64
Greeting UI
“Do What
Works”
Simon

Story Line
65
Greeting UI
Simon
Great advice!
Who said it?

Story Line
66
Greeting UI Fortune Service
Simon
Great advice!
Who said it?

Story Line
67
Greeting UI Fortune Service
Simon
Great advice!
Who said it?

Story Line
68
Simon
Great advice!
Who said it?

Story Line
69
Simon
Great advice!
Who said it?

Story Line
70
Simon
It’s not that
hard a
question...

Story Line
71
Simon
It’s not that
hard a
question...
The SCS Triplets

Story Line
72
Simon
I just want to
know...
The SCS Triplets

Story Line
73
Simon
PIPELINE
ALL THE
THINGS!!!
The SCS Triplets

Act I: The Scaffold
75
●  Objective:
○  Automate the deployment of greeting-ui and fortune-service
a.k.a. Make the pipelines run green
○  Establish environments (PCF spaces) in accordance with SCP
recommendations
●  Approach:
○  Minimum changes to apps (scaffolding, not code)
○  Create a pipeline for each
○  Create PCF spaces
○  Run the pipelines

Greeting and Fortune
●  Spring Boot apps
●  Source code on GitHub
●  Maven
●  No tests
●  Use 5 services
○  MySQL database (for Fortune)
○  Spring Cloud Services: Eureka, Hystrix/Turbine, Config Server
○  Rabbit - for Config Server refresh trigger propagation

●  Add 2 new branches to the app repos
App Scaffold
77

●  Add a Maven wrapper
○  mvn -N io.takari:maven:wrapper
App Scaffold
78

●  Add artifact repo info to app pom.xml (and ~/.m2/settings.xml)
○  distributionManagement.repository.id
○  distributionManagement.repository.url
App Scaffold
79

●  Add artifact repo info to app pom.xml
○  distributionManagement.repository.id
○  distributionManagement.repository.url
●  git push!
App Scaffold
80

Pipeline Configuration - Git and Mvn
81
●  Create a credentials file for each app (copy sample in SCP)
○  Git url and branch of your app
○  Git url and branch of the SCP code base use v1.0.0.M8 or later for cf)
○  Your git name, email, and private key
○  Your maven repo info and credentials

Build - Success!
82
●  At this point, the Build jobs should all succeed
●  App builds, jar uploads to maven repo, git tagged as “dev/<version>”
●  The api compatibility job runs “mvn clean verify -P apicompatibility”, but the
profile does not exist yet, so it really isn’t doing much
●  Takes place on Concourse worker

●  Create app manifest.yml
●  Create a new manifest for SCP (e.g. sc-pipelines.yml)
○  Provide info for SCP to provision services on CF (test* & stage)
●  Git push!
App Scaffold
83
test:
services:
- name: fortune-db
type: broker
broker: cleardb
plan: spark
- name: config-server
type: broker
broker: p-config-server
plan: standard
params:
git:
uri: https://github.com/ciberkleid/app-config
useExisting: true
...

Pipeline Configuration - CF
84
●  Add general CF info to the credentials file
○  paas-type: cf
○  pipeline-descriptor (the sc-pipelines manifest)
○  paas-hostname-uuid (to avoid route collision)
●  Add target & login info for Test, Stage and Prod
○  For Test, you specify a prefix. SCP appends the app name

Cloud Foundry Setup: Test, Stage, Prod
85
●  Create the Test, Stage, and Prod spaces
●  Manually provision services in Prod
○  And maybe Stage
○  But not Test

Test, Stage & Prod - Success!
86
●  App deployed to PCF through Prod
●  Git tagged as prod/<version>
●  Ability to trigger rollback through Concourse

Act I: The Scaffold - Recap
89
●  Objective:
recommendations
●  Approach:
○  Minimum changes to apps (scaffolding, not code)
○  Create a pipeline for each
○  Create PCF spaces
○  Run the pipelines

Nope!
91
Same for apicompatibility and e2e

Act II: The Test
93
●  Objective:
○  Increase effectiveness of and confidence in the pipelines
○  Standardize approach to testing
●  Approach:
○  Use mvn profiles to control test executions
○  Add/organize tests in agreement with SCP framework
○  Enable DB backward compatibility testing

Test Control - Maven Profiles
94
●  Add profiles to the app pom.xml
○  default
○  apicompatibility
○  smoke
○  e2e
<profile>
<id>default</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<configuration>
<includes>
<include>**/*Tests.java</include>
<include>**/*Test.java</include>
</includes>
<excludes>
<exclude>**/smoke/**</exclude>
<exclude>**/e2e/**</exclude>
</excludes>
</configuration>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</profile>

Test Control - Organize Tests
95
●  Organize tests into packages matching the profiles
●  Add tests for each profile

Test Coverage - Fortune Smoke
96
fortune-service
fortune-db

Test Coverage - Fortune E2E
97
greeting-ui
fortune-db
fortune-service
Assuming same domain...

Test Coverage - Greeting Smoke
98
greeting-ui
circuit breaker fallback

Test Coverage - Greeting E2E
99
greeting-ui
fortune-service

Enable Database Rollback Testing
100
●  Flyway
○  OSS DB migration tool
○  For relational databases
○  Convention over configuration
○  Database versioning
○  Migration on app startup (good for CD!)

DB Schema Creation With Flyway
101
●  Disable JPA auto-generated schemas; let flyway do it
●  Follow flyway conventions for SQL file name
●  Tada! Schema is now versioned!

DB Schema Population With Flyway
102
●  Add INSERT statements to V1__init.sql
●  Data is populated upon app startup, but only once per DB version

Act II: The Test - Recap
103
●  Objective:
●  Approach:
○  Use mvn profiles to control test executions
○  Add/organize tests in agreement with SCP framework
○  Enable DB backward compatibility testing

Is THAT Enough?
104
●  Good:
○  You are in a great position to include well-organized, robust testing and
derive a high level of confidence from your pipelines
○  You can ensure safe rollbacks in case of database schema changes
●  Better:
○  Implementing a contract-driven approach will have additional benefits to
your general development practice
○  Inter-team communication will be simpler
○  Contract tests will catch breaking API changes in build rather than stage
○  Troubleshooting will be easier
○  Failure and feedback will be faster

Act III: The Contract
106
●  Objective:
○  Increase reliability - strive to maximize pipeline benefits
○  Encourage contract-based programming practices
●  Approach:
○  Add contracts, stubs, and a stubrunner
○  Enable API backward compatibility testing

The Greeting/Fortune Contract
107
import org.springframework.cloud.contract.spec.Contract
Contract.make {
description("""
should return a fortune string
""")
request {
method GET()
url "/"
}
response {
status 200
body "foo fortune"
}
}

Spring Cloud Contract Basics
108
●  Add the Spring Cloud Contract Maven Plugin to your pom.xml (Fortune)
○  Plugin requires a base class to set up the context and stub out the
service that satisfies the API call (e.g. the DB)
public class BaseClass {
@Before
public void setup() {
FortuneService service = BDDMockito.mock(FortuneService.class);
BDDMockito.given(service.getFortune()).willReturn("foo fortune");
RestAssuredMockMvc.standaloneSetup(new FortuneController(service));
}
}

Spring Cloud Contract Basics
109
●  Add the Spring Cloud Contract Maven Plugin to your pom.xml (Fortune)
○  Plugin will auto-generate tests and stubs (and a stub.jar)
○  Stub jar is uploaded to the maven repo, along with the app jar

For Back-Compatibility, Use Prod Contracts
110
<profile>
<id>apicompatibility</id>
<plugin>
<configuration>
<contractsRepositoryUrl>${repo.with.binaries}</contractsRepositoryUrl>
<contractDependency>
<groupId>${project.groupId}</groupId>
<artifactId>${project.artifactId}</artifactId>
<classifier>stubs</classifier>
<version>${latest.production.version}</version>
</contractDependency>
...
In this case, we want the contracts plugin to generate tests based on contracts outside of the
project (the ones from the latest prod jar on our maven repo). Pipeline injects values in red
dynamically

Greeting: Build with Stubrunner
greeting-ui
or manually configured
WireMock
greeting-ui
stubs
without stubrunner with stubrunner
●  Greeting starts an in-process stubrunner that automatically configures
WireMock

Greeting: Integration Tests Aligned with Stubs
112
For in-process stubrunner, the
server-side Wiremock port
For in-process stubrunner, the
client-side endpoint config

Greeting: Test with Stubrunner (standalone)
greeting-ui
greeting-ui
stubrunner
without stubrunner with stubrunner
Note: Update the
greeting smoke tests
accordingly

Home Stretch: Standalone Stubrunner Config
●  You need a stubrunner app!
○  Clone spring-cloud-samples/cloudfoundry-stub-runner-boot
○  Build it, and upload the jar to your maven repo
●  Your stubrunner app needs a manifest!
○  Put it in your app repo (e.g. greeting-ui repo)
●  You need to tell SCP to deploy the stubrunner app!
○  Do it through the pipeline descriptor (e.g. sc-pipelines.yml)

Home Stretch: Standalone Stubrunner Config
●  Specify the server side (stubrunner) ports
<properties>
<stubrunner.ids>io.pivotal:fortune-service:_latest:stubs:10000</stubrunner.ids>
</properties>
●  Disable Eureka lookup for smoke tests
●  Provide URLs to non-8080 stubrunner ports explicitly

Act III: The Contract - Recap
116
●  Objective:
●  Approach:
○  Add contracts, stubs, and a stubrunner
○  Enable API back compatibility testing

So what have we achieved
117
●  The Scaffold:
recommendations
●  The Test:
●  The Contract

Though Really, You’re Just Beginning...
122
Because now…
●  You can evolve your app faster
●  With (hopefully) better code
●  Consistent testing practices
●  Fast failure and feedback
●  Contract-based API definitions
●  API and DB rollback assurance
●  And…. instant pipelines from source to production!

What About Poor Simon?
123
Simon
WHO SAID
Do What
Works!?

124
Simon
WHO SAID
Do What
Works!?
Contract.make{“””
should return author name
“””)
...

125
Simon
WHO SAID
Do What
Works!?
Fortune Service
Stub

126
Simon
WHO SAID
Do What
Works!?
V2:

ALTER TABLE fortune
CHANGE COLUMN text TO fortune,
ADD author CHAR (50);

127
Simon
WHO SAID
Do What
Works!?

128
Simon
Pivotal says it
all the time!

Simon Spring Cloud Pipelines
129
Oh, right!
I knew that.

Sample Code...
130
●  Github repos showing the
end-state of each “act”
●  Step-by-step instructions will
be published soon and made
accessible through the repo
Readmes

Summary
•  Continuous Deployment allows you to continuously deliver business value
•  Spring Cloud Pipelines gives you OOB tooling to test your software via
•  unit and integration testing
•  contract testing
•  rollback testing
•  You can gradually migrate your applications to start using SC Pipelines
•  SC Pipelines allows you to easily adjust the deployment pipeline to suit your
company’s needs
•  Thanks to PaaS you can easily do A/B & zero downtime deployment

●  Github Analytics: https://github.com/spring-cloud-samples/github-analytics
●  Github Webhook: https://github.com/spring-cloud-samples/github-webhook
●  Fortune Service: https://github.com/ciberkleid/fortune-service
●  Greeting UI: https://github.com/ciberkleid/greeting-ui/
●  SC-Pipelines documentation: https://cloud.spring.io/spring-cloud-pipelines/
●  Pivotal Web Services trial : https://run.pivotal.io/
●  PCF Dev (CF on your laptop) : https://docs.pivotal.io/pcf-dev/
Links

Learn More. Stay Connected.
▪  Read the docs
http://cloud.spring.io/spring-cloud-pipelines/
▪  Talk to us on Gitter
https://gitter.im/spring-cloud/spring-cloud-pipelines
Twitter: twitter.com/springcentral
YouTube: spring.io/video
LinkedIn: spring.io/linkedin
Google Plus: spring.io/gplus

ciberkleid
mgrzejszczak

JavaFXpert

SpringOne Tour Denver - Continuous Delivery of your Application

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to SpringOne Tour Denver - Continuous Delivery of your Application

Similar to SpringOne Tour Denver - Continuous Delivery of your Application (20)

More from VMware Tanzu

More from VMware Tanzu (20)

Recently uploaded

Recently uploaded (20)

SpringOne Tour Denver - Continuous Delivery of your Application