Continuous Delivery Patterns for Contemporary Architectures

Continuous Delivery Patterns for
Contemporary Architectures
Daniel Bryant
@danielbryantuk

Architecture: Expectations versus reality
28/02/2018 @danielbryantuk
“DevOps”

tl;dr
• We are moving from complicated systems to complex systems
• Architecture is becoming more about technical leadership
• We must encode all requirements into a continuous delivery pipeline

@danielbryantuk
• Independent Technical Consultant
• Architecture, DevOps, Java, microservices, cloud, containers
• Continuous Delivery (CI/CD) advocate
• Leading change through technology and teams
bit.ly/2jWDSF7

Setting the Scene

https://medium.com/making-meetup/em-el-pm-venn-diagram-764e79b42baf

Velocity (with stability) is key to business success
Adrian Cockcroft (2015) www.slideshare.net/Indicee/cloud-trends-2015-pdf

Continuous Delivery
• Produce valuable and robust software in short cycles
• Optimising for feedback and learning
• Not (necessarily) Continuous Deployment

Velocity (with stability) is key to business success
“Continuous delivery is achieved when stability and
speed can satisfy business demand.
Discontinuous delivery occurs when stability and speed
are insufficient.”
- Steve Smith (@SteveSmithCD)

Feedback:
- Was our hypothesis proven?
- How can we improve biz,
architecture and ops?

Let’s bring in some containers (or additional packaging)

What are our core challenges with modern application architectures?

Multiple services/pipelines

Gated application deployment
1.
2.
3.

Independent service deployment

Architecture

Simple
(Sense, Categorise, Respond)
Complicated
(Sense, Analyse, Respond)
Complex
(Probe, Sense, Respond)
1990s
Monoliths
Single language
In-house hardware (servers, SAN, networks)
Manual config and scripting
Optimise for Stability (MTBF)
Specialist staff/departments
2010s
Microservices, functions, SaaS-all-the-things
Polyglot languages
Cloud and containers (Datacenter as a Computer)
Software-Defined Everything
Optimise for innovation (and MTTR)
Business teams (“FinDev”, SRE and Platform Team)
2000s
Monoliths, Coarse-grained SOA, SaaS
Frontend/backend language
“Co-lo” or private datacenters
Configuration management
Optimise for Recovery (MTTR)
Generalist teams (Full Stack and “DevOps”)
Chaotic
(Act, Sense, Respond)

Architecture fundamentals
• Coupling
• ”Degree to which components have knowledge of other components”
• Interfaces
• Schema
• Cohesion
• “Degree to which the elements within a component belong together”
• Single reason to change
• Separation of concerns

Coupling, Cohesion and Continuous Delivery
• Design
• Cohesion makes reasoning easy
• Loose coupling reduces impact
• Build, unit and integration
• Cohesion limits dependencies
• Loose coupling allows simulation
• End-to-end tests
• Cohesion/coupling orchestration
• Deployment
• Cohesion minimises number of
components in play
• Coupling leads to “monoliths”
• Observability
• Cohesive is easier to understand
• High coupling typically leads to
large blast radius and “murder
mystery” style debugging

From monolith to…
UI / Biz / Repo
Monolith
Domains
Modules, components,
frameworks, libraries
http://scs-architecture.org/

Self Contained Systems (SCS)
Microservices
Function-as-a-Service
“Serverless”

Monoliths SOA Microservices / SCS FaaS / Serverless
Scope Project Enterprise Product Feature (or glue?)
Focus Swiss Army Knife Reuse, governance,
control
Domain modelling,
SRP, evolution
Function (in/out),
rapid evolution
Organisation Implemented and
maintained (typically) by
single team
Implemented by
different org units.
Maintenance done
elsewhere
Services implemented
and owned by product
teams
Implemented by
pioneers (hipsters?)
Deployment Monolithic deployment Monolithic
orchestration of
multiple services
Services deployed
individually
Functions deployed
individually
Management None Centralised Distributed Chaotic / Orchestrated
Inter-process
communication
None RPC or messaging,
typically via
middleware (ESB/MQ)
RPC via dumb pipes
and smart endpoints,
messaging/events
Events
Pioneers /
stewards
Organisations,
community or individuals
Enterprises and
Vendors
Community and high
perf organisations
Vendors/community
Core Architectural
Constraints
Language and framework Canonical domain
model, standards
Interoperability Cost (Gbs/ms)

Testing 101

Testing: Core concepts
/lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/martinfowler.com/bliki/TestPyramid.html

Testing: Core (microservice) concepts
martinfowler.com/articles/microservice-testing/#testing-progress-3

Testing: Core (microservice) concepts
https://medium.com/@copyconstruct/testing-microservices-the-sane-way-9bb31d158c16

Testing microservice integration

Functional testing: End-to-end

Functional Testing: Outside-in
https://specto.io/blog/2016/8/16/recipe-for-designing-building-testing-microservices/
http://www.thucydides.info/docs/serenity/

Functional

Talking of contracts…

Talking of (service) contracts…
• APIs are service contracts
• Consumer-driven Contracts
• martinfowler.com/articles/consumerDrivenContracts.html

Talking of (service) contracts…
docs.pact.io
cloud.spring.io/spring-cloud-contract
github.com/spring-cloud-samples/spring-cloud-contract-samples

Talking of (messaging) contracts…
• What about messaging?
• Message schema are an API
• www.infoq.com/presentations/contracts-streaming-microservices

Talking of (messaging) contracts…
www.infoq.com/presentations/contracts-streaming-microservices
docs.confluent.io/current/schema-registry/docs/maven-plugin.html

Contract verification
• Is it worth the cost?
• Tradeoff: Trust/comms vs time
• Startups / SMEs
• Enterprises…
• My (anecdotal) experience
• Choreography vs orchestration

https://netflix.github.io/conductor/
http://rockscript.io/
https://nifi.apache.org/docs.html
https://aws.amazon.com/documentation/step-functions/
http://camel.apache.org/

Contract verification
• Is it worth the cost?
• Tradeoff: Trust and comms
• Startups / SMEs
• Enterprises…
• My (anecdotal) experience
• Choreography vs orchestration
• Choreography
• Asserting behaviour (interactions)
• Contracts are part of this!
• Orchestration:
• Verify orchestration DSL
• Asserting state (output)

Measure what matters

Visibility for the business

Testing NFRs in the build pipeline
• Architecture quality
• SonarQube / Code Climate
• Performance and Load testing
• Gatling / Locust / Bees with m’guns
• Security testing
• OWASP Dependency check / Bdd-security
• Docker Bench for Security / CoreOS Clair

Architectural Visibility

Security Visibility: Static Package Scanning
https://github.com/arminc/clair-scanner

Serverless security
“Since the OS is unreachable,
attackers will shift their attention
to the areas that remain exposed
– and first amongst those would
be the application itself.”
• Responsibility for addressing
vulnerable app libraries falls to
you – the function developer.
https://www.infoq.com/articles/serverless-security

Security Visibility: Dependencies
www.owasp.org/index.php/OWASP_Dependency_Check

Delaying NFRs to the ‘Last Responsible Moment’
Newsflash!
Sometimes the
last responsible moment
is up-front
Modern architectures don’t
necessarily make this easier

Wrapping up

Scope Project Enterprise Product Feature (or glue?)
Focus Swiss Army Knife Reuse, governance,
control
Domain modelling,
SRP, evolution
Function (in/out),
rapid evolution
Organisation Implemented and
maintained (typically) by
single team
Implemented by
different org units.
Maintenance done
elsewhere
Services implemented
and owned by product
teams
Implemented by
pioneers (hipsters?)
Deployment Monolithic deployment Monolithic
orchestration of
multiple services
Services deployed
individually
Functions deployed
individually
Management None Centralised Distributed Chaotic / Orchestrated
Inter-process
communication
None RPC or messaging,
typically via
middleware (ESB/MQ)
RPC via dumb
pipes/smart
endpoints,
messaging/events
Events
Pioneers /
stewards
Organisations,
community or individuals
Enterprises and
Vendors
Community and high
perf organisations
Vendors/community
Core Architectural
Constraints
Language and framework Canonical domain
model, standards
Interoperability Cost

Cohesion and coupling
enforced at modules
CD focuses on unit and
end-to-end
functionality
Provide goals and “best
practice” examples
Cohesion and
coupling enforced at
service level
CD focuses on
service integrity
Provide objectives
and standards
Cohesion and
service API level
CD focuses on
service interaction
Provide principles,
guidelines and
“good practice”
Cohesion and
function API level
CD focuses on service
orchestration+output
Provide principles,
guidelines and “good
practice”

In conclusion…
• We are moving from complicated systems to complex systems
• Design and test with coupling and cohesion in mind
• Architecture is becoming more about technical leadership
• Recognise your situation and influence accordingly
• We must encode all requirements into a continuous delivery pipeline
• Both functional and nonfunctional requirements

Thanks for listening…
Twitter: @danielbryantuk
Email: daniel.bryant@tai-dev.co.uk
Writing: https://www.infoq.com/profile/Daniel-Bryant
Talks: https://www.youtube.com/playlist?list=PLoVYf_0qOYNeBmrpjuBOOAqJnQb3QAEtM
Early Access edition
book signing at the
O’Reilly booth now!

Bedtime reading…
blog.christianposta.com/microservices/low-risk-monolith-to-microservice-evolution-part-iii/
builttoadapt.io/whats-your-decomposition-strategy-e19b8e72ac8f

Continuous Delivery Patterns for Contemporary Architectures

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Continuous Delivery Patterns for Contemporary Architectures

Similar to Continuous Delivery Patterns for Contemporary Architectures (20)

More from Daniel Bryant

More from Daniel Bryant (20)

Recently uploaded

Recently uploaded (20)

Continuous Delivery Patterns for Contemporary Architectures