JAX DevOps 2018 "Continuous Delivery Patterns for Modern Architectures"

Continuous Delivery Patterns for
Modern Java Architectures
Daniel Bryant
@danielbryantuk

Architecture: Expectations versus reality
11/04/2018 @danielbryantuk
“DevOps”

tl;dr
• We are moving from complicated systems to complex systems
• Architecture is becoming more about technical leadership
• We must encode all requirements into a continuous delivery pipeline

@danielbryantuk
• Independent Technical Consultant, Product Architect at Datawire
• Architecture, DevOps, Java, microservices, cloud, containers
• Continuous Delivery (CI/CD) advocate
• Leading change through technology and teams
bit.ly/2jWDSF7

Setting the Scene

Continuous Delivery
• Produce valuable and robust software in short cycles
• Optimising for feedback and learning
• Not (necessarily) Continuous Deployment

Velocity (with stability) is key to business success
“Continuous delivery is achieved when stability and
speed can satisfy business demand.
Discontinuous delivery occurs when stability and speed
are insufficient.”
- Steve Smith (@SteveSmithCD)

Feedback:
- Was our hypothesis proven?
- How can we improve biz,
architecture and ops?

Let’s bring in some containers (or additional packaging)

What are our core challenges with modern Java app architectures?

Multiple services/pipelines

Gated application deployment
1.
2.
3.

Independent service deployment

Architecture 101

From monolith to…
UI / Biz / Repo
Monolith
Domains
Modules, components,
frameworks, libraries
http://scs-architecture.org/

Self Contained Systems (SCS)
Microservices
Function-as-a-Service
“Serverless”

Monoliths SOA Microservices / SCS FaaS / Serverless
Scope Project Enterprise Product Feature (or glue?)
Focus Swiss Army Knife Reuse, governance,
control
Domain modelling,
SRP, evolution
Function (in/out),
rapid evolution
Organisation Implemented and
maintained (typically) by
single team
Implemented by
different org units.
Maintenance done
elsewhere
Services implemented
and owned by product
teams
Implemented by
pioneers (hipsters?)
Deployment Monolithic deployment Monolithic
orchestration of
multiple services
Services deployed
individually
Functions deployed
individually
Management None Centralised Distributed Chaotic / Orchestrated
Inter-process
communication
None RPC or messaging,
typically via
middleware (ESB/MQ)
RPC via dumb pipes
and smart endpoints,
messaging/events
Events
Pioneers /
stewards
Organisations,
community or individuals
Enterprises and
Vendors
Community and high
perf organisations
Vendors/community
Core Architectural
Constraints
Language and framework Canonical domain
model, standards
Interoperability Cost (Gbs/ms)

Architecture fundamentals
• Coupling
• ”Components have little or no knowledge of other components”
• Interfaces
• Schema
• Cohesion
• “Degree to which the elements within a component belong together”
• Single reason to change
• Separation of concerns

Coupling, Cohesion and Continuous Delivery
• Design
• Cohesion makes reasoning easy
• Loose coupling reduces impact
• Build, unit and integration
• Cohesion limits dependencies
• Loose coupling allows simulation
• End-to-end tests
• Cohesion/coupling orchestration
• Deployment
• Cohesion minimises number of
components in play
• Coupling leads to “monoliths”
• Observability
• Cohesive is easier to understand
• High coupling typically leads to
large blast radius and “murder
mystery” style debugging

Testing 101

Testing: Core concepts
/lisacrispin.com/2011/11/08/using-the-agile-testing-quadrants/martinfowler.com/bliki/TestPyramid.html

Testing: Core concepts
martinfowler.com/articles/microservice-testing/#testing-progress-3 martinfowler.com/articles/practical-test-pyramid.html

Testing: Core (microservice) concepts
https://medium.com/@copyconstruct/testing-microservices-the-sane-way-9bb31d158c16

Testing microservice integration

Functional testing: End-to-end

Functional Testing: Outside-in
https://specto.io/blog/2016/8/16/recipe-for-designing-building-testing-microservices/
http://www.thucydides.info/docs/serenity/

Functional

Talking of contracts…

Talking of (service) contracts…
• APIs are service contracts
• Consumer-driven Contracts
• martinfowler.com/articles/consumerDrivenContracts.html

Talking of (service) contracts…
docs.pact.io
cloud.spring.io/spring-cloud-contract
github.com/spring-cloud-samples/spring-cloud-contract-samples

Talking of (messaging) contracts…
• What about messaging?
• Message schema are an API
• www.infoq.com/presentations/contracts-streaming-microservices

Talking of (messaging) contracts…
www.infoq.com/presentations/contracts-streaming-microservices
docs.confluent.io/current/schema-registry/docs/maven-plugin.html

Contract verification
• Is it worth the cost?
• Tradeoff: Trust/comms vs time
• Startups / SMEs
• Enterprises…
• My (anecdotal) experience
• Choreography vs orchestration

https://netflix.github.io/conductor/
http://rockscript.io/
https://nifi.apache.org/docs.html
https://aws.amazon.com/documentation/step-functions/
http://camel.apache.org/
https://blog.bernd-ruecker.com/orchestrating-azure-functions-using-bpmn-and-camunda-a-case-study-ff71264cfad6

Contract verification
• Is it worth the cost?
• Tradeoff: Trust and comms
• Startups / SMEs
• Enterprises…
• My (anecdotal) experience
• Choreography vs orchestration
• Choreography
• Verifying behaviour (interactions)
• Contracts are part of this!
• London school TDD
• Orchestration
• Verify state (output)
• Lint/validate orchestration DSL
• Chicago school TDD

Measure what matters

Visibility for the business

Testing NFRs in the build pipeline
• Architecture quality
• SonarQube / Code Climate
• Performance and Load testing
• Gatling / Locust / Bees with m’guns
• Security testing
• OWASP Dependency check / bdd-security
• Docker Bench for Security / CoreOS Clair

Architectural Visibility

Unit Testing Architecture
https://www.archunit.org/

Security Visibility: Basic Java Code Scanning

Security Visibility: Static Package Scanning
https://github.com/arminc/clair-scanner

Serverless security
“Since the OS is unreachable,
attackers will shift their attention
to the areas that remain exposed
– and first amongst those would
be the application itself.”
Responsibility for addressing
vulnerable app libraries falls to
you – the function developer.
https://www.infoq.com/articles/serverless-security

Security Visibility: Dependencies
www.owasp.org/index.php/OWASP_Dependency_Check

Delaying NFRs to the ‘Last Responsible Moment’
Newsflash!
Sometimes the
last responsible moment
is up-front
Modern architectures don’t
necessarily make this easier

Wrapping Up

Scope Project Enterprise Product Feature (or glue?)
Focus Swiss Army Knife Reuse, governance,
control
Domain modelling,
SRP, evolution
Function (in/out),
rapid evolution
Organisation Implemented and
maintained (typically) by
single team
Implemented by
different org units.
Maintenance done
elsewhere
Services implemented
and owned by product
teams
Implemented by
pioneers (hipsters?)
Deployment Monolithic deployment Monolithic
orchestration of
multiple services
Services deployed
individually
Functions deployed
individually
Management None Centralised Distributed Chaotic / Orchestrated
Inter-process
communication
None RPC or messaging,
typically via
middleware (ESB/MQ)
RPC via dumb
pipes/smart
endpoints,
messaging/events
Events
Pioneers /
stewards
Organisations,
community or individuals
Enterprises and
Vendors
Community and high
perf organisations
Vendors/community
Core Architectural
Constraints
Language and framework Canonical domain
model, standards
Interoperability Cost

Cohesion and coupling
enforced at modules
CD focuses on end-to-
end functionality
Provide goals and “best
practice” examples
Cohesion and
coupling enforced at
service level
CD focuses on
service integrity
Provide objectives
and standards
Cohesion and
service API level
CD focuses on
service interaction
Provide principles
and guidelines
Cohesion and
function API level
CD focuses on
service output/state
Provide principles
and guidelines

In conclusion…
• We are moving from complicated systems to complex systems
• Design and test with coupling and cohesion in mind
• Architecture is becoming more about technical leadership
• Recognise your situation and influence accordingly
• We must encode all requirements into a continuous delivery pipeline
• Both functional and nonfunctional requirements

Bedtime reading…
blog.christianposta.com/microservices/low-risk-monolith-to-microservice-evolution-part-iii/

Thanks for listening…
Twitter: @danielbryantuk
Email: daniel.bryant@tai-dev.co.uk
Writing: https://www.infoq.com/profile/Daniel-Bryant
Talks: https://www.youtube.com/playlist?list=PLoVYf_0qOYNeBmrpjuBOOAqJnQb3QAEtM
bit.ly/2jWDSF7
Coming soon!

JAX DevOps 2018 "Continuous Delivery Patterns for Modern Architectures"

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to JAX DevOps 2018 "Continuous Delivery Patterns for Modern Architectures"

Similar to JAX DevOps 2018 "Continuous Delivery Patterns for Modern Architectures" (20)

More from Daniel Bryant

More from Daniel Bryant (20)

Recently uploaded

Recently uploaded (20)

JAX DevOps 2018 "Continuous Delivery Patterns for Modern Architectures"