Consumer-Driven Contract Testing

Consumer-Driven Contract Testing
Implementing a scalable testing strategy for Microservices
Paulo Clavijo Esteban
@pclavijo - January 2018

About me
Principal Software Engineer at Dell-EMC
Organizer Cork Software Crafters meetup
Paulo Clavijo @pclavijo - January 2018
Cork Software CraftersPaulo Clavijo Esteban @pclavijo
paucls.wordpress.com
github.com/paucls

Context
● < 2014 - Monolithic systems
○ Unit tests, Integrations Tests and slow Webdriver E2E tests.
● 2014 - Transition to Microservices architecture and web apps
○ Services tested with Unit tests, Component tests.
○ Web apps with Unit tests and E2E tests using a stub backend.
○ System Integration tests limited to high-value User Journeys.
● 2016 - CD Pipelines
○ Teams doing Continuous Delivery. Feature Toggles.
○ Continuous Deployment only to Staging.
● 2017 - Implemented Consumer-Driven Contract Tests
○ Solved the REST APIs integration issues. Full CD to Production.

Context
A previous attempt of using Contract Tests didn’t work out
● Contract testing purpose and value was not understood properly.
● Introduced partially without bringing real value.
○ Only consumer contracts for Java services without corresponding providers test.
○ Not used on Web apps.
○ No integration with CI/CD pipeline, pull requests.
○ Overhead of writing them didn’t pay off.
● Tests scope was too broad.
● Shared client libraries. No Front-end first / CDC flow.

The Challenge
Multiple feature teams, applications, microservices. Independent
deployments, ...
… REST APIs integration issues.
How to test collaboration between system components?
● Isolated Component Tests, Test Doubles and the Test
Symmetry problem
● Integration Tests?
● System End-to-end tests?
Web-app
to Service
Service to
Service

Contract Tests
Traditionally understood as a separate set of tests that checks that all the calls against
your test doubles return the same results as a call to the remote service would.
Consumer
Remote
Service
Test Double
Unit tests Contract tests

Consumer-Driven Contract Tests
Unit tests in both sides where consumers detail clearly what are the interactions and
behaviours expected from the provider API.
Consumer Provider
Consumer
Contracts
Test Double
Verify on providerRecord consumer
expectations
Unit tests Unit tests

Consumer Driven Contracts (CDC) is a pattern that drives the development of the Provider
from its Consumer's point of view. It is TDD for services.
It is possible to make changes on a provider knowing that consumers won't be impacted!
Consumer A
Consumer B
Consumer C
Provider
Consumer-Driven
Contract
{ Id: 100,
name: “A task” }
{ Id: 100,
name: “A task”
expedite: true }
{ Id: 100,
duration: 20 }
{ Id: 100,
name: “A task”,
expedite: true,
duration: 20 }
Consumer Contract A
Consumer Contract B
Consumer Contract C

They do not test the behaviour of the service in depth but that the inputs and outputs of
service calls contain required attributes.
https://martinfowler.com/articles/microservice-testing

Consumer-Driven Contracts
Great for developing and testing intra-organisation microservices.
● You (or your team/organisation/partner organisation) control the development of both the
consumer and the provider.
● The requirements of the consumer(s) are going to be used to drive the features of the
provider.
● There is a small enough number of consumers for a given provider that the provider team can
manage an individual relationship with each consumer team.
https://docs.pact.io/documentation/what_is_pact_good_for.html

Pact
● Some of the tools that support Consumer-Driven Contract Testing are Pact and
Spring Cloud Contract.
● We chose Pact because:
○ Generation of contracts from the code
○ Provider verification (pact-jvm-provider-junit)
○ Supports multiple languages JVM, Node, JS web apps, …
○ Pact Specification
○ Pact Broker

Pact Broker
● Helps managing, versioning and distributing
contracts.
● Great tool to visualize contracts and system
relationships (dependency graph).
● Contracts as a form of “Live Documentation” for
APIs and interactions between components.

CDC on the Consumer side
Consumer
Contracts
Provider
Contract
Tests
1) Define expected interactions
2) Tests the API Client
code on the consumer
Provider
Contract
Tests
Consumer
Contract
Tests
4) Generates contracts
Consumer
API Client
Pact Mock
Service
3) Verify interactions

1) Define expected interactions
2) Tests the API Client code
3) Verify interactions

4) Pact Mock Service generates contract
files for successful tests
These contracts, in a CDC flow, can
now be shared and used to drive the
development of the provider.

Contract Testing on the Provider side
Consumer
Contracts
Provider
A
P
I
Provider
Contract
Tests
1) Pact auto-generates provider tests
3) Tests the provider
Provider
Contract
Tests
Provider
Contract
Tests
2) Set-up provider state

Auto-generated Pact provider contract tests

Pact uses the state description on the
contract to determine which set-up logic
needs to be executed before running a
generated test.

Let’s implement a new end-point ...

Tests fail when changes in Provider impacts a Consumer

Flexible Matching
Pact provides flexible matching by regular
expressions or type.

Defining Interactions
When we define the expected interactions between a consumer and a provider, the ‘state’ and
‘uponReceiving’ descriptions used should be written in a consistent style.
Later on, on the provider side, Pact uses the uponReceiving description to name the auto-generated
test, and the state description is used to find a corresponding test state.
// Expected Interaction
provider.addInteraction({
state: 'a task with id task-id exists',
uponReceiving: 'a request to delete task task-id',
withRequest: {method: 'DELETE', path: '/tasks/task-id'},
willRespondWith: {status: 204}
});

Defining Interactions
Examples
Successful interaction where data is returned
state: ‘tasks for project with id project-id exist’,
uponReceiving: ‘a request to get tasks for that project’
Successful interaction where no data is returned
state: ‘tasks for project with id project-id do not exist’,
404 error interaction
state: ‘project project-id does not exist’,
Domain validation error interaction
state: ‘a task task-id exists and domain rules exist that prevent its
modification’,
uponReceiving: ‘a request to update that task’
Successful interaction where data is created
state: ‘a project with name project-1 does not exist’,
uponReceiving: ‘a request to create a project with name project-1’

Live Documentation
Contracts are not only used by the tests, we also intent them to be user-friendly. Developers
can consult contracts in the Pact Broker who shows them in a nice documentation style.

CDC in our pipeline

CDC vs Contract Tests
TDD is more than having Unit Tests …
… CDC is more than having Contract Tests
Think in it as Outside-In TDD for a whole system!
Is what we do, a technique and attitude, discipline on the
design and development flow.

What is next?
● Pact v3
○ Contract testing for messages (for services that communicate via event streams and message
queues)
● Pact v4
○ Matching times and dates in a cross-platform manner
○ Additional Matchers
● … Beyond REST APIs. GraphQL?
○ Servers publish a statically typed system specific to their application, and GraphQL provides a
unified language to query data within the constraints of that type system.

Further reading
● Implementing Consumer-Driven Contract tests in Angular, Paulo Clavijo
https://paucls.wordpress.com/2017/08/04/pact-consumer-driven-contract-imple
mentation-in-angular
● Configure Pact JS Consumer Contract tests in Angular, Paulo Clavijo
https://paucls.wordpress.com/2017/08/04/configure-pact-js-consumer-contract-
tests-in-angular
● Sample project on Provider Side
https://github.com/paucls/task_list_api-kotlin-pact

References
● ContractTest, Martin Fowler. https://martinfowler.com/bliki/ContractTest.html
● Pact documentation https://docs.pact.io/documentation/implementation_guides.html
● Testing Strategies in a Microservice Architecture, Toby Clemson
https://martinfowler.com/articles/microservice-testing/
● Integrated Tests Are A Scam, J.B. Rainsberger
http://blog.thecodewhisperer.com/permalink/integrated-tests-are-a-scam
● Consumer-Driven Contracts: A Service Evolution Pattern, Ian Robinson
https://www.martinfowler.com/articles/consumerDrivenContracts.html
● Microservices, Flexible Software Architecture. Eberhard Wolff.
● Contract testing with GraphQL, Andreas Marek
https://www.andimarek.com/post/contract-testing-graphql
● https://github.com/DiUS/pact-jvm/tree/master/pact-jvm-provider-junit

Consumer-Driven Contract Testing

More Related Content

What's hot

Similar to Consumer-Driven Contract Testing

More from Paulo Clavijo

Recently uploaded

Consumer-Driven Contract Testing