From YOW! night presentation in Melbourne and Sydney
Modern distributed architectures are increasingly composed of large numbers of decoupled, asynchronous components. In AWS, these components are plumbed together via services like SQS, Kinesis and S3 often integrated via small and frequent numbers of microservices or lambdas. But how do you test these architectures if they are cloud native?
It’s 2019, and we can do better than deploying the entire stack and running a battery of E2E tests against them.
In his talk, Matt will demonstrate how you can scale development of large-scale systems across teams, technology and process, and unlock the agility of your cloud-native architecture.
See https://www.eventbrite.com.au/e/yow-night-2019-melbourne-modern-testing-jul-23-tickets-63937866881#
3. 3
The New Microservice > After
API
External
Events
API
Outside World
Microservice
Interface
API
Master Data
API
Inside bounded context
Event Emitters
API
Materialised view /
cache
Events and
Processing
Data
Event Handlers
Processing
Offline Processing
4. 4
Provisioning
Amazon ML
The New Microservice > After > Example Team 1 Bounded Context
Place Order API
External
Events
Order Status API
Outside World
Internal Events
Microservice
Interface
Data, Events and
Processing
Suggestions API
Publish Event
to external
subscribers
Insights /
Predictions
Customer /
Other Event
Orders
Push API
Inside bounded context
CRM
Materialised view of
customer
Provisioning Event
External
Events
… API
5. 5
Provisioning
The New Microservice > After > Example Team 1 Bounded Context
External
Events
Outside World
Internal Events
Microservice
Interface
Data, Events and
Processing
Publish Event
to external
subscribers
Insights /
Predictions
Place Order API
Customer /
Other Event
Orders
Inside bounded context
Materialised view of
customer
Provisioning Event
CRM
Marketing
Automation
Customer
Support
6. 6
The New Microservice > After > IoT ExampleThe New Microservice > After > Example 2 > IoT Architecture
API Gateway
Sensor A
Sensor B
Sensor C
IoT Rule A
IoT Rule B
IoT Rule C
AWS IoT Core
Sensor Data A
Sensor Data C
Processed Events
Sensor A
Vendor
Device API
Real-time events
Sensor Data B
Analytics
Device Ingest Process Store
11. 11
Testing The New Microservice > Test Pyramid > Ideal Software Testing Pyramid
https://watirmelon.blog/testing-pyramids/
Don’t forget people - the
ones doing the testing!
Let me make this a bit
clearer for you...
12. 12
Testing The New Microservice > Test Pyramid > Software Testing Trophy
https://kentcdodds.com/blog/write-tests
13. 13
Testing The New Microservice > Test Pyramid > Testing Funnel?
https://medium.com/@copyconstruct/testing-microservices-the-sane-way-9bb31d158c16
14. 14
Testing The New Microservice > Test Pyramid > Round Earth Heuristic
https://www.satisfice.com/blog/archives/4947
15. Today’s focus
Architecture > Microservices
1. Testability
2. Reduce need for UI / end-to-end testing
3. Observability
28. Summary
● Separate protocol handling from
business logic
● Business logic shouldn’t change with
introduction of a new protocol
● Enables testability - Port, Adapter and
Business Logic are independently
testable
Testability > Lambda Pattern > Ports and Adaptors > Summary
29. WHAT WE ARE KNOWN FORTestability > Modularisation
Adapted from https://martinfowler.com/articles/microservice-testing
Adapters
Services
Domain
Repositories
Collaborators
External
Service
Test
modularity
31. WHAT WE ARE KNOWN FOR
31
Options
Testability > Testing Locally
● Use real services
● Stub services (e.g. Localstack)
● Stub SDK (e.g. Moto)
● Local Unit, Integration +
Component tests
33. WHAT WE ARE KNOWN FOR
33
Options
● Feature arms race
● Trustworthy?
● Integrated = slower / harder
● All cloud providers?
● Point at the real services
● Stub services (e.g. Localstack)
● Stub SDK (e.g. Moto)
● Local Unit, Integration +
Component tests
Testability > Testing Locally
Challenges
43. WHAT WE ARE KNOWN FORTestability > Testing Locally > Component Test > Order Provisioning Service
Show me
where?
Adapters
Services
Domain
Repositories
Collaborators
mock
In-memory database
44. Testability > Testing Locally > Component Test > Order Provisioning Service
Component test of
handler
45. Setup dependencies,
inputs and expected
output
Testability > Testing Locally > Component Test > Order Provisioning Service
46. Testability > Testing Locally > Component Test > Order Provisioning Service
Invoke handler with
kinesis-shaped input
47. Testability > Testing Locally > Component Test > Order Provisioning Service
Assert the handler
response
51. Problems with
automated e2e
tests
Integration > Contract Tests > Contracts
● Slow
● Easy to break
● Hard to fix
● Scales badly across teams
● Lots of set up maintenance
● $$ potentially costly
54. WHAT WE ARE KNOWN FOR
54
Mocks
Solved problems New problems
● Hard to keep both sides in
sync
● Fast feedback
● Few dependencies
● No dedicated environment
● Reliable
● Easy to debug
Integration > Contract Tests > Contracts
56. 56
Provisioning
Amazon ML
Integration > Contract Tests > Contracts
Place Order API
External
Events
Order Status API
Outside World
Internal Events
Microservice
Interface
Data, Events and
Processing
Suggestions API
Publish Event
to external
subscribers
Insights /
Predictions
Customer /
Other Event
Orders
Push API
Inside bounded context
CRM
Materialised view of
customer
Provisioning Event
External
Events
… API
57. 57
Provisioning
Amazon ML
Integration > Contract Tests > Contracts Everywhere!
Place Order API
External
Events
Order Status API
Outside World
Internal Events
Microservice
Interface
Data, Events and
Processing
Suggestions API
Publish Event
to external
subscribers
Insights /
Predictions
Customer /
Other Event
Orders
Push API
Inside bounded context
CRM
Materialised view of
customer
Provisioning Event
External
Events
… API
Contract
Contract
Contract
Contract
Contract
Contract
Contract
Contract
66. WHAT WE ARE KNOWN FORTestability > Modularisation > Consumer Test
Adapted from https://martinfowler.com/articles/microservice-testing
Adapters
Services
Domain
Repositories
Collaborators
External
Service
Show me
where?
67. WHAT WE ARE KNOWN FORTestability > Modularisation > Provider Test
Adapted from https://martinfowler.com/articles/microservice-testing
Show me
where?
Adapters
Services
Domain
Repositories
Collaborators
mock
In-memory database
68. 68
Integration > Contract Tests > Contracts
https://martinfowler.com/bliki/TestPyramid.html
Contract
Show me
where?
82. ● Contract tests are faster, easier to
maintain and more reliable than e2e
tests
● Works for both synchronous and async
communication
● Use contract tests to reduce your
end-to-end test reliance
● End-to-end tests closer represent the
user experience
● Contract testing doesn’t replace
functional testing!
Summary
Integration > Contract Tests
98. Summary
98
Summary
Problems
● Microservices + architecture has evolved
● ...and so must the pyramid!
● More diverse integrations
● More glue, cloud-native code
We’re all digital plumbers!
99. ● Use Ports + Adapters (Hexagonal
architecture) to increase modularity
● Enable better testability and test
modularity
● Mock SDK for local tests
● Component tests can help to reduce
need for e2e tests
There is no single test strategy that will
work for all cases!
Structure...
Summary > Structure
100. ● Contract tests are faster and more
reliable than E2E tests
● Use them to reduce your E2E footprint
● Contract tests enable independent
deployment
● They are not functional tests
Never forget forget the end user!
Reduce E2E...
Summary > Reduce reliance on E2E
101. ● Is the property of understanding a
system as illuminated by its outputs
● Allows us to ask questions
● Helps us discover and respond to
unknown, unknowns
It’s not monitoring, but there is significant
implementation overlap.
Observability...
Summary > Observability