The document summarizes the speaker's approach to automated testing at their company Listings. Their key points are:
1. They focus on short feedback loops in testing and builds to help developers identify and fix bugs quickly. Tests and builds should be fast.
2. Their "test pyramid" is non-traditional, with fast broad stack tests using in-memory implementations of side-effecting components like databases.
3. They invest in tooling to provide really good test failure feedback.
4. They only run tests affected by a code change to keep the testing process fast. They modularize their codebase with testing in mind.
2. Who the hell am I?
Jordi Pradel - @agile_jordi - Agilogy
• Good at...
• Software development
• Quality and value obsessed
• Technology enthusiast
• Functional programming
• Doing the best I can at...
• Being father of 2 (4 and 6)
4. Why, oh why?
• So....
1. You should follow Charity Majors @mipsytipsy
2. You should read Daniel Worthington-Bodart's article
3. I'll talk a bit about my approach to automated testing
5. me
Every penny you invest in developers ergonomics
is worth it.
That's indeed a hill I'm willing to die on.
6. Developer ergonomics
My focus today
• Short feedback loops in testing (and builds)
• Builds should run as fast as possible
• Tests should run as fast as possible
• Tests purpose is to
f
ind bugs... so we expect them to fail. When they do
so, the developer should understand what the problem is as faster as
possible.
• Short feedback loops in product: Continuous Delivery
• Also requires fast builds
7. Our approach to testing
Our test pyramid. Spoiler: It isn't one.
Fast broad stack tests
Really good test failure feedback
Test only what changed
9. Our test pyramid
• Unit tests for complex, pure functions (no side effects to test)
• Integration tests for components performing side effects: databases,
f
iles,
message queues, etc.
• Service tests at the business level
• End to end tests to test API's / messaging / ports to get to the services
10. How do we do it?
• Let's say you want to test some "thing" A. Follow these steps:
1. Set up initial (known) state
2. Exercise A and collect result
3. Assert the result collected in 2 is the expected result
4. Collect
f
inal state
5. Assert the
f
inal state collected in 4 is the expected
f
inal state
• What about side effects? (e.g. sending an email?)
12. Daniel Worthington-Bodart
If you have some production constraint that is slow (say Oracle) then
separate the oracle integration tests out from your acceptance tests. I
create fast production quality implementations (in memory, local disk, h2,
hsql, lucene etc) that for-
f
ill the same contract. And by this I mean I have
either an abstract contract test that all implementations inherit or
parametrise the test that takes a list of implementations.
13. • We run broad stack tests
purely in memory
• No actual DynamoDB /
Elasticsearch / ...
• We use In memory
production grade
implementations of side
effecting components
14. Production grade in memory
implementations
• We run the very same tests
over:
• The actual side-effecting
components
• The in memory components
19. Test only what changed
• Team mono-repository
• We can modularize and use
libraries with minimal burden
• We only test (& build & deploy)
those modules / subprojects /
apps that changed
• Modularize with testing in mind
too