In many organisations there’s a tension between the desires of the three amigos:
- business stakeholders can’t validate assumptions unless they’re written in business domain terms
- testers would ideally test everything end to end (vertically)
- developers respond that the testing pyramid encourages us to have more unit tests than integration or end-to-end tests
It is often recommended that some tests that start off as scenarios get pushed ‘down’ into unit tests to keep the execution time under control and constrain the maintenance burden of the feature suite. The trouble with this is that even if the business folk and the testers trusted the developer’s unit tests implicitly (which they often don’t) there’s still the issue of visibility. We no longer have one complete, generally consumable, source living documentation.
An approach that I have been experimenting with uses Cucumber’s tagged hooks to control the amount of application stack that a scenario exercises. This lets us tailor our execution context depending on the runtime of the feature suite and the amount of trust the team has to spare. In the limit, this allows us (where it makes sense) to expose some unit tests as scenarios – keeping our living documentation complete.
The cost, of course, is added complexity.