Agile acceptance testing with f sharpPresentation Transcript
AGILE ACCEPTANCE TESTING WITH F# Zach Bray 2010
Outline Test Driven Development (TDD) Implementing TDD Behaviour Driven Development (BDD) Implementing BDD .NET BDD frameworks Case study: Extending a C++ project
Test Driven Development (TDD) cycle
Unit Tests - Good vs. Bad
Unit Testing “A good unit test needs both to illustrate and define the behavioral contract of the unit in question. Behavior is more than just individual methods…” – KevlinHenney
Behaviour Driven Development “Is an agile software development technique that encourages collaboration between Developers QA Business Participants in a software project.” “It extends TDD by writing test cases in a natural language that non-programmers can read.”
User Stories/Features Small chunks of business value. Creation and priority driven by stake holders.
Acceptance Tests/Scenarios Write in the language of the business domain i.e., high-level of abstraction
Prerequisites for executing scenarios Define an interpretable language. Domain Specific Language (DSL) Ubiquitous language Natural language Choose a BDD framework
The BDD Framework (Unit Test generation)
Scenario description line matching example Types are tagged as language contexts. Each line in the file, e.g., “given a customer buys 2black jumpers” Matches a member function. Use double-ticked for great readability and DRYness
State within scenarios In the previous example state was changed inside the language context. Contexts are created on a per scenario basis. State doesn’t pass through to the next scenario
Context Layer: Benefits of F# Discriminated Unions & Active Patterns Domain Modelling Object Expressions Implement interfaces Terse syntax Readability Units of measure Correctness Double ticked method names Clear & DRY Async Workflows & Agents Concurrency
BDD Patterns: Templates Keeping things DRY with table driven templates. Useful for driving many similar test cases. given a customer buys a <colour> jumper and I have <x> <colour> jumpers left in stock when he returns the jumper for a refund then I should have <y> <colour> jumpers in stock examples: | x | y | colour | | 1 | 2 | green | | 3 | 4 | black | | 5 | 6 | red |
BDD Patterns: Tables Using tables makes data input much more readable. Tables are passed into functions. given the following bids exist in the Cape/Panamax market | qty@price | credit | | 1000@ 12.9 | good | | 1000@ 12.7 | bad | | 800 @ 12.5 | good | and the following offers exist in the Panamax market | qty@price | credit | | 1000@ 12.9 | good | when I observe implied orders for the Cape market then I should see the following bids | qty@price | credit | | 1000@ 0.0 | good |
BDD Patterns: Templated Tables given the following bids exist in the Cape/Panamax market | qty@price | credit | | 1000@ <price1> | <cred1> | | 1000@ 12.7 | bad | | 800 @ 12.5 | good | and the following offers exist in the Panamax market | qty@price | credit | | 1000@ <price2> | <cred2> | when I observe implied orders for the Cape market then I should see the following bids | qty@price | credit | | 1000@ <calPrice> | <calCred> | examples: | price1 | price2 | calPrice | cred1 | cred2 | calCred | | 1.0 | 1.0 | 0.0 | good | good | good | | 1.0 | 0.0 | -1.0 | good | good | good | | 2.50 | 2.50 | 0.0 | bad | good | bad |
Representation of complex scenarios Plain text is dull. Rich text is cool. It makes the point of the scenario pop out. Tables don’t look as ugly! Confluence BDD tool It downloads features from a Confluence Wiki. Then transforms them into executable features.
BDD Patterns: More Lots of examples online. Search for gherkin. Look at http://cukes.info/
Benefits of using BDD Shared understanding of goals and progress. It is independent of the implementation. Use the same tests on many implementations. Regression suite. Nightly regression tests. Reduces QA lag at the start of the project. QA can start specifying behaviour straight away.
Costs of BDD Requires more interaction with stake holders. Requires more development effort than not testing at all.
Costs of not using BDD Cost of wrong requirements. Cost of poor design. Cost of changing implementation and having to refactor all of the tests.
The role of Unit Tests Primary coverage should come from behaviour specifications. Unit Tests should be used pragmatically. Documentation Edge cases
BDD Frameworks General RSpec Cucumber JBehave JSpec etc. .Net specific NBehave Nspec Cuke4Nuke SpecFlow StorEvil
SpecFlow: Integrated environment
Case Study: Indirect Aggression Project Support indirect aggression of implied prices. Requires extension to C++ Library. Verification is biggest problem. Not suited to Unit Tests. Behaviour can be tested at API. F# made it easy to model the API in few lines. Language interpretation was reused. Starting to build up a regression suite.