This talk is entitled “Automate your Agile Acceptance Tests with F#”. Upon compiling the slides I realised that the message I really wanted to get across is “Verifying software....”, provides loads of value as part of an Agile process if done properly.http://en.wikipedia.org/wiki/Behavior_Driven_Developmenthttp://blog.dannorth.net/introducing-bdd/http://en.wikipedia.org/wiki/Acceptance_testinghttp://gojko.net/resources/whatisfitnesse.pdfhttp://gojko.net/2010/06/16/anatomy-of-a-good-acceptance-test/This talk was
Good vs. BadClass names:Description of arrangement for test vs. Name of class being testedState/Arrangement:Set-up state controlled by the setup fun vs. State created for each test inside the test method.Method names:Description of action and expectation vs. Name of method being tested.What is being tested:Behaviour vs. Member functions.
http://www.stickyminds.com/sitewide.asp?Function=edetail&ObjectType=ART&ObjectId=13833&tth=DYN&tt=siteemail&iDyn=2Key part here is “A good unit test needs to define the behavioural contract”These kind of observations were what prompted Dan North into evolving TDD into BDD, which is what we will discuss next.
Just to re-iterate – language is key to good BDD.We don’t want to be speaking a different language to our stake holders.http://upload.wikimedia.org/wikipedia/en/3/38/Avatarjakeneytiri.jpg
This is a description of what BDD is and how it relates to TDD.
In an agile process we specify behaviour in terms of stories and acceptance tests.http://en.wikipedia.org/wiki/User_story#Benefits
Stories can be vague without acceptance tests, which leaves them open to scope creep.Scenarios are written in the language of business domain, and are as easy for business participants to understand as they are for developers.
Allows: - Shared understanding - Diffusion of language into the If the Sapir-Whorf hypothesis is correct and the language you use effects the way you think then this could be very beneficial.
This is a small subset of the frameworks available that enable specification execution.There are many frameworks out and they all tend to follow the gherkin style of interpretation.Therefore they are highly interchangeable without having to change your tests.I have used the bottom three on the right. Each time I have made minor adjustments to allow the double-ticked F# methods.Spec Flow seems to be the framework of choice at the moment largely due to it integration with visual studio. It is very good, and I would recommend it. The only gripe I have is that it requires the features to be stored in an assembly, which doesn’t give you much freedom over where you store your files.
I was recently working on an extension to our calculations system that is about 5 years old and written in C++. My first concern upon working on the system was that I didn’t break the current behaviour. Theimplementation had no tests of any sort so I had no safety net. And the scenarios I needed to test were fairly complex and not particularly suited to unit testing. I decided to make use of our XML API. Using the API as my interface I was able to build a test client using F# and use BDD to drive it and verify expectations.
Agile acceptance testing with f sharp
AGILE ACCEPTANCE TESTING WITH F#<br />Zach Bray<br />2010<br />
Outline<br />Test Driven Development (TDD)<br />Implementing TDD<br />Behaviour Driven Development (BDD) <br />Implementing BDD<br />.NET BDD frameworks<br />Case study: Extending a C++ project<br />
Behaviour Driven Development <br />“Is an agile software development technique that encourages collaboration between<br />Developers<br />QA<br />Business Participants<br /> in a software project.”<br />“It extends TDD by writing test cases in a <br />natural language that non-programmers can read.”<br />
User Stories/Features<br />Small chunks of business value.<br />Creation and priority driven by stake holders.<br />
Acceptance Tests/Scenarios<br />Write in the language of the business domain<br /> i.e., high-level of abstraction<br />
Prerequisites for executing scenarios<br />Define an interpretable language.<br />Domain Specific Language (DSL)<br />Ubiquitous language<br />Natural language<br />Choose a BDD framework<br />
The BDD Framework (Unit Test generation)<br />
Scenario description line matching example<br />Types are tagged as language contexts.<br />Each line in the file, e.g.,<br />“given a customer buys 2black jumpers”<br />Matches a member function.<br />Use double-ticked for great readability and DRYness<br />
State within scenarios<br />In the previous example state was changed inside the language context.<br />Contexts are created on a per scenario basis.<br />State doesn’t pass through to the next scenario<br />
BDD Patterns: Templates<br />Keeping things DRY with table driven templates.<br />Useful for driving many similar test cases.<br />given a customer buys a <colour> jumper<br />and I have <x> <colour> jumpers left in stock<br />when he returns the jumper for a refund<br />then I should have <y> <colour> jumpers in stock<br />examples:<br />| x | y | colour |<br />| 1 | 2 | green |<br />| 3 | 4 | black |<br />| 5 | 6 | red |<br />
BDD Patterns: Tables<br />Using tables makes data input much more readable.<br />Tables are passed into functions.<br />given the following bids exist in the Cape/Panamax market<br />| qty@price | credit |<br />| 1000@ 12.9 | good |<br />| 1000@ 12.7 | bad |<br />| 800 @ 12.5 | good |<br />and the following offers exist in the Panamax market<br />| qty@price | credit |<br />| 1000@ 12.9 | good |<br />when I observe implied orders for the Cape market<br />then I should see the following bids<br />| qty@price | credit |<br />| 1000@ 0.0 | good |<br />
BDD Patterns: Templated Tables<br />given the following bids exist in the Cape/Panamax market<br />| qty@price | credit |<br />| 1000@ <price1> | <cred1> |<br />| 1000@ 12.7 | bad |<br />| 800 @ 12.5 | good |<br />and the following offers exist in the Panamax market<br />| qty@price | credit |<br />| 1000@ <price2> | <cred2> |<br />when I observe implied orders for the Cape market<br />then I should see the following bids<br />| qty@price | credit |<br />| 1000@ <calPrice> | <calCred> |<br />examples:<br />| price1 | price2 | calPrice | cred1 | cred2 | calCred |<br />| 1.0 | 1.0 | 0.0 | good | good | good |<br />| 1.0 | 0.0 | -1.0 | good | good | good |<br />| 2.50 | 2.50 | 0.0 | bad | good | bad |<br />
Representation of complex scenarios<br />Plain text is dull.<br />Rich text is cool.<br />It makes the point of the scenario pop out.<br />Tables don’t look as ugly!<br />Confluence BDD tool<br />It downloads features from a Confluence Wiki.<br />Then transforms them into executable features.<br />
BDD Patterns: More<br />Lots of examples online.<br />Search for gherkin.<br />Look at http://cukes.info/<br />
Benefits of using BDD<br />Shared understanding of goals and progress.<br />It is independent of the implementation.<br />Use the same tests on many implementations.<br />Regression suite.<br />Nightly regression tests.<br />Reduces QA lag at the start of the project. <br />QA can start specifying behaviour straight away.<br />
Costs of BDD<br />Requires more interaction with stake holders.<br />Requires more development effort than not testing at all.<br />
Costs of not using BDD<br />Cost of wrong requirements.<br />Cost of poor design.<br />Cost of changing implementation and having to refactor all of the tests.<br />
The role of Unit Tests<br />Primary coverage should come from behaviour specifications.<br />Unit Tests should be used pragmatically.<br />Documentation<br />Edge cases<br />
Case Study: Indirect Aggression Project<br />Support indirect aggression of implied prices.<br />Requires extension to C++ Library.<br />Verification is biggest problem.<br />Not suited to Unit Tests.<br />Behaviour can be tested at API.<br />F# made it easy to model the API in few lines.<br />Language interpretation was reused.<br />Starting to build up a regression suite.<br />