Agile Acceptance testing with Fitnesse

Bridging the Gap Creating better quality software using Agile Acceptance Testing and Fitnesse Clare McLennan clare.mclennan@gmail.com http://crazy-it-adventures.blogspot.com http://twitter.com/claremclennan

Contents • The Challenge • What means Testability? • Fitnesse • 0 to 90% in 9 months • Our System Test Toolbox • Success Factors

The Challenge

The Challenge • In the beginning, time to market was everything • As our customer base grew it became important that the system always worked • Big culture change • 99.99% uptime target • Releases weekly • System was hard to test

What means testability?

Testability • Any working system is testable • Aim for Easy to Test • How much – Setup – Knowledge is needed to test one aspect of the system? • See podcast, Test Driven Development is Design

Symptoms of Low Testability • Frustrating and slow to test anything • Testers needing continuous help from developers • Developers may believe testers are stupid • Developers avoid system testing and stop after unit testing succeeds • Poor understanding of the System develops • Can’t easily introduce new people to the project

Automated Testing System Test System Predict- Knowledge ability Debugging Status Methods Installer Tools

System Test System Predict- Knowledge ability Auto Debugging Status Tools Test Methods Installer

How We Improved Testability • Created an installer so everyone runs the system in the same way • Created a means to query the system for when processing is finished • Added business time to the system so we instantly test functionality that takes long periods of time Example: Conversion tracking works over a period of 30 days.

Fitnesse

Fitnesse in a Nut Shell • Means of capturing requirements as tests • Tests turn green if passed, red if failed • Requirements stay up to date • Customer or testers write the tests • Programmers write fixture code to make the tests run

Running a Suite of Tests

The Technical Side • Fitnesse is a wiki • Recommend to store tests with the code • Use SLIM (has replaced FIT) • Java, C++, Ruby, Python and more • Test fail when testers write them • Testers can reuse fixtures to create more tests

Sample Fixture

Sample Fixture Code

Script Fixture

Query Fixture

Test Organisation • Tests organised into heirachy of suites • SetUp and TearDown run before each test • Test History of success/failures • Tests can have explanatory text • Fixture toolbox documentation

How To Write Good Tests • Use user language, not programmer mumbo-jumbo • Make each test specific • Write cases not scripts - you should only specify things relevant for this example • Generally, if you can’t do it manually you won’t be able to automate it. • See http://www.concordion.org/Technique.html

Evolution of Our Tests (1) Test uses magic numbers from database – can't see what this test is about

Evolution of Our Tests (2) All this setup...

Evolution of Our Tests (3) ...replaced by this This is a system current implementation detail – not a requirement

Evolution of Our Tests (4) Hang on, these just set up data and don't test anything! We still aren't finished.. Ah better..

0 to 90% in 12 months

Creating Quality Processes Preprocesses • First Fitnesse tests were written to prove it was possible • First testers joined the project But... • Writing automated tests didn't catch on

Creating Quality Processes Stage 1 • QA group was formed to – Recruite and train testers – Write and program the Fitnesse automated tests – Test new functionality But... • QA group struggled to keep up with development effort

Creating Quality Processes Stage 2 • QA group continued to write tests. • QA group responsible for running tests • New Fixture requests were handed over to development team at start of sprint But... • QA group still needed needed system programmers knowledge to keep tests working • Hard to specify upfront all fixtures required • Programmers hated writing fixtures

Creating Quality Processes Stage 3 • Testers joined in development teams. • Testers responsibility to write tests • Dev teams responsibility to get tests running • Dev team given a test box to run tests on • Weekly QA meeting for testers to share changes and ideas

Creating Quality Processes Stage 3 Highly successful!! After initial teething in sprint (3 weeks) everyone was positive about the change

What Happened During Stage 3? • DBA sped up tests • We reduced the number of GUI functionality tests required because of good unit test coverage • Many manual testing issues were resolved • Finally testing and development occurred at the same pace • Programmers embraced writing tests as part of their job to maintain quality

Change in Testers Role • More about ensuring good specifications to prevent bugs • More testing time spent on exploratory testing • Better relationships with programmers • Less dull work • More influence on how the system is written to make testing easier

Our System Test Toolbox

Our System Test Toolbox • Ask User fixture • Business time • Staged Deployment • Separate Functionality, Gui Functionality, Gui Layout, Load, Full System, Sanity and Full system tests • Close communication between testers and programmers to find optimal test strategies

Ask User

Ask User • Mix and match human and automated processes • Allows tests to be written and run before all automation is ironed out – Example: Gui testing will eventually be automated with Selenium • User created objects can be referred to in Fitnesse tests • Simple idea but really practical!

Business Time • Changes the current time in the system • Allows testing of scenarios that take a long time • Only in testing mode • Low risk, as if forgotten system still works correctly in production environment

Staged Deployment • Think Beta testing • Test throughly, then do a partial release to – Only some customers, or – For a small proportion of the daily impressions, or – Run old and new system side by side • Gives a more accurate test

Types of testing Unit – Programmers tests. Pinpoint bugs quickly. Functionality – Pure, automatic testing of the whole system. All processes triggered Gui Functionality – Test functionality of Gui page at a time. All other objects required are generated as for functionality testing

Types of Testing Gui Layout – Pure gui test of layout, updating. Load – Use real database, or extra large database Full System – Main functionality,with processes on timers, as in production Sanity Check – Final check performed before a each release

Success Factors

Outcomes • Automated testing of the system takes one hour, plus some quick manual tests • Programmers attitude has changed from expecting outsiders to validate the system, to sharing responsibility for this task • System more easy to test, deploy, monitor and manage • A self-running QA process which is continually improved by development teams

Success Factors • Realistic time frame (12 months) • Treated building automated testing system as a project of it’s own • Influence over design of system to be tested • Started with high ROI, hard to manual test, functionality • Get tests working, then perfect • Gently challenged company culture

Future Improvements? • Automate GUI functionality tests with Selenium • Start sprints with a Specification Workshop • Improve Load tests - run them on a cloud • Move system knowledge from Fitnesse, back into the system • Make tests more user orientated

To learn more read Bridging the Communications Gap by Gojko Adzic

References • Fitnesse http://www.fitnesse.org • Bridging the Communications Gap by Gojko Adzic • Test Driven Development is Design - The Last Word on TDD, Hansel Minutes Podcast starring Scott Bellware and Scott Hanselman • Hints and Tips [for writing acceptance tests] by David Peterson http://www.concordion.org/Technique.html

http://crazy-it-adventures.blogspot.com	295
http://www.slideshare.net	58
http://crazy-it-adventures.blogspot.co.nz	14
http://crazy-it-adventures.blogspot.in	3
http://www.crazy-it-adventures.blogspot.co.nz	3
http://crazy-it-adventures.blogspot.com.au	2
http://crazy-it-adventures.blogspot.ca	2
http://crazy-it-adventures.blogspot.de	2
http://www.crazy-it-adventures.blogspot.com	1
http://crazy-it-adventures.blogspot.com.br	1

Agile Acceptance testing with Fitnesse

by ClareMcLennan on Jun 20, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

10 Embeds 381

Statistics

Agile Acceptance testing with Fitnesse — Presentation Transcript