Acceptance Test Driven Development

Acceptance Test Driven Development Naresh Jain naresh@agilefaqs.com Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 1

Tutorial Schedule Brainstorming Overview about Acceptance Criteria and Tests Demo Some Exercises Overview of Tools Details look at Fit, FitNesse and FitLibrary Hands on Session Patterns and Anti Patterns Questions welcomed all the time Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 2

Welcome Continuum Hopes and Concerns Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 3

Warmup Scenarios Pick one scenario and in relation to your scenario, what are the speciﬁc observable results that will tell you that the activity has been successfully completed? Going out for Movie (THX sound and Digital projection) Going out for meal (one veg.) Going shopping ($50) [10 Minutes] Present back to the group your ﬁndings. [3 minutes per group] Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 4

Acceptance Criteria and Tests: Deﬁnition Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 5

Acceptance Criteria & Tests: Deﬁnition Acceptance Criteria Is a set of conditions that the Story must meet for it to be accepted as complete Is typically provided by the customer or product owner. Is not a replacement for conversation. Is one of the results of the conversation Acceptance Criteria are NOT tests Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 6

Writing acceptance criteria Acceptance Criteria should contain: ACTOR VERB – DESCRIBING A BEHAVIOR OBSERVABLE RESULT To accommodate pre-conditions Acceptance Criteria can be expressed as Given [Precondition] When [Actor + Action] Then [Observable Result] Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 7

Acceptance Criteria & Tests: Deﬁnition Acceptance Tests Acceptance Criteria + Examples (data + scenarios) Acceptance Tests Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 8

Demo Roman Numerals to Decimal Conversion Example Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 9

Thinking in Tables Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 10

Thinking in Tables Only Tables Execute Ignored Executed Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 11

Thinking in Tables Foundational Table Structure Name of Fixture Interaction with Application Table structure depends on type of Fixture Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 12

Thinking in Tables 3 Foundation Fixtures Column Fixture Row Fixture Action Fixture copyright (c) 2007 by Micah Martin & Naresh Jain. All rights reserved. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 13

Thinking in Tables Column Fixture Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 14

Thinking in Tables Row Fixture Analogous to comparing against rows in a database table Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 15

Thinking in Tables Action Fixture Think GUI window Counter Window Counter: public class CountFixture extends Fixture { private int counter = 0; Counter: 6 public void count() { Count counter++; } public int counter() { return counter; } } Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 16

FitLibrary Extension to FIT Written by Rick Mugridge Adds some handy Fixtures Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 17

FitLibrary FitLibrary Fixtures ArrayFixture for ordered lists SetFixture for unordered lists SetUpFixture Supports Graphics Tree structures Nested Tables Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 18

FitLibrary DoFixture • Broken tables • Highly readable • Flexibility Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 19

Tools FIT Framework for Integrated Tests Created by Ward Cunningham Open Source The most accepted solution for agile acceptance testing Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 20

Tools FitNesse Environment build around FIT Makes everything easier Created by Object Mentor, Inc. Open Source Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 21

FIT FitNesse - Stand alone web server - Is a wiki - Tests written in HTML - Tests written in wiki text - Tests are executed on the command line - Tests are executed from within the wiki - Tables are executed - Translates tests into HTML - Non-table markup is ignored - Uses FIT to execute tests - Tables map to Fixtures - Supports test suites - Fixtures are code that is aware of the - Supports variables in tests system - Supports test refactoring - Supplies foundational Fixtures - Written in Java - Implementations ported to many - Supports FIT implementations in any languages language Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 22

Acceptance Criteria and Tests: A Critical Piece of Agile Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 23

Acceptance Criteria & Tests: A Critical Piece of Agile Traditional Approach 1 May 1 Jul 1 Sep 1 Nov Analysis Design Implementation ERD DFD DD ST Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 24

Key Questions Business Facing Are we building the right product? Are we building the product right? Technology Facing Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 25

Brian Marick’s Test Categorization Business Facing Supports Programming Critique product Technology Facing Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 26

Test Categorization Business Facing Drives Development Acceptance Testing Exploratory Testing Critique product Low-ﬁ prototypes UI and Usability Testing Unit Testing Performance Testing Technology Facing Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 27

Acceptance Criteria & Tests: A Critical Piece of Agile The Agile Approach May 1 Jul 1 Sep 1 Nov 1 Analysis Test Implementation Design Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 28

Test Driven Development Add a Test Pass Run the Test Fail TDD Rhythm - Test, Code, Refactor Make a little change Fail Run the Test Pass Refactor Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 29

Acceptance Test Driven Development Iteration Automated P Acceptance E Acceptance R Criteria Tests F O Automated R Story M T Unit Test Automated E E UI Tests N S C T E S Automated Acceptance Tests Acceptance Exploratory Criteria Testing Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 30

Mike Cohn’s Testing Pyramid Small in Number GUI Tools: Selenium, Sahi, Watir, Abbot, Frankenstein Tests At least one per story Acceptance Tools: Fit, FitNesse, RSpec, JBehave Tests At least one per class or module Unit Tests Tools: xUnit, TestNG Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 31

Acceptance Tests: A Critical Piece of Agile Criteria for DONE Every story must have at least one Acceptance Test A story is not DONE until it passes it’s Acceptance Tests Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 32

Acceptance Tests: A Critical Piece of Agile Manual Acceptance Tests Manual Acceptance Tests Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 33

Why Acceptance Tests? Criteria for Completion Great Collaboration tool Source of Feedback Real data to measure progress Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 34

Data From Acceptance Tests Total ATs Failing ATs Passing ATs 90 72 54 36 18 0 1 2 3 4 5 6 7 8 9 10 Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 35

Acceptance Tests Are Automated Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 36

Acceptance Tests: A Critical Piece of Agile The Button How often would you press it? When would you press it? Who would press it? Testers, Developers, Managers, Customers, Spectators, etc. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 37

Criteria for DONE + Automated Executable Speciﬁcation Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 38

Acceptance Tests: A Critical Piece of Agile Executable Speciﬁcation A new paradigm for testing Puts quality ﬁrst Removes ambiguity from requirements Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 39

Who Writes Acceptance Tests? Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 40

Who Writes Acceptance Tests? The Customer The Customer Role Stake holder Business Analyst Quality Assurance Product Owner Developer Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 41

Who Writes Acceptance Tests? Tests Get Technical The “Customer” may need technical help to write tests Developers and QAs are technical Pair test authoring Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 42

Who Writes Acceptance Tests? Business Rules Get Fuzzy Sometimes developers need help understanding tests Customers know business rules Pair test implementation Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 43

Exercise #1 Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 44

Exercise #1 The Login Test Write a test plan, in plain text, for the business rules of logging in. Web application User credentials are stored in relational database Successful login redirects to “Welcome” page Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 45

Writing Good Acceptance Tests Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 46

Writing Good Acceptance Tests Login Test Possibilities 1. Direct browser to URL for login page 1. Enter the username ‘wallace’ Build a Testable Environment First Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 47

Writing Good Acceptance Tests Login Test Possibilities 1. Add some users to the system 3. Enter a value into the username ﬁeld Be Speciﬁc Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 48

Writing Good Acceptance Tests Tests are Examples Use concrete examples Specify concrete behavior No ambiguity allowed Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 49

Writing Good Acceptance Tests Login Test Possibilities 1. Insert into User table values (’wallace’, ‘ilikecheeze’) 2. Open a browser to the URL http:// localhost/myapp Avoid Implementation Details Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 50

Good Acceptable Criteria and Tests S - SPECIFIC - Explicitly deﬁned and deﬁnite M - MEASURABLE - Possible to observe and quantify A - ACHIEVABLE - Capable of existing or taking place R - RELEVANT - Having a connection with the story T - TIME-BOUND – When will the outcome be observed Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 51

Writing Good Acceptance Tests Avoid Implementation Details Acceptance Tests View UI Model and Presenter Business Tier Data Store Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 52

Writing Good Acceptance Tests Login Test: Possible Solution Add user to system: (’wallace’, ‘ilikecheeze’) Process login with username ‘wallace’ and password ‘blah’ Check login failed Process login with username ‘wallace’ and password ‘ilikecheeze’ Check login succeeded Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 53

Tools Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 54

Tools Commercial Tools WinRunner TestPartner EggPlant Silk QTP TestComplete RFT Squish WindowTester Are not suitable for Acceptance Testing in an Agile environment Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 55

Tools Open Source Options FIT Sahi Frankenstein FitNesse Watir SharpRobo Selenium Abbot RSpec/JBehave Among the few tools that support Test Driven Development Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 56

Wiki Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 57

Wiki What is it? A collaborative web site Editable by any Created by Ward Cunningham Every project should have one http://c2.com/wiki http://en.wikipedia.com Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 58

Wiki Creating Tests Use Wiki syntax to create a page with test tables Label the page as a Test Page Use a page name of the form Test… Turn on the Test property Make sure your Fixtures are in the classpath Use !path widget Mechanics !path values are concatenated Java command to start FitServer is executed Testable HTML is passed to FitServer FitServer runs the tests Results are passed back to FitNesse Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 59

Wiki Creating Suites There are 2 ways to make Suites Set the Suite property Create a page with the Suite property Created test pages inside this page When the suite is executed, all child test pages will be included in the suite execution Use the !see widget !see <name of test page> All “included” tests pages will be included in the suite execution Run a Suite by clicking the Suite button Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 60

Hands-on Session Conference Proposal Submission Portal Some sample Stories Should be able to submit new proposal Should be able to list all submitted proposal Submitting proposal with same title should display appropriate error message Should be able to delete submitted proposal based on the title Should be able to delete submitted proposal based on the title Should be able to search proposals by title Should be able to search proposals by ID Should be able to find all proposal by an author's name Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 61

Break http://www.ﬁtnesse.org Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 62

Patterns Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 63

Patterns Organizing Tests Allowing customers to add new tests without breaking the build Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 64

Patterns Version Control Keeping the acceptance test in version control with the code. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 65

Patterns Cross-Functional Pairing Using FitNesse based acceptance tests for collaboration between cross- functional team members. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 66

Patterns ATDD Acceptance Test Driven Development Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 67

Patterns CSTT Cleanup, Setup, Test, Teardown Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 68

Patterns Independent Tests Tests shouldn’t depend on each other. Tests leave the system in the same state it started in. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 69

Patterns Dynamic Stubbing Avoiding complications of external systems. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 70

Patterns Non-Production Setup/Teardown Using non-production light weigh code for setup and teardown. Helps test only what you want to test. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 71

Patterns Suite Levels Creating different levels of suites depending on the depth/level of feedback desired. Smoke, Current Iteration/Sprint, Regression Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 72

Patterns DRY Using !include to avoid repeating yourself. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 73

Patterns Make it Real Write ATs as close as possible to the real environment. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 74

Patterns Fixture Evolution Allow Fixture implementation to evolve over time. Treat ﬁxtures as ﬁrst class citizens. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 75

Patterns At Least One Test/Story Every story should have at least one acceptance test Avoid long/multipurpose tests. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 76

Anti-Patterns Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 77

Anti-Patterns Developer ATs Developers writing acceptance tests by themselves, for themselves. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 78

Anti-Patterns Unit Testing Don’t write ATs at the unit testing level Unit tests are implementation speciﬁc ATs are NOT implementation speciﬁc Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 79

Anti-Patterns QA Testing Tool Hard to write tests up front. Perhaps only on large projects. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 80

Anti-Patterns Silver Bullet Trying to use FitNesse for all types of Acceptance Tests UI testing XML testing Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 81

Anti-Patterns Test After Writing tests after the code is already written. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 82

Anti-Patterns Hidden Test Data Hiding test data in the ﬁxtures. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 83

Anti-Patterns Implementation Dependent ATs Making test pages (tables) dependent on implementation details and data structures. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 84

Anti-Patterns Logging in Your Fixtures Putting log statements or print statements in the ﬁxture code. Fixtures are probably too complicated. Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 85

Reference Portions of this presentation is adopted from Micah Martin’s Introduction to Automated Acceptance Tests Presentation Kent Beck, Test Driven Development By Example. "Agile Testing Directions" - Brian Marick http://www.opensourcetesting.org/ Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 86

The End http://www.ﬁtnesse.org Licensed Under Creative Commons by Naresh Jain Saturday, November 29, 2008 87