Executable Specifications Agile Palooza

Executable Speciﬁca/ons Documents for So7ware that Chris Sterling Partner at Sterling Barton Cer/ﬁed Scrum Trainer Agile Consultant Thursday, January 14, 2010 1

Executable Speciﬁca/ons -‐ Topics to Discuss • What are Executable Speciﬁca/ons? • Why Create Them? • Acceptance Tests • When to Use • An Approach with Example • Available Tools 2 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 2

Executable Specifica/ons • Unambiguous defini/on of desired so7ware behavior • Executable documenta/on • Repeatable and specific tests • Regression tests to validate exis/ng so7ware behavior • The second-‐most detailed specifica/on of the customer’s request 4 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 4

Increasing Cost of Release Stabiliza/on Periods 500,000 375,000 250,000 125,000 Release 1 Release 2 Release 3 0 Release 4 Release 5 Release 6 Cost of Fixing Defects Cost for Feature Dev 6 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 6

Agile tes/ng happens at two levels • Acceptance Tests tell us whether the system does what the customer expects (“building the right code”) • Programmer Tests deﬁne whether the system does what the developers expect (“building the code right”) 11 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 11

Acceptance Tests • Are also called customer tests or func/onal tests • Tell us whether the system does what the customer expects • Enable Developers to know they’ve sa/sﬁed requirements • Can be run automa/cally at any /me by anyone • Helps us build the “right” so7ware • “Running, Tested Features”* * “A Metric Leading to Agility” – Ron Jeffries http://www.xprogramming.com/xpmag/jatRtsMetric.htm 12 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 12

User Stories Story 14 As a customer I want to check my order status online so that I can know when to expect my package 13 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 13

User Stories Story 14 As a customer I want to check my order status online so that I can know when to expect my package A small piece of business value that can be delivered in an iteration 13 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 13

What is a User Story?* • CARD – Token represen4ng the requirement. It's used in planning. Notes are wriDen on it, reﬂec4ng priority and cost • CONVERSATION – The requirement itself is communicated from customer to programmers through conversa4on (The conversa4on is largely verbal, but is oMen supplemented with documents) • CONFIRMATION – The conﬁrma4on provided by the acceptance test is what makes possible the simple approach of card and conversa4on – When the conversa4on about a card gets down to the details * “Essential XP: Card, Conversation, Confirmation” – Ron Jeffries http://www.xprogramming.com/xpmag/EXPCardConversationConfirmation.htm 14 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 14

Conﬁrma/on through Acceptance Tests • Product Owner makes ﬁrst pass at Acceptance Criteria • During Itera/on Planning Acceptance Criteria are discussed • Final Acceptance Criteria for each User Story is product of nego/a/on between Delivery Team and Product Manager • Should be short, easy to understand statements Story 14 As a customer, I want to check my order status online so that I can know when to expect my package 15 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 15

Conﬁrma/on through Acceptance Tests • Product Owner makes ﬁrst pass at Acceptance Criteria • During Itera/on Planning Acceptance Criteria are discussed • Final Acceptance Criteria for each User Story is product of nego/a/on between Delivery Team and Product Manager • Should be short, easy to understand statements Story 14 Acceptance Criteria •View status as “waiting for pickup”, As a customer, I want to check “en route” or “delivered” my order status online so that I •Date of each step in route can know when to expect my •Estimated time of delivery package 15 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 15

Take on the Role of a User -‐ “What should the so7ware do next for me?” This ques/on helps you to decide what the next acceptance test should model 16 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 16

Tradi/onal balance of tests Manual GUI Easy to Create Acceptance Very familiar – what we always do Tests Typically tedious How do we know coverage? Need Automation specialists Automated Automation good for performance GUI Tests Seems like we always rewrite Sometimes fragile Unit Tests What is Development testing? How do we know what these are? How do we know when they fail? (Originally discussed by Mike Cohn) 19 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 19

Mike Cohn’s Tes/ng Pyramid Small number Automate most GUI Acceptance Tests Drive development and acceptance Executable Specifications Do the most Create Test Driven Design Unit Tests (Also “borrowed” from Mike Cohn) 20 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 20

Working with Legacy So7ware Define Analyze What Acceptance Might Be Criteria for Affected by Requirement Requirement 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21

Working with Legacy So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21

Working with Legacy So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now Write Failing Executable Specifications for How it Should Work 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21

Working with Legacy So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now Carefully Write Failing Modify Source Executable Code to Specifications Implement for How it Requirement Should Work 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21

Working with Legacy So7ware Define Analyze What Write Acceptance Might Be Executable Criteria for Affected by Specifications Requirement Requirement for How it Works Now Execute Carefully Write Failing Executable Modify Source Executable Specifications Code to Specifications to Verify Implement for How it Change Requirement Should Work 21 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 21

Working with Legacy So7ware • Focus on Wri/ng Executable Specifica/ons for New Requirements and Func/onality that may be Affected • Stabilize Legacy So7ware with Black-‐box Tests (Executable Specifica/ons) • Incrementally Improve the Internal So7ware Design • Try to Iden/fy where Programmer Tests can be Added • Execute Regression Test Suite that Includes Executable Specifica/ons Frequently (more than once per day if possible) 22 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 22

So7ware Test Automa/on Manual Test Script Login as Step 1 Administrato r Create Step 2 Order with 1 Item Click “Check Step 3 Out” Button Verify Item in Step 4 Shopping Cart 23 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 23

So7ware Test Automa/on Manual Test Script Login as Step 1 Administrato Automate r Create Step 2 Order with 1 Item Click “Check Step 3 Out” Button Verify Item in Step 4 Shopping Cart 23 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 23

Commercial Oﬀ-‐the-‐Shelf (COTS) Create Version 1: Executable Specifications for COTS Configurations & Customizations Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24

Commercial Oﬀ-‐the-‐Shelf (COTS) Create Version 1: Executable Specifications for COTS Configurations & Customizations Software Version 2: COTS Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24

Commercial Oﬀ-‐the-‐Shelf (COTS) Create Version 1: Executable Specifications for COTS Configurations & Customizations Software Execute Existing Version 2: Executable Specifications to COTS Find Upgrade Path Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24

Commercial Oﬀ-‐the-‐Shelf (COTS) Create Version 2: Executable Specifications for COTS Configurations & Customizations Software 24 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 24

Fit Workﬂow* • Starts with whiteboard conversa/on • Customer (SME, business person, product manager, etc.) informally describe new feature • Programmers and testers ask ques/ons *Source: http://fit.c2.com/ 27 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 27

Fit Workﬂow* • Customer, working with delivery team, reﬁnes examples into tables • Use business-‐friendly tools, such as Microso7 Excel and Word, to capture test tables *Source: http://fit.c2.com/ 28 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 28

Fit Workﬂow* • Delivery team creates Fit “ﬁxtures” (small piece of code that translates test tables into execu/on tests against the so7ware) *Source: http://fit.c2.com/ 32 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 32

Fit Workﬂow* • Execute Fit document • Click to edit Master text styles against so7ware • At ﬁrst some tests should be failing (red) *Source: http://fit.c2.com/ 33 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 33

Fit Workﬂow* • Delivery team • Click to edit Master text styles collaborates with customer to incrementally enhance test tables • Delivery team implements so7ware to meet test execu/on (green) *Source: http://fit.c2.com/ 34 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 34

Fit Workﬂow* • Document is kept for regression tes/ng • Document is included in automated build to ensure everything keeps working • As ques/ons arise about func/onality an example is added and Fit reports the answer *Source: http://fit.c2.com/ 35 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 35

Organizing FIT tests • Maintain suite of regression tests from past itera/ons that always pass • Run “regression” tests with build. • Maintain a suite of “in progress” tests for the current itera/on – Begin the itera4on with all tests failing – End the itera4on with most tests passing • At the end of the itera/on, move newly passing tests into regression suite – Beware the Fitnesse “Refactor/Move” command 37 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 37

Automated Regression Tes/ng Application Under Development Automated Design Changes Regression New Test Run Feature 38 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 38

Manual Regression Tes/ng • Tes/ng was taking 75 person hours during 2 full test runs consis/ng of: – Comprehensive manual regression tes4ng – Data conversion and valida4on • Cost for tes/ng was $17,000 each itera/on 40 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 40

Introducing Fit into Tes/ng Process • A7er 8 itera/ons team had introduced healthy amount of Fit ﬁxtures and automated tests • Reduced 70+ hour test run/me down to 6 hours which now included: – Fit automated regression tes4ng – Data conversion and valida4on automated with Fit ﬁxtures • Reduced cost of tes/ng each itera/on from $17,000 to $7,000 41 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 41

Table-‐based Acceptance Tes/ng Tools • Fit – hlp://ﬁt.c2.com/ • FitNesse – hlp://www.ﬁtnesse.org/ • RobotFramework -‐ hlp://code.google.com/p/ robonramework/ 43 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 43

Programmer Acceptance Tes/ng Tools • Cucumber -‐ hlp://cukes.info/ • Rspec -‐ hlp://rspec.info/ • JBehave -‐ hlp://jbehave.org/ • JMeter -‐ hlp://jakarta.apache.org/jmeter/ • xUnit (JUnit/NUnit/CppUnit/etc…) 44 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 44

Web UI Agile Tes/ng Tools • Open Source Web UI Tes/ng tools – StoryTestIQ – WATiR – Selenium – Canoo Web Test • GUI tests tend to be more brille • Tests wrilen and maintained incrementally • Most of these tools have code integra/on 45 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 45

What about tradi/onal tes/ng tools? • Why don’t we use commercial products such as SilkTest, TestDirector, QuickTest Pro, etc. for Agile tes/ng? – Expensive – Automated tools are record-‐and-‐playback; briDle – Ties our tests to the UI implementa4on – Manual tools (TestDirector) take too long to run. – Work ﬁne as an interim strategy (especially if you already have the licenses) – Consider adding open source Agile Tes4ng tool as a component of your tes4ng strategy 46 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 46

Presenter: Chris Sterling Partner at Sterling Barton • Technology Consultant, Cer/ﬁed Scrum Trainer and Agile Coach • Consults on so7ware technology across a spectrum of industries • Consults organiza/ons on Agile development, management, and enterprise prac/ces • Founder of Interna/onal Associa/on of So7ware Architects (IASA) Puget Sound chapter • University of Washington Lecturer: Agile Developer Cer/ﬁcate Program Email: chris@SterlingBarton.com Our Blog: hlp://www.getngagile.com Follow me on Twiler: @csterwa 48 Copyright © 2010 Sterling Barton. All rights reserved. Thursday, January 14, 2010 48

Executable Specifications Agile Palooza

by Chris Sterling on May 28, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 150

Statistics

Executable Specifications Agile Palooza — Presentation Transcript

http://chrissterling.gettingagile.com	70
http://www.gettingagile.com	70
http://www.slideshare.net	10