PPTX, PDF6,946 views

Moving Towards Zero Defects with Specification by Example

The document discusses the concept of 'specification by example' as a method to improve software quality and speed by using executable examples rather than traditional requirements. It emphasizes the importance of collaboration in testing and automating the examples to ensure the product is built right the first time, suggesting that achieving zero defects is not impossible. The document provides various examples and a simple framework to implement this approach effectively.

Technology◦

(Track Sponsor)Moving towards zero defects with Specification By ExampleSteve Rogalsky@srogalskywinnipegagilist.blogspot.com

Choose 2 of these topics to discuss:Talk about your approach to quality and the results of that approach.Talk about your approach to requirements and how you keep those requirements up to date.Talk about any previous experience with automated testing.Talk about your team’s current bug tracking process.Talk about zero defects – Is it possible? Is it responsible? Cost effective?

HYPOTHESIS:Specification By Example improves Quality & Speed

What is Specification By Example?Goal: To build the right thing the first time.

What is Specification By Example?Also known as / similar to: ATDD – Acceptance Test Driven Development BDD – Behaviour Driven Development

To do this we:1.WRITE EXAMPLES (Acceptance Tests)(up front but not UP FRONT)instead of requirementsGiven muppet <Animal> When measuring <Craziness> Then return <10>Given muppet <Animal> When <Drumming> Then return <Phenomenal Skillz>Given muppet <Animal> When <talking> Then return <Grunt>SPECIFICATION BY EXAMPLE

To do this we:2.TEST AS SOON AS POSSIBLEFIRSTincollaboration with the developers and customers

AN EXAMPLE:Requirement: Bring home something small from EuropeWhat I brought:What she wanted:

ANOTHER EXAMPLEUser Story: As an employee I want to receive overtime payFor each week, hourly employees are paid:2 times their wage for each hour worked on Sundays and holidaysa standard wage per hour for the first 40 hours worked1.5 times their wage for each hour after the first 40 hours

(40*$20) = $800.00a standard wage per hour for the first 40 hours worked

(40*$20) + (5*$20*1.5) = $950.001.5 times their wage for each hour after the first 40 hours

2 times their wage for each hour worked on Sundays and holidays(40*$20) + (8*$20*1.5) +(8*$20*2) = $1,360.00

2 times their wage for each hour worked on Sundays and holidays(40*$20) + (8*$20*1.5) +(8*$20*2 ) = $1,360.00*1.5

Executable SpecificationsCredit: This example is from “The Role of Quality Assurance in Lean-Agile” – Allan Shalloway

Where should we focus our automated testing?Later…Focus Here FirstUIServicesTDDClasses/FunctionsCredit: Testing Triangle first shown by Patrick Wilson-Welsh

What does it take to set this up?It is simpler than this…'''

Steps:Download FitNesse (free)Run FitNesseAdd A Reference to your projectCreate a Fixture per set of examplesWrite the examplesPress a button

How to do it?TesterCustomer and TeamAutomate ExamplesExpand into ExamplesChoose StoryHigh Level TestsPasses All tests = doneReviewCode / TDDThink about howDeveloper

Airplane Factory!Steps: Fold the paper in half

Open the folded paper up and make two triangle folds for the ‘nose’ of the plane

Re-fold the plane in half and fold two wings (one on each side…)

Moving Towards Zero Defects with Specification by Example

1.
(Track Sponsor)Moving towardszero defects with Specification By ExampleSteve Rogalsky@srogalskywinnipegagilist.blogspot.com
2.
Choose 2 ofthese topics to discuss:Talk about your approach to quality and the results of that approach.Talk about your approach to requirements and how you keep those requirements up to date.Talk about any previous experience with automated testing.Talk about your team’s current bug tracking process.Talk about zero defects – Is it possible? Is it responsible? Cost effective?
5.
A story ofZero Defects
6.
A SCIENCE EXPERIMENT
7.
HYPOTHESIS:Specification By Exampleimproves Quality & Speed
8.
DEFINITIONS:Acceptance Testing Is...
9.
What is SpecificationBy Example?Goal: To build the right thing the first time.
10.
What is SpecificationBy Example?Also known as / similar to: ATDD – Acceptance Test Driven Development BDD – Behaviour Driven Development
11.
To do thiswe:1.WRITE EXAMPLES (Acceptance Tests)(up front but not UP FRONT)instead of requirementsGiven muppet <Animal> When measuring <Craziness> Then return <10>Given muppet <Animal> When <Drumming> Then return <Phenomenal Skillz>Given muppet <Animal> When <talking> Then return <Grunt>SPECIFICATION BY EXAMPLE
12.
To do thiswe:2.TEST AS SOON AS POSSIBLEFIRSTincollaboration with the developers and customers
14.
To do thiswe:3.AUTOMATE our examples
15.
AN EXAMPLE:Requirement: Bringhome something small from EuropeWhat I brought:What she wanted:
16.
ANOTHER EXAMPLEUser Story:As an employee I want to receive overtime payFor each week, hourly employees are paid:2 times their wage for each hour worked on Sundays and holidaysa standard wage per hour for the first 40 hours worked1.5 times their wage for each hour after the first 40 hours
17.
(40*$20) = $800.00astandard wage per hour for the first 40 hours worked
18.
(40*$20) + (5*$20*1.5)= $950.001.5 times their wage for each hour after the first 40 hours
19.
2 times theirwage for each hour worked on Sundays and holidays(40*$20) + (8*$20*1.5) +(8*$20*2) = $1,360.00
20.
2 times theirwage for each hour worked on Sundays and holidays(40*$20) + (8*$20*1.5) +(8*$20*2 ) = $1,360.00*1.5
21.
Executable SpecificationsCredit: Thisexample is from “The Role of Quality Assurance in Lean-Agile” – Allan Shalloway
22.
Defect Tracking
25.
Where should wefocus our automated testing?Later…Focus Here FirstUIServicesTDDClasses/FunctionsCredit: Testing Triangle first shown by Patrick Wilson-Welsh
26.
What does ittake to set this up?It is simpler than this…'''
27.
Steps:Download FitNesse (free)RunFitNesseAdd A Reference to your projectCreate a Fixture per set of examplesWrite the examplesPress a button
28.
How to doit?TesterCustomer and TeamAutomate ExamplesExpand into ExamplesChoose StoryHigh Level TestsPasses All tests = doneReviewCode / TDDThink about howDeveloper
29.
THE EXPERIMENT
30.
Airplane Factory!Steps: Fold the paper in half
31.
Open the folded paper up and make two triangle folds for the ‘nose’ of the plane
32.
Re-fold the plane in half and fold two wings (one on each side…)
33.
Finally, fold two wing flapsCONCLUSIONS

Editor's Notes

#2 SETUPFitNesse example- Test that these work and are prepped and then close them down: - Run C:\\_Projects\\SampleProjects\\SpecByExample\\server.FitNesse.bat - Open http://localhost:8088 in FireFox - Confirm tests are still all passing. - In FitNesse, remove page SpeakerInformation- Open C# project C:\\_Projects\\SampleProjects\\SpecByExample\\PrairieDevCon\\PrairieDevCon.sln - In VS, open "CalculateWeeklyPay" - In VS, remove LookupSpeakerInformation.cs- Setup 2 tables at the front for the experiment- Open C:\\_Projects\\FitNesse\\ folderOpen script: C:\\Users\\srogalsky\\Dropbox\\Presentations\\PrDC11\\SpecificationByExample\\ScriptForDemo.docx2. ExperimentPut out paperSetup 2 tablesSetup dividersPut examples in the dividersOpen spreadsheet: C:\\Users\\srogalsky\\Dropbox\\Presentations\\PrDC11\\SpecificationByExample\\AirplaneMfgResults.xlsPut instructions on both side – for QA, for team
#3 Table Talk: Learners form small collaborative groups and discuss what they already know about the topic and any questions they have that are related to the topic. (Training from the back of the room)Ask them to:Form groups of 3 to 5Assign a note takerChoose 2 topicsReport a one or two sentence summary of each topicWrite the results on big post-its
#4 Table Talk: Learners form small collaborative groups and discuss what they already know about the topic and any questions they have that are related to the topic. (Training from the back of the room)Ask them to:Form groups of 3 to 5Assign a note takerChoose 2 topicsReport a one or two sentence summary of each topicWrite the results on big post-its
#5 Table Talk: Learners form small collaborative groups and discuss what they already know about the topic and any questions they have that are related to the topic. (Training from the back of the room)Ask them to:Form groups of 3 to 5Assign a note takerChoose 2 topicsReport a one or two sentence summary of each topicWrite the results on big post-its
#6 Introduce myself here first.Tell my story here of the first project using this. 6 defects in 12 person months of work. 3 mediums, 3 lows. Arguably no defects since the 6 found were not identified as tests.Points:Cost?Is there a cost to TDD?What is the cost of a defect? (have you seen that chart?)The later you find it, the more screwed you are.Defects that matter“Like” button vs FB securityBuild it right the first timeBuild each piece little by little. Perfect on top of perfect. Simple…Normalization of devianceBoiling the frogIf you are using iterations, you need to regression test each iterationHow can you afford not to do some automated examples? Do you want to regression test your whole app manually at the end of each iteration? After each story is completed?Click a buttonZero defects is going be to required on more and more projects.Google carSmart cars that self regulate speed in car pool lanes once you enter themLots of Health initiativesWe’re moving to be ‘more’ dependent on perfect apps, not lessThe difference between what you think you should do and what you actually do is the degree to which you suckCory Haines
#7 HypothesisDefinitionsProof
#9 The stuff that QA and the customer does before saying it is done.Functional TestingAcceptance TestingNot Unit TestingWe’re not talking about TDD (Test Driven Development) today. Although related, TDD applies more to design, flexible code and unit testing. We’re talking about testing at a higher level – customer, QA, etc.
#10 Happy customersRemove frustrations
#11 Happy customersRemove frustrations
#12 Makes requirements less ambiguous – an example laterGWT example shown, but lots of ways to do this – start with your existing test case formats and see what works for you.If you can’t do the rest of ATDD, do this! A great place to start.
#13 Catches misunderstandings early so they aren’t duplicated throughout other user storiesImproves communication between QA and developers and customers (whole team!)Reduces the time spent writing, reading, understanding, arguing items in a defect trackerWhen we test, we execute the examples together that we wrote earlier
#14 This is a silly as a high school end of year exam that is worth 60%. If we fail, it is as much the teacher’s fault as ours.
#15 To prove early that the system works as expected and eliminate the waste of reworkRequires and investment time up front required to write the tests – pay off is laterRegression testing effort disappears (click a button)
#16 To prove early that the system works as expected and eliminate the waste of reworkRequires an investment of time up front to write the tests – pay off is laterRegression testing effort disappears (click a button)
#17 example courtesy Allan Shalloway - The Role of Quality Assurance in Lean-Agile
#18 An example using FitNesseWe run this test against the code and see it passes!
#19 Now with some overtime
#20 Now with overtime and holiday hours
#21 Now with the corrected codeThis is the beginning of the “aha” moment
#22 Putting it all together – this is what your executable requirements document now looks like.Complete traceabilityPush button testing and regression now existsCan replace all business logic in your requirements documentCan replace much of your bug tracking effortThis should be the ‘aha’ moment.
#28 Show Quick example using C#, FitNesse, PrDC APIStart the fitnesse batch fileShow the added referenceShow the fixture code for “CalculateWeeklyPay”Open FitNesse (http://localhost:8088/)Show the markup language for “WeeklyPayWithOvertime”Add an example for 40 hours and 10/hour = 500Press the buttonFix the errorShow HistoryShow the suiteRun the suiteShow the defect we found and fixed in the example earlier.Comment: This was my own biggest hurdle – starting…
#29 Janet Gregory’s slide modified. Talk about the process. Talk about not Up Front, but just in time – one story at a time.Follow this little roll play:Script:=========Choose story (create a new FitNesse page and put this in there): Create “SpeakerInformation” pageAs an attendee I want to know some information about a speaker so that I can make decisions on which sessions to attend2. High Level Tests (add to FitNesse page):a short bio* name and contact info* social media information3. Expand into examples:'''User Story: Speaker Information'''''As an attendee I want to know some information about a speaker so that I can make decisions on which sessions to attend''* a short bio* name and contact info* social media information'''Scenario: Show the contact information for a speaker'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!Given a '''<Speaker Name>'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!When I look up their information on the website!-&nbsp;&nbsp;&nbsp;&nbsp;-!Then it should return the '''<Email>'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!And '''<Twitter Id>'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!And '''<Blog>'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!And '''<Company Name>'''!-&nbsp;&nbsp;&nbsp;&nbsp;-!And '''<Picture>''''''Examples:'''!|Lookup Speaker Information ||Speaker Name |Email? |Twitter Id?|Blog? |Company Name?|Picture? ||Steve Rogalsky|steve.rogalsky@protegra.com|@srogalsky |winnipegagilist.blogspot.com/|Protegra |steverogalsky.jpg|4. Review together5. Automate examplesusing System;using System.Collections.Generic;using System.Linq;using System.Text;namespace PrairieDevCon.FitnesseTests{ public class LookupSpeakerInformation : fit.ColumnFixture { public string SpeakerName; public string TwitterId; public string Blog; public string CompanyName; public string Picture; public string Email() {varspeakerInformation = PrairieDevCon.Services.SpeakerService.GetSpeakerByName(SpeakerName);TwitterId = speakerInformation.Twitter; Blog = speakerInformation.Blog;CompanyName = speakerInformation.Company; Picture = speakerInformation.Picture; return speakerInformation.Email; } }}6. Run the tests until done!
#30 At this point we gather our volunteers and explain the experiment.Explain:First, the teams may not talk from this point forwardQAPreferably acted out by a non-QA person(ask who are the testers, and pick a non-tester)Will accept/reject the end productHas a little leeway in accepting – see the ‘test cases’ provided for you. As long as it is pretty close, you can accept it. I am your customer, so I can help you with that.If it isn’t right, please write up the defect on your paper using words or pictures and hand the defect log and the product back to the develop team to fixAsk the QA to person to write good defects – as a dev he/she will know how frustrating an unclear defect isManufacturing team = DevelopersThere will be a few steps to build the end productThese steps are similar to the steps that you create to build software – the classes, database tables, UI, layers, etc. QA won’t test the individual classes and steps, but they will perform functional or acceptance testing on the final productYou can pair program if you like, it is up to you to determine the best way to accomplish the task – but again, NO TALKINGTeam 1Will not use ATDD – they will test at the end and to simulate this they will not see the ‘test cases’ that QA ownsTeam 2Will use ATDD – they will create their test case at the beginning to simulate the effort of automating the tests up front and they will have access to the ‘test cases’ throughout developmentBoth teamsReminder – no talkingReminder – no fighting or blaming – everyone is doing the best they can with what they have been givenDon’t copy the other team’s product, the requirements are similar, but the test cases may not be the sameTry to be as successful as possible without cheatingIf you have a defect, don’t throw out the product and start fresh, but fix the defective productWe’ll be timing both teams and then examine the results through a value stream before we debrief with questions
#31 Once the exercise is done:First: Fill out the spreadsheetSecond: Ask the observers to reportThird – ask questions:How did it go?How did you feel as you were developing (calm on one side, not on the other)What went well?What didn’t go well?Refer back to the original questions asked and their answers – how does this fit?Fourth: my observationsHow it related to agile: removing barriers; build the right thing the first time; fail fast not slow

Moving Towards Zero Defects with Specification by Example

More Related Content

What's hot

Similar to Moving Towards Zero Defects with Specification by Example

More from Steve Rogalsky

Recently uploaded

Moving Towards Zero Defects with Specification by Example

Editor's Notes