Moving Towards Zero Defects with Specification by Example
Upcoming SlideShare
Loading in...5
×
 

Moving Towards Zero Defects with Specification by Example

on

  • 5,355 views

Love tracing bugs in a defect tracking system? Love the bug-fix cycle? If so, then don't come to this presentation. We'll be discussing how Specification by Example (also known as Acceptance Test ...

Love tracing bugs in a defect tracking system? Love the bug-fix cycle? If so, then don't come to this presentation. We'll be discussing how Specification by Example (also known as Acceptance Test Driven Development) will help move you towards a zero defect system by building the right thing the first time.

Statistics

Views

Total Views
5,355
Views on SlideShare
2,199
Embed Views
3,156

Actions

Likes
1
Downloads
41
Comments
0

14 Embeds 3,156

http://blog.zhariy.com 3052
http://winnipegagilist.blogspot.com 68
http://winnipegagilist.blogspot.ca 20
http://winnipegagilist.blogspot.nl 3
http://winnipegagilist.blogspot.in 2
http://translate.googleusercontent.com 2
http://winnipegagilist.blogspot.co.uk 2
http://winnipegagilist.blogspot.ch 1
http://winnipegagilist.blogspot.se 1
http://winnipegagilist.blogspot.com.au 1
http://winnipegagilist.blogspot.de 1
http://winnipegagilist.blogspot.co.nz 1
http://www.blogger.com 1
http://www.linkedin.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • SETUPFitNesse example- Test that these work and are prepped and then close them down: - Run C:\\_Projects\\SampleProjects\\SpecByExample\\server.FitNesse.bat - Open http://localhost:8088 in FireFox - Confirm tests are still all passing. - In FitNesse, remove page SpeakerInformation- Open C# project C:\\_Projects\\SampleProjects\\SpecByExample\\PrairieDevCon\\PrairieDevCon.sln - In VS, open "CalculateWeeklyPay" - In VS, remove LookupSpeakerInformation.cs- Setup 2 tables at the front for the experiment- Open C:\\_Projects\\FitNesse\\ folderOpen script: C:\\Users\\srogalsky\\Dropbox\\Presentations\\PrDC11\\SpecificationByExample\\ScriptForDemo.docx2. ExperimentPut out paperSetup 2 tablesSetup dividersPut examples in the dividersOpen spreadsheet: C:\\Users\\srogalsky\\Dropbox\\Presentations\\PrDC11\\SpecificationByExample\\AirplaneMfgResults.xlsPut instructions on both side – for QA, for team
  • Table Talk: Learners form small collaborative groups and discuss what they already know about the topic and any questions they have that are related to the topic. (Training from the back of the room)Ask them to:Form groups of 3 to 5Assign a note takerChoose 2 topicsReport a one or two sentence summary of each topicWrite the results on big post-its
  • Table Talk: Learners form small collaborative groups and discuss what they already know about the topic and any questions they have that are related to the topic. (Training from the back of the room)Ask them to:Form groups of 3 to 5Assign a note takerChoose 2 topicsReport a one or two sentence summary of each topicWrite the results on big post-its
  • Table Talk: Learners form small collaborative groups and discuss what they already know about the topic and any questions they have that are related to the topic. (Training from the back of the room)Ask them to:Form groups of 3 to 5Assign a note takerChoose 2 topicsReport a one or two sentence summary of each topicWrite the results on big post-its
  • Introduce myself here first.Tell my story here of the first project using this. 6 defects in 12 person months of work. 3 mediums, 3 lows. Arguably no defects since the 6 found were not identified as tests.Points:Cost?Is there a cost to TDD?What is the cost of a defect? (have you seen that chart?)The later you find it, the more screwed you are.Defects that matter“Like” button vs FB securityBuild it right the first timeBuild each piece little by little. Perfect on top of perfect. Simple…Normalization of devianceBoiling the frogIf you are using iterations, you need to regression test each iterationHow can you afford not to do some automated examples? Do you want to regression test your whole app manually at the end of each iteration? After each story is completed?Click a buttonZero defects is going be to required on more and more projects.Google carSmart cars that self regulate speed in car pool lanes once you enter themLots of Health initiativesWe’re moving to be ‘more’ dependent on perfect apps, not lessThe difference between what you think you should do and what you actually do is the degree to which you suckCory Haines
  • HypothesisDefinitionsProof
  • The stuff that QA and the customer does before saying it is done.Functional TestingAcceptance TestingNot Unit TestingWe’re not talking about TDD (Test Driven Development) today. Although related, TDD applies more to design, flexible code and unit testing. We’re talking about testing at a higher level – customer, QA, etc.
  • Happy customersRemove frustrations
  • Happy customersRemove frustrations
  • Makes requirements less ambiguous – an example laterGWT example shown, but lots of ways to do this – start with your existing test case formats and see what works for you.If you can’t do the rest of ATDD, do this! A great place to start.
  • Catches misunderstandings early so they aren’t duplicated throughout other user storiesImproves communication between QA and developers and customers (whole team!)Reduces the time spent writing, reading, understanding, arguing items in a defect trackerWhen we test, we execute the examples together that we wrote earlier
  • This is a silly as a high school end of year exam that is worth 60%. If we fail, it is as much the teacher’s fault as ours.
  • To prove early that the system works as expected and eliminate the waste of reworkRequires and investment time up front required to write the tests – pay off is laterRegression testing effort disappears (click a button)
  • To prove early that the system works as expected and eliminate the waste of reworkRequires an investment of time up front to write the tests – pay off is laterRegression testing effort disappears (click a button)
  • example courtesy Allan Shalloway - The Role of Quality Assurance in Lean-Agile
  • An example using FitNesseWe run this test against the code and see it passes!
  • Now with some overtime
  • Now with overtime and holiday hours
  • Now with the corrected codeThis is the beginning of the “aha” moment
  • Putting it all together – this is what your executable requirements document now looks like.Complete traceabilityPush button testing and regression now existsCan replace all business logic in your requirements documentCan replace much of your bug tracking effortThis should be the ‘aha’ moment.
  • Show Quick example using C#, FitNesse, PrDC APIStart the fitnesse batch fileShow the added referenceShow the fixture code for “CalculateWeeklyPay”Open FitNesse (http://localhost:8088/)Show the markup language for “WeeklyPayWithOvertime”Add an example for 40 hours and 10/hour = 500Press the buttonFix the errorShow HistoryShow the suiteRun the suiteShow the defect we found and fixed in the example earlier.Comment: This was my own biggest hurdle – starting…
  • Janet Gregory’s slide modified. Talk about the process. Talk about not Up Front, but just in time – one story at a time.Follow this little roll play:Script:=========Choose story (create a new FitNesse page and put this in there): Create “SpeakerInformation” pageAs an attendee I want to know some information about a speaker so that I can make decisions on which sessions to attend2. High Level Tests (add to FitNesse page):a short bio* name and contact info* social media information3. Expand into examples:'''User Story: Speaker Information'''''As an attendee I want to know some information about a speaker so that I can make decisions on which sessions to attend''* a short bio* name and contact info* social media information'''Scenario: Show the contact information for a speaker'''!-    -!Given a ''''''!-    -!When I look up their information on the website!-    -!Then it should return the ''''''!-    -!And ''''''!-    -!And ''''''!-    -!And ''''''!-    -!And '''''''''Examples:'''!|Lookup Speaker Information ||Speaker Name |Email? |Twitter Id?|Blog? |Company Name?|Picture? ||Steve Rogalsky|steve.rogalsky@protegra.com|@srogalsky |winnipegagilist.blogspot.com/|Protegra |steverogalsky.jpg|4. Review together5. Automate examplesusing System;using System.Collections.Generic;using System.Linq;using System.Text;namespace PrairieDevCon.FitnesseTests{ public class LookupSpeakerInformation : fit.ColumnFixture { public string SpeakerName; public string TwitterId; public string Blog; public string CompanyName; public string Picture; public string Email() {varspeakerInformation = PrairieDevCon.Services.SpeakerService.GetSpeakerByName(SpeakerName);TwitterId = speakerInformation.Twitter; Blog = speakerInformation.Blog;CompanyName = speakerInformation.Company; Picture = speakerInformation.Picture; return speakerInformation.Email; } }}6. Run the tests until done!
  • At this point we gather our volunteers and explain the experiment.Explain:First, the teams may not talk from this point forwardQAPreferably acted out by a non-QA person(ask who are the testers, and pick a non-tester)Will accept/reject the end productHas a little leeway in accepting – see the ‘test cases’ provided for you. As long as it is pretty close, you can accept it. I am your customer, so I can help you with that.If it isn’t right, please write up the defect on your paper using words or pictures and hand the defect log and the product back to the develop team to fixAsk the QA to person to write good defects – as a dev he/she will know how frustrating an unclear defect isManufacturing team = DevelopersThere will be a few steps to build the end productThese steps are similar to the steps that you create to build software – the classes, database tables, UI, layers, etc. QA won’t test the individual classes and steps, but they will perform functional or acceptance testing on the final productYou can pair program if you like, it is up to you to determine the best way to accomplish the task – but again, NO TALKINGTeam 1Will not use ATDD – they will test at the end and to simulate this they will not see the ‘test cases’ that QA ownsTeam 2Will use ATDD – they will create their test case at the beginning to simulate the effort of automating the tests up front and they will have access to the ‘test cases’ throughout developmentBoth teamsReminder – no talkingReminder – no fighting or blaming – everyone is doing the best they can with what they have been givenDon’t copy the other team’s product, the requirements are similar, but the test cases may not be the sameTry to be as successful as possible without cheatingIf you have a defect, don’t throw out the product and start fresh, but fix the defective productWe’ll be timing both teams and then examine the results through a value stream before we debrief with questions
  • Once the exercise is done:First: Fill out the spreadsheetSecond: Ask the observers to reportThird – ask questions:How did it go?How did you feel as you were developing (calm on one side, not on the other)What went well?What didn’t go well?Refer back to the original questions asked and their answers – how does this fit?Fourth: my observationsHow it related to agile: removing barriers; build the right thing the first time; fail fast not slow

Moving Towards Zero Defects with Specification by Example Moving Towards Zero Defects with Specification by Example Presentation Transcript

  • (Track Sponsor)
    Moving towards zero defects with Specification By Example
    Steve Rogalsky
    @srogalsky
    winnipegagilist.blogspot.com
  • Choose 2 of these topics to discuss:
    Talk about your approach to quality and the results of that approach.
    Talk about your approach to requirements and how you keep those requirements up to date.
    Talk about any previous experience with automated testing.
    Talk about your team’s current bug tracking process.
    Talk about zero defects – Is it possible? Is it responsible? Cost effective?
  • A story of Zero Defects
  • A SCIENCE EXPERIMENT
  • HYPOTHESIS:
    Specification By Example improves Quality & Speed
  • DEFINITIONS:
    Acceptance Testing Is...
  • What is
    Specification By Example?
    Goal: To
    build the right thing
    the first time.
  • What is
    Specification By Example?
    Also known as / similar to:
    ATDD – Acceptance Test Driven Development
    BDD – Behaviour Driven Development
  • To do this we:
    1.
    WRITE EXAMPLES
    (Acceptance Tests)
    (up front but not UP FRONT)
    instead of requirements
    Given muppet <Animal>
    When measuring <Craziness>
    Then return <10>
    Given muppet <Animal>
    When <Drumming>
    Then return <Phenomenal Skillz>
    Given muppet <Animal>
    When <talking>
    Then return <Grunt>
    SPECIFICATION BY EXAMPLE
  • To do this we:
    2.
    TEST AS SOON
    AS POSSIBLE
    FIRST
    in
    collaboration
    with the developers and customers
  • To do this we:
    3.
    AUTOMATE our examples
  • AN EXAMPLE:
    Requirement:
    Bring home something small from Europe
    What I brought:
    What she wanted:
  • ANOTHER EXAMPLE
    User Story:
    As an employee I want to receive overtime pay
    For each week, hourly employees are paid:
    2 times their wage for each hour worked on Sundays and holidays
    a standard wage per hour for the first 40 hours worked
    1.5 times their wage for each hour after the first 40 hours
  • (40*$20) = $800.00
    a standard wage per hour for the first 40 hours worked
  • (40*$20) + (5*$20*1.5) = $950.00
    1.5 times their wage for each hour after the first 40 hours
  • 2 times their wage for each hour worked on Sundays and holidays
    (40*$20) +
    (8*$20*1.5) +
    (8*$20*2)
    = $1,360.00
  • 2 times their wage for each hour worked on Sundays and holidays
    (40*$20) +
    (8*$20*1.5) +
    (8*$20*2 ) = $1,360.00
    *1.5
  • Executable Specifications
    Credit: This example is from “The Role of Quality Assurance in Lean-Agile” – Allan Shalloway
  • Defect Tracking
  • Where should we focus our automated testing?
    Later…
    Focus Here First
    UI
    Services
    TDD
    Classes/
    Functions
    Credit: Testing Triangle first shown by Patrick Wilson-Welsh
  • What does it take to set this up?
    It is simpler than this…
    '''
  • Steps:
    Download FitNesse (free)
    Run FitNesse
    Add A Reference to your project
    Create a Fixture per set of examples
    Write the examples
    Press a button
  • How to do it?
    Tester
    Customer
    and Team
    Automate Examples
    Expand into Examples
    Choose Story
    High Level Tests
    Passes All tests = done
    Review
    Code / TDD
    Think about how
    Developer
  • THE EXPERIMENT
  • Airplane Factory!
    Steps:
    • Fold the paper in half
    • Open the folded paper up and make two triangle folds for the ‘nose’ of the plane
    • Re-fold the plane in half and fold two wings (one on each side…)
    • Finally, fold two wing flaps
  • CONCLUSIONS
  • TO SUMMARIZE
    Specification By Example:
    Communicate!
    Write Executable Examples instead of requirements
    The Tooling is simple to use and free
    Zero Defects isn’t impossible – build it right the first time
    Trash your Defect Tracker
  • THANKS!
    Questions?
    Links
    Contact Info
    http://gojko.net/fitnesse/
    steve.rogalsky@protegra.com
    http://fitnesse.org/
    @srogalsky
    winnipegagilist.blogspot.com
    http://seleniumhq.org/
    http://www.slideshare.net/SteveRogalsky/moving-towards-zero-defects-with-specification-by-example