Testing tests
Upcoming SlideShare
Loading in...5
×
 

Testing tests

on

  • 403 views

Strategies for improving the quality of automated tests.

Strategies for improving the quality of automated tests.

Wednesday Last Track Selenium Conference 2011

Statistics

Views

Total Views
403
Views on SlideShare
403
Embed Views
0

Actions

Likes
0
Downloads
7
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n

Testing tests Testing tests Presentation Transcript

  • Testing TestsIdeas for better quality test software
  • Brian Colfer brian.colfer@gmail.com @briancolfer
  • Introduction• Testing software is subject to error• False negatives are the main problem• Use Various strategies to help detect when there is a problem
  • Defects in tests• Same source as product defects:• Misunderstood requirements• Logic errors
  • Two types• False positives: type I errors we detect something when it doesnt exist• False negatives: type II errors we dont detect something when it does exit
  • Testing the test framework• Not controversial• Normal test tools and techniques• Frameworks are normal products
  • Track defects in tests• Have QA framework and tests as a part of defect tracking system• Have a release process for software framework and tests
  • Type I errors• Not as much of a worry but costs• Further examination will catch the error• Costs in delays and analysis effort
  • Type II errors• Insidious, you dont know that you dont know• Costs of the defect: normal user defect costs, repair costs and test repair costs.• Deterioration of confidence in software
  • Strategies• Unit tests• Reviews• Quality Control metrics
  • Unit tests• Unit tests validate that test code is correct not the application under test• Use TDD or BDD to write tests• Run unit tests as part of a CI practice for test development
  • Reviews• Step through the code looking for defects• Make it a frequent practice• Keep the scope digestible
  • Quality control metrics• Traditional QC doesn’t apply to software • QC will look for changes in power and sensitivity of the testing • Regression tests are supposed to not find errors• Track defects found by automated tests• Establish a baseline rate of defect detection
  • Multi method testing• Use fault injection to find test errors and ungraceful application failure• Exploratory testing• Don’t make your tests too DRY