How to Break Software: Robustness Edition
Upcoming SlideShare
Loading in...5
×
 

How to Break Software: Robustness Edition

on

  • 86 views

Have you ever worked on a project where you felt testing was thorough and complete—all of the features were covered and all of the tests passed—yet in the first week in production the software had ...

Have you ever worked on a project where you felt testing was thorough and complete—all of the features were covered and all of the tests passed—yet in the first week in production the software had serious issues and problems? Join Dawn Haynes to learn how to inject robustness testing into your projects to uncover those issues before release. Robustness—an important and often overlooked area of testing—is the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions. By expanding basic tests and incorporating specific robustness attacks, Dawn shows you how to catch defects that commonly show up first in production. She offers strategies for making robustness testing a project-level concern so those defects get the priority they deserve and are fixed before release. Join Dawn to learn about robustness tests you can add to your suite and execute in just a few minutes—even if your test team is over-tasked and under-resourced.

Statistics

Views

Total Views
86
Views on SlideShare
86
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    How to Break Software: Robustness Edition How to Break Software: Robustness Edition Document Transcript

    • TH AM Tutorial 4/30/13 8:30AM How to Break Software: Robustness Edition Presented by: Dawn Haynes PerfTestPlus, Inc. Brought to you by: 340 Corporate Way, Suite 300, Orange Park, FL 32073 888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
    • Dawn Haynes Dawn Haynes is COO, principal trainer, and consultant for PerfTestPlus, Inc., and a former director of the Association for Software Testing. Dawn’s unique blend of experience, humor, and effectiveness at providing tools and techniques that help students at all levels generate new approaches to common and complex software testing problems has resulted in her international recognition as an elite trainer of testers. She provides consulting services and is a frequent speaker at testing conferences, local groups, and intimate gatherings of testers. Dawn has more than twenty-eight years of experience supporting, administering, developing, and testing software and hardware systems—from small business operations to large corporate enterprises.
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 1
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 2
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 3
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 4
    • © 2013 PerfTestPlus, Inc. Random Strikes Exploratory Testing Bug Safari Fault Injection Error Guessing © 2013 PerfTestPlus, Inc. 5
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 6
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 7
    • © 2013 PerfTestPlus, Inc. Description                Miles © 2013 PerfTestPlus, Inc. 8
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 9
    • © 2013 PerfTestPlus, Inc. Attack • Directed and focused attempt to evaluate the quality, especially reliability, of a test object by attempting to force specific failures to occur. Ref: How to Break Software - Whittaker © 2013 PerfTestPlus, Inc. 10
    • Input Output Data Computation  Force all error messages to occur  Force software to use default values  Explore data types & character sets  Overflow input buffers  Find interacting inputs  Repeat inputs/sequence  Force different outputs to be generated for same input  Force invalid outputs to be generated  Force output properties to change  Force the screen to be refreshed  Apply inputs using a variety of initial conditions  Force a data structure to store too many or too few values  Try to modify alternate data constraints  Experiment with invalid operand and operator combinations  Exploit recursion  Force computation results to be too large or too small  Find features that share data or interact poorly Ref: How to Break Software - Whittaker © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 11
    • © 2013 PerfTestPlus, Inc.  Fill the file system to capacity  Force media to be busy or unavailable  Damage the media File Kernel  Assign invalid file name  Vary access permissions  Vary/corrupt file contents  Restrict or constrain resources like memory, CPU, threading, etc.  Make unavailable (uninstall, move or rename files, kill process) Other  Restrict or block access (disable methods, close ports, etc.) Software  Concurrency check (execute actions in other interacting software) Ref: How to Break Software - Whittaker © 2013 PerfTestPlus, Inc. 12
    • © 2013 PerfTestPlus, Inc. Error Guessing • A test design technique where the experience of the tester is used to anticipate what defects might be present in the component or system under test as a result of errors made, and to design tests specifically to expose them. © 2013 PerfTestPlus, Inc. 13
    • Business rules; logic • Subvert rules • Bend/break logic Workflows; state models Concurrency Algorithms • Skip steps • Illegal U-turn • Create zombies • Interacting models • Doing the same thing • Doing different things • Bust sorting logic • Trick search operations • Subvert parsing schemes © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 14
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 15
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 16
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 17
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 18
    • © 2013 PerfTestPlus, Inc. Robustness Testing Reliability Testing • Testing to determine the robustness of the software product. • The process of testing to determine the reliability of a software product. • An attribute of a component or system specified or implied by requirements documentation (for example reliability, usability or design constraints). [After IEEE 1008] Source: ISTQB Glossary, 2007 Feature © 2013 PerfTestPlus, Inc. 19
    • Validation Fault Tolerance • Confirmation by examination and through provision of objective evidence that the requirements for a specific intended use or application have been fulfilled. [ISO 9000] • The capability of the software product to maintain a specified level of performance in cases of software faults (defects) or of infringement of its specified interface. [ISO 9126] See also reliability, robustness. Source: ISTQB Glossary, 2007 © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 20
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 21
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 22
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 23
    • © 2013 PerfTestPlus, Inc. © 2013 PerfTestPlus, Inc. 24
    • Source: ISTQB Glossary, 2007 © 2013 PerfTestPlus, Inc. Robustness The ability of software to deliver value during normal use without stressing out users (or failing) The ability of software to deliver value consistently – throughout the longevity, intermittence, or frequency of interactions Reliability © 2013 PerfTestPlus, Inc. 25