• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Test design techniques: Structured and Experienced-based techniques
 

Test design techniques: Structured and Experienced-based techniques

on

  • 11,112 views

Test Design techniques: Structured and Experienced-based techniques

Test Design techniques: Structured and Experienced-based techniques

Statistics

Views

Total Views
11,112
Views on SlideShare
11,106
Embed Views
6

Actions

Likes
8
Downloads
0
Comments
0

4 Embeds 6

http://alef.fiit.stuba.sk 2
http://pinterest.com 2
http://static.slidesharecdn.com 1
https://www.linkedin.com 1

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • what is test case coverage? What is code coverage? How can you ensure that your test case covers all requirements? could you name some coverage items? + coverage items may be requirements, function calls, modules, objects, menu options, screens, buttons, transition steps, dropdown items, business transactions… + EP: percentage of equivalence partitions exercised + BVA: percentage of boundaries exercised + Decision tables: percentage of business rules or decision table columns tested + State transition testing: Percentage of states visited Percentage of (valid) transitions exercised Percentage of pairs of valid transitions exercised (1-switch coverage, 2-switch coverage) Percentage of invalid transitions exercised (state table)- What is the coverage interested by business analysts, testers or users?- What is the coverage interested by developers?
  • In reality, 100% statement coverage is time-consuming and could be achieved only by strict requirement from client. Normally, 80% of code coverage is good enough.
  • - To reach 100%SC, choose what test data? How about 1, 7? Test data? How many percentage? How about 1, 2, 6, 7? Test data? How many percentage?
  • Answer: aP = 40, Q = 70, %SC = ?  7/8P = 60, Q = 60, %SC = ?  8/8
  • Answer: b1, 2, 3, 4, 51, 2, 3, 5Outlook does not disappear, %DC = 50%
  • Reference BS7925-2, Software Testing Foundations (ISTQB)
  • a= (b+c) counts as 1 predicate
  • Answer: P = 4, C = 5
  • Answer: P = 5, C = 6
  • Answer: P = 5, C = 6
  • One drawback of code coverage measurement is that it measures coverage of what has been written, i.e. the code itself; it cannot say anything about the software that has not been written. If a specified function has not been implemented, specification-based testing techniques will reveal this. If a function was omitted from the specification, then experience-based techniques may find it. But structure-based techniques can only look at a structure which is already there. Best practice is to use all those techniques: spec-based and structure-based followed by exp-based testing techniques. Keep in the mind of objective of each technique.
  • Test objective is the most important

Test design techniques: Structured and Experienced-based techniques Test design techniques: Structured and Experienced-based techniques Presentation Transcript

  • Test Design techniques: Structured and Experienced-based techniques
    Author: Khuong Nguyen
  • Structured based testing techniques
    Cyclomatic complexity
    Experienced based testing techniques
    Choosing appropriate testing techniques
    Sample exam
    Agenda
  • Understand and apply common Structured-Based Testing techniques
    Know how to calculate Cyclomatic Complexity
    Understand Experienced-Based Testing techniques
    Learn to choose appropriate testing techniques in specific circumstances
    Objectives
  • Software testing foundations
    Certified Tester Foundation Level Syllabus 2007
    Software Testing Dictionary
    BS7925-2: Component Testing
    References
  • Structure-based techniques
  • Structure-based testing/white-box testing is based on an identified structure of the software or system
    It is a method for writing a set of white-box test cases that exercise the paths in the code
    How to measure thoroughly the test cases exercise the code
    Introduction
  • Test coverage measures in some specific way the amount of testing performed by a set of tests.
    Wherever we can count things and can tell whether or not each of those things has been tested by some test, then we can measure coverage
    What is test coverage?
  • Statement testing and coverage
    Decision testing and coverage
    Other structure-based techniques
    Basic white box test case design techniques
  • In component testing, statement coverage is the assessment of the percentage of executable statements that have been exercised by a test case suite. (TRUE OR FALSE)
    Statement testing derives test cases to execute specific statements, normally to increase statement coverage.
    Every statement is requested to be executed at least one
    Definition of statement coverage
  • Translate the source code to control flow graph
    Defines nodes(Statement) and control flow between statements(edges)
    Conditional statements(IF, CASE) and loops(WHILE, FOR) have more than one edges going out
    Verify that each statement have been executed
    Steps
  • Examples
    IF
    1
    IF
    2
    WHILE
    3
    4
    6
    ENDIF
    7
    ENDIF
    1 If (a>0) then
    2 If (b<1) then
    3 While (c<2)
    4 c++;
    5 End while
    End IF
    7 End IF
    5
    Node, statement
    Edge, Control flow
  • Statement coverage
    IF
    1
    IF
    2
    WHILE
    3
    4
    6
    ENDIF
    7
    ENDIF
    100%SC: All nodes can be reached by a single test case
    1, 2, 3, 4, 5, 6, 7
    5
  • It is a very weak criterion
    Value of the technique:
    Unreachable source code (dead code) can be detected
    Empty ELSE-parts are not considered
    Statement Coverage
    Coverage is measured by using test tools
  • What is minimum number of test cases required for full statement coverage?
    1 test for statement coverage
    2 tests for statement coverage
    3 tests for statement coverage
    Read P
    Read Q
    IF P+Q > 100 THEN
    Print “Large”
    ENDIF
    If P > 50 THEN
    Print “P Large”
    ENDIF
    Exercise
  • Decision coverage, related to branch coverage, is the assessment of the percentage of decision outcomes (TRUE and FALSE)
    Require every decision outcome: all possible cases-statement
    Decision testing is a form of control flow testing as it generates a specific flow of control through the decision points
    Definition of decision/branch coverage
  • Translate the source code to control flow graph
    Defines nodes(Statement) and control flow between statements(edges)
    Conditional statements(IF, CASE) and loops(WHILE, FOR) have more than one edges going out
    Verify that all possible branches of the control flow are tested
    Steps
  • It is a stronger criterion than statement coverage
    Be able to detect missing statement in empty branches
    Decision coverage is stronger than statement coverage: 100% decision coverage guarantees 100% statement coverage, but not vice versa.
    Decision coverage requires test of every decision outcome:
    both THEN and ELSE in the IF-statement
    all possibilities for the CASE-statement and fall-through case
    both execution of the FOR loop and the bypass
    Decision/branch Coverage
  • Examples
    IF
    1
    IF
    2
    WHILE
    3
    4
    6
    ENDIF
    7
    ENDIF
    1 If (a>0) then
    2 If (b<1) then
    3 While (c<2)
    4 c++;
    5 End while
    6 End IF
    7 End IF
    5
    Node, statement
    Edge, Control flow
  • Decision coverage
    IF
    1
    IF
    2
    WHILE
    3
    4
    6
    ENDIF
    7
    ENDIF
    100%DC: All branches are executed
    Some edges are executed more than one
    5
    1, 2, 3, 5, 3, 4, 5, 6, 7
    1, 2, 6, 7
    1, 7
    • Branch coverage = (Number of executed branches / total number of branches) * 100%
  • What is minimum number of test cases required for full decision coverage?
    1 test for decision coverage
    2 tests for decision coverage
    3 tests for decision coverage
    Switch PC on
    Start “outlook”
    IF outlook appears THEN
    Send an email
    Close outlook
    Endif
    Exercise
  • For object-oriented system, statement as well as decision coverage is inadequate.
    • Additional coverage criteria are necessary
     Tools often support determining coverage, coverage data can be used to detect uncalled methods or program parts
    Limitations of the techniques
  • Data flow Testing
    Branch Condition Testing
    Branch Condition Combination Testing
    Modified Condition Decision Testing
    LCSAJ Testing
    Other structure-based test techniques
  • There are stronger levels of structural coverage beyond decision coverage (condition coverage and multiple condition coverage)
    The concept of coverage can also be applied at other test levels (e.g. at integration level) where the percentage of modules, components or classes that have been exercised by a test case suite could be expressed as module, component or class coverage
    Tool support is useful for the structural testing of code
    The basis of white box techniques is the source code. Adequate test case design techniques chosen depending on the complexity of the program structure
    Other structure-based test techniques (cont.)
  • Testing contains the combination of different techniques
    The expected risk guides to select testing techniques and the intensity of execution
    Basis of the selection of the white box technique is the structure of the test object
    Which techniques should be chosen?
  • Cyclomatic Complexity
  • Cyclomatic complexity or conditional complexity:
    Was developed by McCabe
    Is the software metric (measurement) to measure the complexity of the program’s source code
    Definition
  • M = E − N + 2P
    M = cyclomatic complexity
    E = the number of edges (links) of the graph
    N = the number of nodes of the graph
    P = the number of connected components
    Number of predicates + 1
    How to calculate
  • Examples of cyclomatic complexity
    E=1, N=2, P=1
    M=1-2+2=1
    E=4, N=4, P=1
    M=4-4+2=2
    E=2, N=4, P=2
    M=2-4+4=2
    E=4, N=5, P=1
    M=4-5+2=1
  • Cyclomatic complexity of programming constructs
    1
    1 if E then
    2 A
    else
    B
    4 C
    2
    3
    4
    E = 4, N = 4, P =1
    M = 2
  • C
  • C
  • Predicate notes P = ?
    Complexity C = ?
  • Predicate notes P = ?
    Complexity C = ?
  • Predicate notes P = ?
    Complexity C = ?
  • Complex systems are
    hard to understand
    hard to change
    hard to reuse
    McCabe found that modules with a cyclomatic complexity greater than 10 were hard to test and error prone.
    Look for procedures with high cyclomatic complexity and rewrite them, focus testing on them, or focus reviewing on them.
    Metrics view
  • Experience-based techniques
  • Experienced-based test design technique: Procedure to derive and/or select test cases based on the tester’s experience, knowledge and intuition
    The techniques can be useful in identifying special tests not easily captured by formal techniques, especially when applied after more formal approaches
    Definition
  • Tester select test cases to uncover expected problems based on the skills, experienced and knowledge of the tester
    The term of error guessing is used very often in practice
    A commonly used experienced-based technique is error guessing
    Introduction
  • Exploratory testing
    Error guessing
    Basic experience-based test techniques
  • Error guessing is a test design technique where the experience of the tester is used to anticipate what defect might be present in the system under test.
    Testers design tests specifically to expose errors
    Error Guessing - Definition
  • Error guessing is a technique that should be used as a complement to other formal techniques
    Testers anticipate defects based on their experience, skills
    Knowledge in developing similar applications and using similar technologies are also used when designing test cases.
    “Guess” where errors happened based on intuition and experience of tester.
    Error guessing
  • These defect and failure lists can be built based on experience, available defect and failure data, and from common knowledge about why software fails.
    Error guessing
  • Exploratory testing is test design technique where the tester actively controls the design of the test as those tests are performed and uses information gained while testing to design new and better test
    Exploratory testing is exploring, finding out about the software:
    What it does
    What it doesn’t do
    What works
    What doesn’t work
    Definition of Exploratory testing
  • Exploratory testing is a hands-on approach in which testers are involved in minimum planning and maximum test execution
    Activities in exploratory testing is concurrent:
    Test planning
    Test design
    Test execution
    Test logging
    Learning
    Main features of exploratory testing
  • Result of one test case influence the design and execution of further test cases
    During testing, a “mental” model of program under test is created. It contains:
    How the program works
    How it behaves
    How it should behave
    Focus on finding further aspect/behavior of the program.
    Main features of exploratory testing
  • Planning: involve the creation of the test charter that includes:
    Test charter
    Short declaration of scope
    Time-boxed of effort
    Objectives and possible approaches to be used
    Test design and test execution are performed parallel:
    Do not have formal documents: test conditions, test cases, test scripts…
    Some notes written during exploratory testing session
    Test logging is undertaken as test execution is performed.
    The key aspect of exploratory testing is learning about the software, its strengths and weaknesses
    How to implement
  • Approach
    Execute few test cases
    Analyze results
    Consider “special” to next test cases
    Knowledge about the test object under test is collected
  • Possible elements of the test object are “explored” to decide which parts will be tested
    Result of Exploratory testing helps to determine which test techniques can be applied if there is time left
    The test of “test charter” should not take more than one or two hours of uninterrupted test time
    Benefit
  • The technique is applicable when:
    There no poor specification
    The time is severely restricted because it uses much less time than other techniques
    It can serves to complement formal testing
    It can serve to check on the test process, to help ensure that the most serious defects are found.
    When we use exploratory testing
  • Why? With which goal is the test run?
    What is to be tested?
    How? Which testing method should be used?
    Which problems should be found?
    Common questions for executing test charter
  • Choosing test techniques
  • What are the pros and cons of:
    Specification-based testing techniques
    Structure-based testing techniques
    Experience-based testing techniques
    What is the best practice?
    When to use what?
  • The choice of which test techniques to use depends on a number of factors:
    the type of system
    regulatory standards
    customer or contractual requirements
    level of risk, type of risk
    test objective, documentation available
    knowledge of the testers
    time and budget
    development life cycle, use case models
    previous experience of types of defects found.
    Which techniques should be chosen?
  • Testing contains the combination of different techniques
    The critically and the expected risk guide to select testing techniques and the intensity of execution
    Basis of the selection of the white box technique is the structure of the test object
    Which techniques should be chosen?
  • Thank You
    KhuongNguyen, Senior Quality Control Engineer
    Post graduate student, Australian Business School, Funds Management, UNSW, 2032, AustraliaEmail: khuong0602@gmail.com; op_khuong_nguyen@yahoo.com