Damian Gordon
   Component Testing
   Integration Testing
   System Testing
   Acceptance Testing
   Component Testing
   Component Testing

   Also known as Unit Testing, Module Testing,
    and Program Testing
   Component Testing

    ◦ Searching for defects in, and verifies the
      functioning of software that are separately
      testable.
    ◦ This testing can be done in isolation of other parts
      of the system, using stubs and drivers.
                  A            A          Driver




                  B           Stub          B
   Component Testing

    ◦ May include the testing of the functionality and
      specific non-functional characteristics such as
      resource behaviour, e.g. Memory leaks
    ◦ Also Robustness testing or performance testing
    ◦ Also structure testing, e.g. Decision coverage
   Integration Testing
   Integration Testing

    ◦ Testing interfaces between components, as well
      as interactions to different parts of the system,
      such as an operating system, and a file system.

    ◦ Component Integration Testing is testing
      interactions between components
    ◦ System Integration Testing is testing the
      interaction of the developed system with other
      systems
   Integration Testing

    ◦ It’s generally better to integrate components to
      each other in groups, and then test that process,
      rather than a “big bang” integration when all
      discrete components are integrated
      simultaneously.
    ◦ With the “big bang” approach, it is very difficult to
      trace the cause of failures to one specific
      component.
    ◦ The opposite approach is integrating one
      component at a time, which can often be too time
      consuming.
   Integration Testing

    ◦ This incremental approach to integration leads to
      a number of possible approaches:

      Top-Down: Test starts at the top, e.g. From the GUI
       or main menu
      Bottom-up: One component at a time
      Functional Incremental: Integration and testing on
       the basis of functionality
   System Testing
   System Testing

   Looking at the behaviour of the whole
    system as defined by the scope of a
    development project.
   It may include tests based on risks and/or
    requirements specification, business
    processes, use cases, or other high-level
    descriptions of system behaviour,
    interactions with the operating system, and
    system resources.
   System Testing

   It is most often the final testing on behalf of
    the development to verify that the system
    delivered meets the specification.
   System Testing

   It should include investigation of both the
    functional and non-functional requirements
    of the system
    ◦ Typical non-functional tests include performance
      and reliability.
    ◦ Typical functional tests include black-box testing.
   System Testing

   It must be undertaken in a controlled test
    environment, with software versions,
    testware, and test data.
   The test environment needs to match the
    live environment as much as possible.
   Acceptance Testing
   Acceptance Testing

   This is testing done by the customers/users,
    potentially as well as other stakeholders.
   The goal of this testing is to establish
    confidence in the system, focussing on
    issues such as fit-for-purpose, and
    usability.
   Acceptance Testing

   Two types of acceptance testing:
    ◦ The user acceptance test focuses mainly on
      functionality thereby validating the fitness-for-
      use of the system by the users.
    ◦ The operational acceptance test validates whether
      the system meets the requirements for operation.
   Acceptance Testing

   Other types of acceptance testing:
    ◦ Contract acceptance tests performed against a
      contract’s acceptance criteria for producing the
      software.
    ◦ Compliance acceptance tests are preformed
      against regulations such as governmental, legal or
      safety regulations.
   Acceptance Testing

   If the software is intended for the mass market,
    customer testing is impractical, but feedback is
    needed, so it’s often done in a two stage
    process
    ◦ The first is Alpha Testing, this takes place at the
      developers site, a cross-section of potential users are
      invited to use the system, and developers observe the
      users and note problems.
    ◦ The second is Beta Testing, sends the system out to a
      cross-section of the users, who install it, and use it
      under real-world conditions. The users send records of
      incidents with the system to the development
      organisation where the defects are repaired.
   Functional Testing
   Non-functional Testing
   Structural Testing
   Confirmation and Regression Testing
   Functional Testing
   Functional Testing

   The function of a system is “what it does”.
   Typically described in requirements
    document, or in use cases.
   Can be based on ISO 9124, but focusing on
    suitability, interoperability, security,
    accuracy, and compliance.
   ISO is an international
    standard-setting body
    composed of
    representatives from
    various national
    standards
    organizations.
    Founded on February
    23, 1947, the
    organization
    promulgates
    worldwide proprietary,
    industrial and
    commercial standards.
    It has its headquarters
    in Geneva,
    Switzerland.
   Functional Testing

   Two types, Requirements-based testing and
    Business-process-based testing
    ◦ Requirements-based testing use the functional
      specification to develop tests.
    ◦ Business-process-based testing uses the
      knowledge of business processes.
   Non-Functional Testing
   Non-Functional Testing

   Looking at quality characteristics, looking at
    how well something is being done, or how
    fast.
   Includes things like, performance testing,
    load testing, stress testing, usability testing,
    maintainability testing, reliability testing,
    and portability testing.
   Non-Functional Testing

   Can be based on ISO 9124, but focusing on
    reliability, usability, efficiency,
    maintainability, and portability.
   Structural Testing
   Structural Testing

   Looking at the system architecture or
    structure of the system or component.
   Often used as a way of measuring the
    thoroughness of the testing through
    coverage of a set of structural elements of
    coverage items.
START




                    Read in A




                    Does A/2
Print “It’s   Yes     give a    No   Print “It’s
  Odd”              remainder          Even”
                         ?




                      END
START




                Read in A and
                      B




          Yes                   No
Print A            A>B?              Print B




                    END
START



                       Read in A, B
                          and C




Yes              Yes                  No                Yes
         A>C?             A>B?             B>C?


            No                               No




      Print A            Print C              Print B




                          END
START


A=1


             A=A+1

  Is    No
             Print A
A==6?



Yes

 END
   Confirmation and Regression Testing
   Confirmation and Regression Testing

   Testing changes to software, two general
    types:
    ◦ Confirmation Testing
    ◦ Regression Testing
   Confirmation and Regression Testing

   Confirmation Testing
   When a test fails and we determine the
    cause of the failure, once that defect is
    fixed, we need to rerun the test again to
    confirm that the error has been corrected.
   It is important to rerun the test in the exact
    same way – same input, same data, and
    same environment.
   Confirmation and Regression Testing

   Regression Testing
   Testing to check if there hasn’t been any
    “unexpected side-effects” as a result of
    some change.

Software Testing 3/5

  • 1.
  • 3.
    Component Testing  Integration Testing  System Testing  Acceptance Testing
  • 4.
    Component Testing
  • 5.
    Component Testing  Also known as Unit Testing, Module Testing, and Program Testing
  • 6.
    Component Testing ◦ Searching for defects in, and verifies the functioning of software that are separately testable. ◦ This testing can be done in isolation of other parts of the system, using stubs and drivers. A A Driver B Stub B
  • 7.
    Component Testing ◦ May include the testing of the functionality and specific non-functional characteristics such as resource behaviour, e.g. Memory leaks ◦ Also Robustness testing or performance testing ◦ Also structure testing, e.g. Decision coverage
  • 8.
    Integration Testing
  • 9.
    Integration Testing ◦ Testing interfaces between components, as well as interactions to different parts of the system, such as an operating system, and a file system. ◦ Component Integration Testing is testing interactions between components ◦ System Integration Testing is testing the interaction of the developed system with other systems
  • 10.
    Integration Testing ◦ It’s generally better to integrate components to each other in groups, and then test that process, rather than a “big bang” integration when all discrete components are integrated simultaneously. ◦ With the “big bang” approach, it is very difficult to trace the cause of failures to one specific component. ◦ The opposite approach is integrating one component at a time, which can often be too time consuming.
  • 11.
    Integration Testing ◦ This incremental approach to integration leads to a number of possible approaches:  Top-Down: Test starts at the top, e.g. From the GUI or main menu  Bottom-up: One component at a time  Functional Incremental: Integration and testing on the basis of functionality
  • 12.
    System Testing
  • 13.
    System Testing  Looking at the behaviour of the whole system as defined by the scope of a development project.  It may include tests based on risks and/or requirements specification, business processes, use cases, or other high-level descriptions of system behaviour, interactions with the operating system, and system resources.
  • 14.
    System Testing  It is most often the final testing on behalf of the development to verify that the system delivered meets the specification.
  • 15.
    System Testing  It should include investigation of both the functional and non-functional requirements of the system ◦ Typical non-functional tests include performance and reliability. ◦ Typical functional tests include black-box testing.
  • 16.
    System Testing  It must be undertaken in a controlled test environment, with software versions, testware, and test data.  The test environment needs to match the live environment as much as possible.
  • 17.
    Acceptance Testing
  • 18.
    Acceptance Testing  This is testing done by the customers/users, potentially as well as other stakeholders.  The goal of this testing is to establish confidence in the system, focussing on issues such as fit-for-purpose, and usability.
  • 19.
    Acceptance Testing  Two types of acceptance testing: ◦ The user acceptance test focuses mainly on functionality thereby validating the fitness-for- use of the system by the users. ◦ The operational acceptance test validates whether the system meets the requirements for operation.
  • 20.
    Acceptance Testing  Other types of acceptance testing: ◦ Contract acceptance tests performed against a contract’s acceptance criteria for producing the software. ◦ Compliance acceptance tests are preformed against regulations such as governmental, legal or safety regulations.
  • 21.
    Acceptance Testing  If the software is intended for the mass market, customer testing is impractical, but feedback is needed, so it’s often done in a two stage process ◦ The first is Alpha Testing, this takes place at the developers site, a cross-section of potential users are invited to use the system, and developers observe the users and note problems. ◦ The second is Beta Testing, sends the system out to a cross-section of the users, who install it, and use it under real-world conditions. The users send records of incidents with the system to the development organisation where the defects are repaired.
  • 23.
    Functional Testing  Non-functional Testing  Structural Testing  Confirmation and Regression Testing
  • 24.
    Functional Testing
  • 25.
    Functional Testing  The function of a system is “what it does”.  Typically described in requirements document, or in use cases.  Can be based on ISO 9124, but focusing on suitability, interoperability, security, accuracy, and compliance.
  • 26.
    ISO is an international standard-setting body composed of representatives from various national standards organizations. Founded on February 23, 1947, the organization promulgates worldwide proprietary, industrial and commercial standards. It has its headquarters in Geneva, Switzerland.
  • 27.
    Functional Testing  Two types, Requirements-based testing and Business-process-based testing ◦ Requirements-based testing use the functional specification to develop tests. ◦ Business-process-based testing uses the knowledge of business processes.
  • 28.
    Non-Functional Testing
  • 29.
    Non-Functional Testing  Looking at quality characteristics, looking at how well something is being done, or how fast.  Includes things like, performance testing, load testing, stress testing, usability testing, maintainability testing, reliability testing, and portability testing.
  • 30.
    Non-Functional Testing  Can be based on ISO 9124, but focusing on reliability, usability, efficiency, maintainability, and portability.
  • 31.
    Structural Testing
  • 32.
    Structural Testing  Looking at the system architecture or structure of the system or component.  Often used as a way of measuring the thoroughness of the testing through coverage of a set of structural elements of coverage items.
  • 33.
    START Read in A Does A/2 Print “It’s Yes give a No Print “It’s Odd” remainder Even” ? END
  • 34.
    START Read in A and B Yes No Print A A>B? Print B END
  • 35.
    START Read in A, B and C Yes Yes No Yes A>C? A>B? B>C? No No Print A Print C Print B END
  • 36.
    START A=1 A=A+1 Is No Print A A==6? Yes END
  • 37.
    Confirmation and Regression Testing
  • 38.
    Confirmation and Regression Testing  Testing changes to software, two general types: ◦ Confirmation Testing ◦ Regression Testing
  • 39.
    Confirmation and Regression Testing  Confirmation Testing  When a test fails and we determine the cause of the failure, once that defect is fixed, we need to rerun the test again to confirm that the error has been corrected.  It is important to rerun the test in the exact same way – same input, same data, and same environment.
  • 40.
    Confirmation and Regression Testing  Regression Testing  Testing to check if there hasn’t been any “unexpected side-effects” as a result of some change.