Software Engineering Software Quality Assurance & Testing: Supplementary Materials

1. Software Engineering
(CSI 321)
Software Quality Assurance & Testing:
Supplementary Materials

2. QA vs. Software Testing
 Software Testing:
• Testing is the process of executing a system or
component under specified conditions with the
intent of finding defects and to verify that it satisfies
specified requirements.
• Testing is a product-oriented activity.
• Testing is oriented to bug-detection.
• Testing is one of the most important parts of quality
assurance (QA).

3. QA vs. Testing
 Software Quality Assurance:
• Defined as a planned and systematic approach to the
evaluation of the quality of and adherence to software
product standards, processes, and procedures.
• An umbrella activity that is applied throughout the software
process.
• Consists of a means of monitoring the software engineering
processes and methods used to ensure quality.
• An effective approach to producing high quality software.
• QA is a process-oriented activity.
• QA is oriented to bug-prevention.

4. What is Quality Control(QC)?
• QC is the set of activities designed to evaluate the quality of
developed product.
• QC is a responsibility internal to the team.
• QC is a product-oriented activity.
• The term QC is used in a production/hardware manufacturing
environment, where a large number of physical items are produced
and shipped. Each of the items has to go through a testing process
to ensure that the quality of the product is good enough for
shipment; otherwise the item is rejected. The quality check is
conducted by the quality control group within the manufacturing
organization, and the person who conducts the testing is called a
quality controller.

5. Verification vs. Validation
• Validation:
– Validation is a process that ensures the software product
meets the customer requirements.
– It is a high-level activity.
– Building the correct product
• Verification:
– Verification is a process that ensures the software product
works properly.
– It is a low-level activity.
– Building the product correctly

6. Testing Levels
1. Unit Testing
2. Integration Testing
3. System Testing
4. Acceptance Testing

7. Testing Levels
Unit Testing:
• Testing of individual software components
• First level of dynamic testing
• Typically white-box testing
• Usually done by programmers
• AKA: Component testing, module testing

8. Testing Levels
Integration Testing:
• Testing of two or more units/modules together
• Objective is to detect Interface defects between
units/modules

9. Testing Levels
System Testing:
• Conducted on complete, integrated system
• Ensures entire integrated system meets
requirements
• Black-box in nature
• Done before Performance testing

10. Testing Levels
Acceptance Testing:
• Formal testing for product evaluation
• Performed by customers/end users (preferably)
• Verifies functionality and usability of the software
• Prior to software being released to live operation

11. Testing Techniques
• Basically two types –
1) White-box testing
2) Black-box testing

12. BBT vs. WBT
Black box testing:
• View components as opaque
• Based on requirements and functionality
• Without any knowledge of internal design, code or
language.
• AKA : Functional testing, behavioral testing

13. BBT vs. WBT
White box testing:
• View components as transparent
• Based on knowledge of the internal logic
• Done by programmers (usually)
• AKA: Structural testing, Glass-box testing, Clear-box
testing

14. Manual Testing vs. Automated Testing
Manual Testing:
• Oldest and most rigorous type of software testing
• Requires a tester to perform manual test operations
– Hard to repeat
– Not always reliable
– Costly (time consuming, labor intensive)

15. Manual Testing vs. Automated Testing
Automated Testing:
• Testing employing software tools
• Execute tests without manual intervention
– Fast
– Repeatable
– Reliable
– Reusable
– Programmable
– Saves time

16. Alpha Testing vs. Beta Testing
Alpha testing:
• Alpha testing is performed by potential users/customers or an
independent test team at the developer’s site.
• Conducted when code is mostly complete or contains most of
the functionality and prior to users being involved.
• Minor design changes may still be made as a result of such
testing. Sometimes a select group of users are involved.

17. Alpha Testing vs. Beta Testing
Beta testing:
• Beta testing is done at the customer’s site by the customer in the open
environment.
• Testing when development and testing are essentially completed and final
bugs and problems need to be found before final release.
• Typically done by end-users or others, not by programmers. Betas are
often widely distributed or even distributed to the public at large in hopes
that they will buy the final product when it is released.
• Selected users receive the system first and report problems back to the
developer.
• Beta testing is a type of acceptance testing involving a software product
to be marketed for use by many users.

18. What is a Bug?
• A bug is a defect/fault in a system which causes the
system to perform in an unintended or unanticipated
manner.

19. Bug: Severity vs. Priority
• Severity:
– Severity of a bug is the impact of a bug on the system’s
ability to function.
– Severity is a measure of how bad(impact).
• Priority:
– Priority of a bug is how important it is for a bug to be fixed.
– Priority is a measure of when (time).

20. What is a Test Plan?
• A test plan is a document that describes the objectives, scope,
approach, resources, schedule and focus of software testing
activities.
• A test plan gives detailed testing information regarding an
upcoming testing effort. In other words, a test plan is a
systematic approach to testing a system and typically
contains a detailed understanding of what the eventual
workflow will be.
• Organizations may follow standard test plan guidelines (e.g.
IEEE, CMM ) or they can have their own customized test plan
outlines.

21. What is a Test Case?
• A test case is a document that describes an input,
action, or event, and its expected results, in order to
determine if a feature of an application is working
correctly.
• In other words, a test case is document specifying
inputs, predicted results and a set of execution
conditions for a test item.

22. Test Suite
• The collection of individual test cases that will be run
in a test sequence until some stopping criteria are
satisfied is called a test suite.
• A collection of tests used to validate the behavior of
a product.

23. Test Log
• A chronological record of all relevant details about
the execution of a test.

24. Test Bed
• An execution environment configured for testing.
• A test bed may consists of specific hardware, OS,
network topology, configuration of AUT, other
application or system software.

25. Test Script
• A test script is commonly used to refer to the
instructions for a particular test that will be
carried out by an automated testing tool.

26. Test Deliverables
 What are the typical Test Deliverables? / What is to
be delivered as part of the test plan?
– Test plan
– Test case
– Test scripts
– Test matrix
– Defect log
– Test Summary Report

27. What is a successful Test Plan?
• A successful test plan is one that verifies that all
functions specified in the requirements are included
in the final system.
• Such a test plan provides 100% test coverage and is
accomplished with minimum cost to resources and
schedule.

28. Test Plan: How to develop?
• Test plan is developed based on the requirements
and functionality of the system.
– FRD (Functional Requirements Documents)
– SRS (Software Requirements Specification)
– Technical specifications
– User manuals (if there is any)
– Use Cases

29. Quality Assurance Beyond Testing
• Although software testing plays a central role in
software quality assurance and is the most
commonly performed QA activity, it is neither the
only viable nor the most effective QA technique
under all circumstances.
• There are many other QA activities beyond testing –
– Review (Inspection/FTR/Walkthrough)
– Formal verification
– Defect prevention
– Fault tolerance