Testing

Software testing 3. Test types Dr. Balla Katalin

Contents Testing or inspection? Test types

Testing The process that helps understanding the correctness, completeness and quality of the software Testing cannot entirely prove the correctness of the software. Lack of errors can be demonstrated only by using formal methods.

Testing or review / inspection / auditing? Review, inspection, audit: are in fact a way of testing. However… In everyday discussions we usually use „testing” for code, and „review / inspection / audit” for something else than code (eg. specification, plans, user documentation etc.)

Inspection, review, audit Inspection: assessment of any work product, executed by trained professionals, according to well-defined rules. Scope: find defects. Basic idea: we do not find our own bugs… Inspection: can be applied in any phase of development to: specification, plans, code, test cases… Inspection: often used for checking the code

Inspection Might reduce testing cost to 20-30 % of development cost. Good inspection might result in omitting unit test and integration test. System test will be enough. Eg: inspection of system design compared to specification, code inspection compared to system design etc.

Test types Many groupings are possible redundancy, overlap Eg: grouping by: Tested item Place in lifecycle Person doing the test Usage of test results Techniques used…

Test types Tested item Place in lifecycle Person doing the test Usage of test results Techniques used…

Test types By tested item Component / unit -test Integration test System test

Test types by tested item Unit test / component test Units / components are tested against specification. One unit: one or more procedures, functions Is a testing method to check the correctness of a part of the code. Basic idea: develop a test case for all non-trivial functions, methods of a module, having mutually separate test cases (one test case test exactly one item, does not interfere with others)

Integration test Units are integrated and the interfaces are tested against their specification It is usually executed after unit testing, before system testing Uses modules already unit-tested It has the scope to check the requirements regarding the functionality of larg(er) items. Interfaces between modules are checked. Testing is done by simulating cooperation between modules. Basic idea: we always use units that are tested already Test types by tested item

System test The entire system is tested against its specification Input to test: (sub)system(s) that have successfully pass their test System test must be understood by the user. In many cases system test and acceptance test are connected. Test types by tested item

Composing the test Test the units / components Integrate components into subsystems- test subsystems Continue integration, until the entire system can be tested Test types by tested item

Test types by tested item Are connected in fact to life cycle phases Requirement specification High level design Unit test Integration test System test Detailed design Coding

Test types by tested item Regression test: we prove that all that was functioning in previous build is still functioning Experience: each fifth modification injects a new bug Progression testing : we assume that integration test has been successfully performed, and we test only the new functions

Test types Tested item Place in lifecycle Person doing the test Usage of test results Techniques used

Test types by place in the lifecycle Show the „readiness” of the software, the degree of progress and the part still to be done in the software development Basic cases: Pre-alfa status („half-way ready”, not yet complete) Alfa status (new elements have been added) Beta status (we are actively looking for bugs) Stable status (most severe bugs have been corrected) Release-candidate (Microsoft: „RTM” – release to manufacturing) (ready to be released) Release / gold status (released) … Further intermediate states can be identified …

Test types by place in the lifecycle Pre-alfa test: developer test of ceratin functions Alfa test: internal system test by developer Beta test: restricted number of users are testing the system (Gamma test, gamma version: funny, ironical name for a system full of bugs) Gamma, delta, omega tests: tests of more and more developed releases

Test types by person doing the test Designer test Developer test User test Test executed jointly, by more stakeholders Joint reviews

Test types by usage of the test results A decision (official) is taken based on the test results Testing is executed to… Qualify the system – qualification test Hand over / accept the system – acceptance test Show that the system is operating as planned - operational test Install the system – installation test Jointly review the system – joint review Verify the system – verification / assessment …

Test types by usage of the test results Qualification testing: System qualification testing is performed to demonstrate to the acquirer that system requirements have been met. It covers the system requirements in the system/subsystem specifications and in associated interface requirements specifications.

Test types by usage of the test results Acceptance test acceptance testing is testing performed on a system . In some engineering subdisciplines , it is known as functional testing, black-box testing , release acceptance , QA testing, application testing, confidence testing, final testing, validation testing, usability testing , or factory acceptance testing. In most environments, acceptance testing by the system provider is distinguished from acceptance testing by the customer (the user or client) prior to accepting transfer of ownership. In such environments, acceptance testing performed by the customer is known as beta testing , user acceptance testing , end user testing, site (acceptance) testing, or field (acceptance) testing.

Test types by usage of the test results Operational test An operational test is performed before handing over each version of a software. The software is put into real operation only if the operational test was successful. This test ensures that the software is entering into operation, is operating and stops operating according to the plan.

Test types by usage of the test results Installation test Test performed outside the development environment In some cases installation requires setting / modifying of some parameters in the running environment. In such cases it is useful to have a back-up procedure.

Test types by usage of the test results Joint review This process provides the framework for interactions between the reviewer and the reviewee. They may as well be the acquirer and the supplier respectively. At a joint review, the reviewee presents the status and products of a life cycle activity of a project to the reviewer for comment (or approval). The reviews are at both management and technical levels. (ISO 12207)

Test types by usage of the test results Audit This process provides the framework for formal, contractually established audits of a supplier's products or services. At an audit, the auditor assesses the auditee's products and activities with emphasis on compliance to requirements and plans. An audit may well be conducted by the acquirer on the supplier. (ISO 12207)

Test types by techniques used At different levels different testing techniques are used. At all levels the desired result can be achieved by combining multiple techniques.

Test types by techniques used Static testing Dynamic testing Structural testing Functional testing These will be discussed in detail in the following lectures

Test types by techniques used Some consider to be techniques: Validation Validation determines whether the final, as-built system fulfills its specific intended use. The extent of validation depends upon the project's criticality. Validation does not replace other evaluations, but supplements them. Verification This process provides the evaluations related to verification of a product or service of a given activity. Verification determines whether the requirements for a system are complete and correct and that the outputs of an activity fulfill the requirements or conditions imposed on them in the previous activities. The process covers verification of process, requirements, design, code, integration, and documentation. Verification does not alleviate the evaluations assigned to a process; on the contrary, it supplements them. Verification or validation may be conducted by the acquirer, the supplier, or an independent party. When they are executed by an organization independent of the supplier or developer, they are called independent verification and validation (IV&V) Process. ISO 12207

Test types by other criteria There are other g r oupings also. New groups can be formed by combining elements of the previously presented ones. By (place in the life cycle + participants in the tes t ing + test results) By algorithm used in testing …

Test types by other criteria „Manual” testing Automated testing Testing done based on scripts developed by the user or tester Testing using automated testing tools

Test types by other criteria Load testing Load testing is a generic term covering Performance Testing and Stress Testing.

Test types by other criteria Performance testing Performed, from one perspective, to determine how fast some aspect of a system performs under a particular workload. It can also serve to validate and verify other quality attributes of the system, such as scalability, reliability and resource usage.

Test types by other criteria Stress testing Testing conducted to evaluate a system or component at or beyond the limits of its specified requirements to determine the load under which it fails and how. A graceful degradation under load leading to non-catastrophic failure is the desired result. Often Stress Testing is performed using the same process as Performance Testing but employing a very high level of simulated load

Test types by other criteria Model based testing Test cases are generated automatically from a model describing the behaviour of the system Eg: UML - Activity diagram

Test types by other criteria Usability testing Usability testing focuses on measuring a human-made product's capacity to meet its intended purpose. Examples of products that commonly benefit from usability testing are web sites or web applications, computer interfaces , documents, or devices. Usability testing measures the usability , or ease of use, of a specific object or set of objects, whereas general human-computer interaction studies attempt to formulate universal principles.

Test types by other criteria Usability testing Usability testing generally involves measuring how well test subjects respond in four areas: time, accuracy, recall, and emotional response. The results of the first test can be treated as a baseline or control measurement; all subsequent tests can then be compared to the baseline to indicate improvement. Time on Task -- How long does it take people to complete basic tasks? (For example, find something to buy, create a new account, and order the item.) Accuracy -- How many mistakes did people make? (And were they fatal or recoverable with the right information?) Recall -- How much does the person remember afterwards or after periods of non-use? Emotional Response -- How does the person feel about the tasks completed? (Confident? Stressed? Would the user recommend this system to a friend?)

Test types by other criteria Stability testing Has the scope to unders t and whether a system will be able to function, what is the probability that it fails

Test types by other criteria Security Testing: Process to determine that an Information System protects data and maintains functionality as intended. The six basic security concepts that need to be covered by security testing are: confidentiality, integrity, authentication, authorisation, availability and non-repudiation.

Test types by other criteria Security Testing – basic concepts Confidentiality A security measure which protects against the disclosure of information to parties other than the intended recipient that is by no means the only way of ensuring confidentiality. Integrity A measure intended to allow the receiver to determine that the information which it receives has not been altered in transit or by other than the originator of the information. Integrity schemes often use some of the same underlying technologies as confidentiality schemes, but they usually involve adding additional information to a communication to form the basis of an algorithmic check rather than the encoding all of the communication. Authentication A measure designed to establish the validity of a transmission, message, or originator. Allows a receiver to have confidence that information it receives originated from a specific known source.

Test types by other criteria Security Testing – basic concepts Authorization The process of determining that a requester is allowed to receive a service or perform an operation. Access control is an example of authorization. Availability Assuring information and communications services will be ready for use when expected. Information must be kept available to authorized persons when they need it. Also authority to operate Non-repudiation A measure intended to prevent the later denial that an action happened, or a communication that took place etc. In communication terms this often involves the interchange of authentication information combined with some form of provable time stamp

Test types by other criteria Conformance testing / type testing Executed to see if a system is conform or not to the requirements of a standard or law. Many standards contain conformance criteria. Conformance testing is often executed by third parties, eg. the standardization body.

What we talked about… Testing or inspection? Test types

Testing

More Related Content

What's hot

Viewers also liked

Similar to Testing

More from nazeer pasha

Recently uploaded

Testing