Topics Attribute of good software Introduction of non functional testing Functional & non functional requirement Non-Functional testing techniques Introduction of performance testing

Attribute of good software

Introduction of non functional testing

Functional & non functional requirement

Non-Functional testing techniques

Introduction of performance testing

Attribute of good software The ISO 9126 international standard lists six quality attributes for systems. these are then broken down further sub-attributes. Functionality Interoperability Security Reliability-Availability – Recoverability (Dependability) Maintainability Usability Efficiency - Performance Portability - Installability

The ISO 9126 international standard lists six quality attributes for systems. these are then broken down further sub-attributes.

Functionality

Interoperability

Security

Reliability-Availability – Recoverability (Dependability)

Maintainability

Usability

Efficiency - Performance

Portability - Installability

Attribute of good software cont…

Introduction of Non Functional Testing Functional Testing Functional testing demonstrates WHAT the product does Non Functional Testing Non-functional testing demonstrates HOW WELL the product behaves *** People’s desire to purchase/use a product may be more affected by non-functional qualities than they are by functionality

Functional Testing

Functional testing demonstrates WHAT the product does

Non Functional Testing

Non-functional testing demonstrates HOW WELL the product behaves

*** People’s desire to purchase/use a product may be more affected by non-functional qualities than they are by functionality

Functional Requirement A requirement that specifies a function that a system or system component must be able to perform. Non Functional Requirement Building systems with only functional requirements is relatively easy, but ultimately pointless. No-one wants a slow, unreliable system that costs a fortune to modify. ‘ Non-functional requirements’ is that they specify all the remaining requirements not covered by the functional requirements. Nonfunctional requirements specify the system’s ‘quality characteristics’ or ‘quality attributes’. Functional & non functional requirements

Functional Requirement

A requirement that specifies a function that a system or system component must be able to perform.

Non Functional Requirement

Building systems with only functional requirements is relatively easy, but ultimately pointless. No-one wants a slow, unreliable system that costs a fortune to modify.

‘ Non-functional requirements’ is that they specify all the remaining requirements not covered by the functional requirements.

Nonfunctional requirements specify the system’s ‘quality characteristics’ or ‘quality attributes’.

Functional & non functional requirements cont.. Qualitative requirements Don’t ask for a system that is: – “ Easy to use” – “ Faster than a cheetah on speed” – “ As reliable as possible” The difficulty with purely qualitative requirements is that they are neither measurable nor testable

Qualitative requirements

Don’t ask for a system that is:

– “ Easy to use”

– “ Faster than a cheetah on speed”

– “ As reliable as possible”

The difficulty with purely qualitative requirements is that they are neither measurable nor testable

Functional & non functional requirements cont.. Qualitative requirements Ideally all nonfunctional requirements need to be specified in a way that makes them understandable by all of the stakeholders. One way of achieving this is by using what are known as ‘SMART’ requirements. S-M-A-R-T

Qualitative requirements

Ideally all nonfunctional requirements need to be specified in a way that makes them understandable by all of the stakeholders.

One way of achieving this is by using what are known as ‘SMART’ requirements.

S-M-A-R-T

Functional & non functional requirements cont.. S pecific We want requirements that are both precise and thorough M easurable while ‘measurable’ should encourage the use of quantitative testable requirements that have been assigned an actual value to both achieve and test against. A cceptable ‘ Acceptable’ requirements should satisfy the needs of the customers and users. R ealisable The main point of ‘realisable’ requirements is that they should be realistic and achievable given the constraints of the project T raceable while ‘traceable’ requirements should allow traceability both back to the user requirements and forwards to the subsequent implementation and tests.

S pecific

We want requirements that are both precise and thorough

M easurable

while ‘measurable’ should encourage the use of quantitative testable requirements that have been assigned an actual value to both achieve and test against.

A cceptable

‘ Acceptable’ requirements should satisfy the needs of the customers and users.

R ealisable

The main point of ‘realisable’ requirements is that they should be realistic and achievable given the constraints of the project

T raceable

while ‘traceable’ requirements should allow traceability both back to the user requirements and forwards to the subsequent implementation and tests.

Non-Functional testing techniques Memory Management Reliability Usability Maintainability Configuration Portability Recovery Disaster Recovery

Memory Management

Reliability

Usability

Maintainability

Configuration

Portability

Recovery

Disaster Recovery

Non-Functional testing techniques cont… Performance Stress Security Procedure Interoperability Compatibility Installability Conversion

Performance

Stress

Security

Procedure

Interoperability

Compatibility

Installability

Conversion

Reliability IEEE 610.12:1990 definition - Ability of a system or component to perform its required functions under stated conditions for a specified period of time Ability of software to perform without failure under specified condition for either a specified period of time or for a specified number of transaction The major difference between reliability testing and functional (defect) testing is that with reliability testing the test inputs aim to mimic real life (i.e. the operational profile) rather than being aimed at exercising specific functions or finding specific types of fault. Metrics required to measure the reliability - for continuously-running systems metric such as the mean time to failure (MTTF) and probability of failure on demand (POFOD) are generally used.

IEEE 610.12:1990 definition - Ability of a system or component to perform its required functions under stated conditions for a specified period of time

Ability of software to perform without failure under specified condition for either a specified period of time or for a specified number of transaction

The major difference between reliability testing and functional (defect) testing is that with reliability testing the test inputs aim to mimic real life (i.e. the operational profile) rather than being aimed at exercising specific functions or finding specific types of fault.

Metrics required to measure the reliability - for continuously-running systems metric such as the mean time to failure (MTTF) and probability of failure on demand (POFOD) are generally used.

Maintainability IEEE 610.12:1990 – the ease which a software systems or component can be modified to correct faults, improve performance, or other attributes, or adapt to a changed environment fault removal is typically only a small part of the overall maintenance burden. The majority of maintenance tasks are concerned with implementing new requirements and adapting to new environments, such as a new operating system. Measure of software maintainability are: MTTR (Mean time to repair) and MTTM (Mean time to modify)

IEEE 610.12:1990 – the ease which a software systems or component can be modified to correct faults, improve performance, or other attributes, or adapt to a changed environment

fault removal is typically only a small part of the overall maintenance burden. The majority of maintenance tasks are concerned with implementing new requirements and adapting to new environments, such as a new operating system.

Measure of software maintainability are:

MTTR (Mean time to repair) and

MTTM (Mean time to modify)

Maintainability cont.. Indirect measure of software maintainability Software design Language and operative system Design and coding standards Modularity Documentation Structure coding techniques Code reliability Test case suite and regression testing

Indirect measure of software maintainability

Software design

Language and operative system

Design and coding standards

Modularity

Documentation

Structure coding techniques

Code reliability

Test case suite and regression testing

Availability IEEE 610.12:1990 The degree to which a system or component is operational and accessible when required for use The probability that the system will be functioning correctly at any given time Example Service Help desk must be available 24/7 Phone system must: be available at least 98% of the time Unavailable no more than 2% of the time There is a close relationship between reliability, availability and maintainability. A (mythical) perfectly reliable system never fails and so is always available, and also requires no maintenance to fix faults

IEEE 610.12:1990 The degree to which a system or component is operational and accessible when required for use

The probability that the system will be functioning correctly at any given time

Example

Service Help desk must be available 24/7

Phone system must:

be available at least 98% of the time

Unavailable no more than 2% of the time

There is a close relationship between reliability, availability and maintainability. A (mythical) perfectly reliable system never fails and so is always available, and also requires no maintenance to fix faults

Recovery IEEE 610.12:1990-The restoration of system, program, database, or other system resource to prior state following a failure or externally caused disaster

IEEE 610.12:1990-The restoration of system, program, database, or other system resource to prior state following a failure or externally caused disaster

Usability ISO/IEC 9126-1 (2000) - Usability: the capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions. Comparing usability testing with… Functional Testing concentrates on whether the software performs the task correctly, it does not test how well the user is able to use software to perform the tasks.

ISO/IEC 9126-1 (2000) - Usability: the capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions.

Comparing usability testing with…

Functional Testing concentrates on whether the software performs the task correctly, it does not test how well the user is able to use software to perform the tasks.

Configuration Configuration testing is testing the product under a different number of hardware or software configurations. Hardware and software can sometimes have an unlimited number of configurable items and this would be too much to test, so it is advisable that the product be tested with, at least, the minimum, the maximum configuration and a few different configurations in between.

Configuration testing is testing the product under a different number of hardware or software configurations.

Hardware and software can sometimes have an unlimited number of configurable items and this would be too much to test, so it is advisable that the product be tested with, at least, the minimum, the maximum configuration and a few different configurations in between.

Security BS7925-1:1998 Testing whether the system meets its specified security objectives So what are the security objectives ? Confidentiality e.g. Information leakage Integrity e.g. using spoofing to fool software to accept input it should reject, or by corrupting data held in the system Availability e.g. by denying access to authorised users There are various security mechanisms that are used to implement these areas, such as: Authentication Authorisation Auditing

BS7925-1:1998 Testing whether the system meets its specified security objectives

So what are the security objectives ?

Confidentiality e.g. Information leakage

Integrity e.g. using spoofing to fool software to accept input it should reject, or by corrupting data held in the system

Availability e.g. by denying access to authorised users

There are various security mechanisms that are used to implement these areas, such as:

Authentication

Authorisation

Auditing

Performance Performance Requirement - IEEE 610.12:1990 A requirement that imposes conditions on a functional requirement; for example a requirements that specifies the speed, accuracy, or memory usage with which a given function must be performed Performance -IEEE 610.12:1990 The degree to which a system or component accomplishes its designated functions within given constraints, such as speed, accuracy, or memory usage. Performance Testing – IEEE 610.12:1990, BS-7925-1:1998 Testing conducted to evaluate the compliance of a system or component with specified performance requirements.

Performance Requirement - IEEE 610.12:1990

A requirement that imposes conditions on a functional requirement; for example a requirements that specifies the speed, accuracy, or memory usage with which a given function must be performed

Performance -IEEE 610.12:1990

The degree to which a system or component accomplishes its designated functions within given constraints, such as speed, accuracy, or memory usage.

Performance Testing – IEEE 610.12:1990, BS-7925-1:1998

Testing conducted to evaluate the compliance of a system or component with specified performance requirements.

Performance Testing Performance testing helps to determine whether software will meet the performance requirements such as Response time Concurrency Resource consumption and Volumetrics Performance testing touches on many areas including: Load testing Stress testing Volume testing Soak testing Scalability

Performance testing helps to determine whether software will meet the performance requirements such as

Response time

Concurrency

Resource consumption and

Volumetrics

Performance testing touches on many areas including:

Load testing

Stress testing

Volume testing

Soak testing

Scalability

Performance Testing cont.. Request for service System 1) Done correctly 2) Done incorrectly 3) Can not do Time (Response Time) Rate (Throughput ) Resource (Utilisation) Reliability Availability

Thank You !!!

Thank You !!!