The document discusses various types of non-functional testing including performance, reliability, maintainability, availability, recovery, usability, configuration, and security testing. It provides definitions and examples of how to test each type of non-functional requirement. Performance testing aims to evaluate how well a system performs under different loads, and involves measuring response times, throughput, and resource utilization. Non-functional requirements are as important as functional requirements in building quality software.

1. Topics Attribute of good software Introduction of non functional testing Functional & non functional requirement Non-Functional testing techniques Introduction of performance testing

2. Attribute of good software The ISO 9126 international standard lists six quality attributes for systems. these are then broken down further sub-attributes. Functionality Interoperability Security Reliability-Availability – Recoverability (Dependability) Maintainability Usability Efficiency - Performance Portability - Installability

3. Attribute of good software cont…

4. Introduction of Non Functional Testing Functional Testing Functional testing demonstrates WHAT the product does Non Functional Testing Non-functional testing demonstrates HOW WELL the product behaves *** People’s desire to purchase/use a product may be more affected by non-functional qualities than they are by functionality

5. Functional Requirement A requirement that specifies a function that a system or system component must be able to perform. Non Functional Requirement Building systems with only functional requirements is relatively easy, but ultimately pointless. No-one wants a slow, unreliable system that costs a fortune to modify. ‘ Non-functional requirements’ is that they specify all the remaining requirements not covered by the functional requirements. Nonfunctional requirements specify the system’s ‘quality characteristics’ or ‘quality attributes’. Functional & non functional requirements

6. Functional & non functional requirements cont.. Qualitative requirements Don’t ask for a system that is: – “ Easy to use” – “ Faster than a cheetah on speed” – “ As reliable as possible” The difficulty with purely qualitative requirements is that they are neither measurable nor testable

7. Functional & non functional requirements cont.. Qualitative requirements Ideally all nonfunctional requirements need to be specified in a way that makes them understandable by all of the stakeholders. One way of achieving this is by using what are known as ‘SMART’ requirements. S-M-A-R-T

8. Functional & non functional requirements cont.. S pecific We want requirements that are both precise and thorough M easurable while ‘measurable’ should encourage the use of quantitative testable requirements that have been assigned an actual value to both achieve and test against. A cceptable ‘ Acceptable’ requirements should satisfy the needs of the customers and users. R ealisable The main point of ‘realisable’ requirements is that they should be realistic and achievable given the constraints of the project T raceable while ‘traceable’ requirements should allow traceability both back to the user requirements and forwards to the subsequent implementation and tests.

9. Non-Functional testing techniques Memory Management Reliability Usability Maintainability Configuration Portability Recovery Disaster Recovery

10. Non-Functional testing techniques cont… Performance Stress Security Procedure Interoperability Compatibility Installability Conversion

11. Reliability IEEE 610.12:1990 definition - Ability of a system or component to perform its required functions under stated conditions for a specified period of time Ability of software to perform without failure under specified condition for either a specified period of time or for a specified number of transaction The major difference between reliability testing and functional (defect) testing is that with reliability testing the test inputs aim to mimic real life (i.e. the operational profile) rather than being aimed at exercising specific functions or finding specific types of fault. Metrics required to measure the reliability - for continuously-running systems metric such as the mean time to failure (MTTF) and probability of failure on demand (POFOD) are generally used.

12. Maintainability IEEE 610.12:1990 – the ease which a software systems or component can be modified to correct faults, improve performance, or other attributes, or adapt to a changed environment fault removal is typically only a small part of the overall maintenance burden. The majority of maintenance tasks are concerned with implementing new requirements and adapting to new environments, such as a new operating system. Measure of software maintainability are: MTTR (Mean time to repair) and MTTM (Mean time to modify)

13. Maintainability cont.. Indirect measure of software maintainability Software design Language and operative system Design and coding standards Modularity Documentation Structure coding techniques Code reliability Test case suite and regression testing

14. Availability IEEE 610.12:1990 The degree to which a system or component is operational and accessible when required for use The probability that the system will be functioning correctly at any given time Example Service Help desk must be available 24/7 Phone system must: be available at least 98% of the time Unavailable no more than 2% of the time There is a close relationship between reliability, availability and maintainability. A (mythical) perfectly reliable system never fails and so is always available, and also requires no maintenance to fix faults

15. Recovery IEEE 610.12:1990-The restoration of system, program, database, or other system resource to prior state following a failure or externally caused disaster

16. Usability ISO/IEC 9126-1 (2000) - Usability: the capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions. Comparing usability testing with… Functional Testing concentrates on whether the software performs the task correctly, it does not test how well the user is able to use software to perform the tasks.

17. Configuration Configuration testing is testing the product under a different number of hardware or software configurations. Hardware and software can sometimes have an unlimited number of configurable items and this would be too much to test, so it is advisable that the product be tested with, at least, the minimum, the maximum configuration and a few different configurations in between.

18. Security BS7925-1:1998 Testing whether the system meets its specified security objectives So what are the security objectives ? Confidentiality e.g. Information leakage Integrity e.g. using spoofing to fool software to accept input it should reject, or by corrupting data held in the system Availability e.g. by denying access to authorised users There are various security mechanisms that are used to implement these areas, such as: Authentication Authorisation Auditing

19. Performance Performance Requirement - IEEE 610.12:1990 A requirement that imposes conditions on a functional requirement; for example a requirements that specifies the speed, accuracy, or memory usage with which a given function must be performed Performance -IEEE 610.12:1990 The degree to which a system or component accomplishes its designated functions within given constraints, such as speed, accuracy, or memory usage. Performance Testing – IEEE 610.12:1990, BS-7925-1:1998 Testing conducted to evaluate the compliance of a system or component with specified performance requirements.

20. Performance Testing Performance testing helps to determine whether software will meet the performance requirements such as Response time Concurrency Resource consumption and Volumetrics Performance testing touches on many areas including: Load testing Stress testing Volume testing Soak testing Scalability

21. Performance Testing cont.. Request for service System 1) Done correctly 2) Done incorrectly 3) Can not do Time (Response Time) Rate (Throughput ) Resource (Utilisation) Reliability Availability

22. Thank You !!!