Non Functional Testing


Published on

Published in: Technology
1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Non Functional Testing

  1. 1. Topics <ul><li>Attribute of good software </li></ul><ul><li>Introduction of non functional testing </li></ul><ul><li>Functional & non functional requirement </li></ul><ul><li>Non-Functional testing techniques </li></ul><ul><li>Introduction of performance testing </li></ul>
  2. 2. Attribute of good software <ul><li>The ISO 9126 international standard lists six quality attributes for systems. these are then broken down further sub-attributes. </li></ul><ul><li>Functionality </li></ul><ul><ul><li>Interoperability </li></ul></ul><ul><ul><li>Security </li></ul></ul><ul><li>Reliability-Availability – Recoverability (Dependability) </li></ul><ul><li>Maintainability </li></ul><ul><li>Usability </li></ul><ul><li>Efficiency - Performance </li></ul><ul><li>Portability - Installability </li></ul>
  3. 3. Attribute of good software cont…
  4. 4. Introduction of Non Functional Testing <ul><li>Functional Testing </li></ul><ul><li>Functional testing demonstrates WHAT the product does </li></ul><ul><li>Non Functional Testing </li></ul><ul><li>Non-functional testing demonstrates HOW WELL the product behaves </li></ul><ul><li>*** People’s desire to purchase/use a product may be more affected by non-functional qualities than they are by functionality </li></ul>
  5. 5. <ul><li>Functional Requirement </li></ul><ul><li>A requirement that specifies a function that a system or system component must be able to perform. </li></ul><ul><li>Non Functional Requirement </li></ul><ul><li>Building systems with only functional requirements is relatively easy, but ultimately pointless. No-one wants a slow, unreliable system that costs a fortune to modify. </li></ul><ul><li>‘ Non-functional requirements’ is that they specify all the remaining requirements not covered by the functional requirements. </li></ul><ul><li>Nonfunctional requirements specify the system’s ‘quality characteristics’ or ‘quality attributes’. </li></ul>Functional & non functional requirements
  6. 6. Functional & non functional requirements cont.. <ul><li>Qualitative requirements </li></ul><ul><li>Don’t ask for a system that is: </li></ul><ul><ul><li>– “ Easy to use” </li></ul></ul><ul><ul><li>– “ Faster than a cheetah on speed” </li></ul></ul><ul><ul><li>– “ As reliable as possible” </li></ul></ul><ul><li>The difficulty with purely qualitative requirements is that they are neither measurable nor testable </li></ul>
  7. 7. Functional & non functional requirements cont.. <ul><li>Qualitative requirements </li></ul><ul><li>Ideally all nonfunctional requirements need to be specified in a way that makes them understandable by all of the stakeholders. </li></ul><ul><li>One way of achieving this is by using what are known as ‘SMART’ requirements. </li></ul><ul><li>S-M-A-R-T </li></ul>
  8. 8. Functional & non functional requirements cont.. <ul><ul><li>S pecific </li></ul></ul><ul><ul><li>We want requirements that are both precise and thorough </li></ul></ul><ul><ul><li>M easurable </li></ul></ul><ul><ul><li>while ‘measurable’ should encourage the use of quantitative testable requirements that have been assigned an actual value to both achieve and test against. </li></ul></ul><ul><ul><li>A cceptable </li></ul></ul><ul><ul><li>‘ Acceptable’ requirements should satisfy the needs of the customers and users. </li></ul></ul><ul><ul><li>R ealisable </li></ul></ul><ul><ul><li>The main point of ‘realisable’ requirements is that they should be realistic and achievable given the constraints of the project </li></ul></ul><ul><ul><li>T raceable </li></ul></ul><ul><li> while ‘traceable’ requirements should allow traceability both back to the user requirements and forwards to the subsequent implementation and tests. </li></ul>
  9. 9. Non-Functional testing techniques <ul><li>Memory Management </li></ul><ul><li>Reliability </li></ul><ul><li>Usability </li></ul><ul><li>Maintainability </li></ul><ul><li>Configuration </li></ul><ul><li>Portability </li></ul><ul><li>Recovery </li></ul><ul><li>Disaster Recovery </li></ul>
  10. 10. Non-Functional testing techniques cont… <ul><li>Performance </li></ul><ul><li>Stress </li></ul><ul><li>Security </li></ul><ul><li>Procedure </li></ul><ul><li>Interoperability </li></ul><ul><li>Compatibility </li></ul><ul><li>Installability </li></ul><ul><li>Conversion </li></ul>
  11. 11. Reliability <ul><li>IEEE 610.12:1990 definition - Ability of a system or component to perform its required functions under stated conditions for a specified period of time </li></ul><ul><li>Ability of software to perform without failure under specified condition for either a specified period of time or for a specified number of transaction </li></ul><ul><li>The major difference between reliability testing and functional (defect) testing is that with reliability testing the test inputs aim to mimic real life (i.e. the operational profile) rather than being aimed at exercising specific functions or finding specific types of fault. </li></ul><ul><li>Metrics required to measure the reliability - for continuously-running systems metric such as the mean time to failure (MTTF) and probability of failure on demand (POFOD) are generally used. </li></ul>
  12. 12. Maintainability <ul><li>IEEE 610.12:1990 – the ease which a software systems or component can be modified to correct faults, improve performance, or other attributes, or adapt to a changed environment </li></ul><ul><li>fault removal is typically only a small part of the overall maintenance burden. The majority of maintenance tasks are concerned with implementing new requirements and adapting to new environments, such as a new operating system. </li></ul><ul><li>Measure of software maintainability are: </li></ul><ul><ul><li>MTTR (Mean time to repair) and </li></ul></ul><ul><ul><li>MTTM (Mean time to modify) </li></ul></ul>
  13. 13. Maintainability cont.. <ul><li>Indirect measure of software maintainability </li></ul><ul><li>Software design </li></ul><ul><li>Language and operative system </li></ul><ul><li>Design and coding standards </li></ul><ul><ul><li>Modularity </li></ul></ul><ul><ul><li>Documentation </li></ul></ul><ul><ul><li>Structure coding techniques </li></ul></ul><ul><ul><li>Code reliability </li></ul></ul><ul><li>Test case suite and regression testing </li></ul>
  14. 14. Availability <ul><li>IEEE 610.12:1990 The degree to which a system or component is operational and accessible when required for use </li></ul><ul><li>The probability that the system will be functioning correctly at any given time </li></ul><ul><li>Example </li></ul><ul><ul><li>Service Help desk must be available 24/7 </li></ul></ul><ul><ul><li>Phone system must: </li></ul></ul><ul><ul><ul><li>be available at least 98% of the time </li></ul></ul></ul><ul><ul><ul><li>Unavailable no more than 2% of the time </li></ul></ul></ul><ul><li>There is a close relationship between reliability, availability and maintainability. A (mythical) perfectly reliable system never fails and so is always available, and also requires no maintenance to fix faults </li></ul>
  15. 15. Recovery <ul><li>IEEE 610.12:1990-The restoration of system, program, database, or other system resource to prior state following a failure or externally caused disaster </li></ul>
  16. 16. Usability <ul><li>ISO/IEC 9126-1 (2000) - Usability: the capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions. </li></ul><ul><li>Comparing usability testing with… </li></ul><ul><li>Functional Testing concentrates on whether the software performs the task correctly, it does not test how well the user is able to use software to perform the tasks. </li></ul>
  17. 17. Configuration <ul><li>Configuration testing is testing the product under a different number of hardware or software configurations. </li></ul><ul><li>Hardware and software can sometimes have an unlimited number of configurable items and this would be too much to test, so it is advisable that the product be tested with, at least, the minimum, the maximum configuration and a few different configurations in between. </li></ul>
  18. 18. Security <ul><li>BS7925-1:1998 Testing whether the system meets its specified security objectives </li></ul><ul><li>So what are the security objectives ? </li></ul><ul><ul><ul><li>Confidentiality e.g. Information leakage </li></ul></ul></ul><ul><ul><ul><li>Integrity e.g. using spoofing to fool software to accept input it should reject, or by corrupting data held in the system </li></ul></ul></ul><ul><ul><ul><li>Availability e.g. by denying access to authorised users </li></ul></ul></ul><ul><li>There are various security mechanisms that are used to implement these areas, such as: </li></ul><ul><ul><ul><li>Authentication </li></ul></ul></ul><ul><ul><ul><li>Authorisation </li></ul></ul></ul><ul><ul><ul><li>Auditing </li></ul></ul></ul>
  19. 19. Performance <ul><li>Performance Requirement - IEEE 610.12:1990 </li></ul><ul><li>A requirement that imposes conditions on a functional requirement; for example a requirements that specifies the speed, accuracy, or memory usage with which a given function must be performed </li></ul><ul><li>Performance -IEEE 610.12:1990 </li></ul><ul><li>The degree to which a system or component accomplishes its designated functions within given constraints, such as speed, accuracy, or memory usage. </li></ul><ul><li>Performance Testing – IEEE 610.12:1990, BS-7925-1:1998 </li></ul><ul><li>Testing conducted to evaluate the compliance of a system or component with specified performance requirements. </li></ul>
  20. 20. Performance Testing <ul><li>Performance testing helps to determine whether software will meet the performance requirements such as </li></ul><ul><ul><li>Response time </li></ul></ul><ul><ul><li>Concurrency </li></ul></ul><ul><ul><li>Resource consumption and </li></ul></ul><ul><ul><li>Volumetrics </li></ul></ul><ul><li>Performance testing touches on many areas including: </li></ul><ul><ul><li>Load testing </li></ul></ul><ul><ul><li>Stress testing </li></ul></ul><ul><ul><li>Volume testing </li></ul></ul><ul><ul><li>Soak testing </li></ul></ul><ul><ul><li>Scalability </li></ul></ul>
  21. 21. Performance Testing cont.. Request for service System 1) Done correctly 2) Done incorrectly 3) Can not do Time (Response Time) Rate (Throughput ) Resource (Utilisation) Reliability Availability
  22. 22. <ul><li>Thank You !!! </li></ul>