Download It
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Download It

on

  • 652 views

 

Statistics

Views

Total Views
652
Views on SlideShare
652
Embed Views
0

Actions

Likes
0
Downloads
11
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Download It Presentation Transcript

  • 1. Software Testing and Validation COEN 345 Software Quality Assurance Overview Dr. Rachida Dssouli dssouli@ciise.concordia.ca http://www.ciise.concordia.ca COEN 345 1
  • 2. Definitions  Software  Software quality  Software quality assurance  Errors, faults, failures  Cause of errors COEN 345 2
  • 3. What is a software? IEEE definition, ISO 1997 sec. 3.11 and ISO/IEC 9000-3 SEC. 3.14  Software is:  Computer programs (Code)  Procedures  Documentation  Data necessary for operating the software system COEN 345 3
  • 4. Software quality  SQ- IEEE definition  1) The degree to which a system, component, or process meets specified requirements.  2) The degree to which a system, component, or process meets customer or user needs or expectations COEN 345 4
  • 5. Software Quality Pressman’s definition  SQ is defined as  Conformance to explicitly functional stated functional and performance requirements, explicitly documented development standards, and implicit characteristics that are expected of all professionally developed software. This definition suggests 3 requirements to be met by the developer: Specific functional requirements (output) The software quality standards mentioned in the contract Good Software Engineering Practices COEN 345 5
  • 6. Quality Problem Users Decision maker client Request software Quality Benefits Developers Intermediate Products Decision maker COEN 345 provider 6
  • 7. Quality Policy and Quality System Expressed and implicit Requirements Quality Policies Implements Product (quality) Quality system COEN 345 7
  • 8. Quality Software Assurance  IEEE definition  Quality assurance is 1. A planed and systematic pattern of all actions necessary to provide adequate confidence that an item or product conforms to established technical requirements. 2. A set of activities designed to evaluate the process by which the products are developed or manufactured. COEN 345 8
  • 9. Software Quality Assurance (cont.)  SQA – expanded definition  A systematic, planed set of actions necessary to provide adequate confidence that the software development process or the maintenance process of the software system product conforms to established functional technical requirements as well as with the managerial requirements of keeping the schedule and operating within budgetary confines. COEN 345 9
  • 10. Quality Control versus SQA  Quality Control (QC) is a set of activities carried out with the main objective of withholding products from shipment if they do not qualify.  Quality Assurance (QA) is meant to minimize the costs of quality by introducing a variety of activities throughout the development process and maintenance process in order to prevent the causes of errors, detect them, and correct them in the early stages of the development. As a result, quality assurance substantially reduces the rate of non-qualifying products. COEN 345 10
  • 11. The Uniqueness of SQA  High complexity  Invisibility of software  Nature of the product development  Production process  Need for SQA methodologies and tools COEN 345 11
  • 12. Why do we care about Quality?  Development cost  Correction cost& source  Specification : 6%  Req. & Specification 56%  Design : 5%  Design : 24%  Coding : 7%  Coding : 10%  V&V (Testing) : 15%  Other: 10%  Maintenance : 67% COEN 345 12
  • 13. Relative cost of error correction 1000 100 10 1 R qm ts D ev Tes t Sys Tes t C ode D esi gn COEN 345 O per at ion 13
  • 14. Distinction between software errors, faults and failures  Software Error: section of the code that are incorrect as a result of grammatical, logical or other mistakes made by a system analyst , a programmer, or another member of the software development team  Software Fault are software errors that causes the incorrect functioning of the software during a specific application.  Software Failure, software faults become software failures only when they are “activated, COEN 345 14
  • 15. Causes of software errors  Faulty requirements  Client- developer communication failures  Deliberate deviations from software requirements  Logical design errors  Coding errors  Non compliance with documentation and coding instructions  Shortcoming of the testing process  Procedure errors  Documentation errors COEN 345 15
  • 16. Cost of quality COEN 345 16
  • 17. Cost of Quality Cost of quality --> includes all costs incurred in the pursuit of quality or perform quality related work Quality cost includes: - prevention cost: - quality planning - formal technical reviews - testing equipment - training - appraisal cost: - in-process and inter-process inspection - equipment calibration and maintenance - testing - failure cost: internal failure cost: - rework, repair, and failure mode analysis external failure cost: - complaint resolution - product return and replacement - help line support 345 COEN 17 - warranty work
  • 18. Views of quality  Product view: – functionality, correctness,  Business view: performance, usability ... – fitness for purpose, – this is what we usually timeliness, customer mean by quality satisfaction  Process view: – does it sell? – efficiency, cost & schedule, – producing the right product failures (rework), reuse, at the right time for the right learning, work satisfaction, price? ... – this is usually the ultimate – this is what process basis of decision management is concerned about COEN 345 18
  • 19. Quality Concepts Software quality assurance is an umbrella activity that is applied throughout the software process SQA encompasses: (1) a quality management approach (2) effective software engineering technology (3) formal technical reviews (4) a multi-tiered testing strategy (5) document change control (6) software development standard and its control procedure (7) measurement and reporting mechanism Quality refers to measurable characteristics of a software. These items can be compared based on a given standard Two types of quality control: - Quality design -> the characteristics that designers specify for an item. includes: requirements, specifications, and the design of the system. - Quality of conformance -> the degree to which the design specification are followed. It focuses on implementation based on the design. COEN 345 19
  • 20. Software Quality Assurance (SQA) • SQA is a collection of activities during software development that focus on increasing the quality of the software being produced • SQA includes – Analysis, design, coding and testing methods and tools – Formal Technical reviews during software development – A multi-tiered testing strategy – Control of software documentation and the changes made to it – Procedures to ensure compliance with software development standards – Software measurement and reporting mechanisms • SQA is often conducted by an independent group in the organization Often this group has the final veto over the release of a software product COEN 345 20
  • 21. SQA Activities 1. Application of Technical Methods • Tools to aid in the production of a high quality specification i.e. specification checkers and verifiers • Tools to aid in the production of high-quality designs i.e. design browsers, checkers, cross-references, verifiers • Tools to analyze source code for quality 2. Formal Technical Reviews Group analysis of a specification or design to discover errors 3. Software Testing 4. Enforcement of standards • Specification and design standards • Implementation standards, e.g portability • Documentation standards • Testing standards COEN 345 21
  • 22. SQA Activities 5. Control of Change • Formal management of changes to the software and documentation • Changes require formal request to approving authority • Approving authority makes decision on which changes get implemented and when • Programmers are not permitted to make unapproved changes to the software • Opportunity to evaluate the impact and cost of changes before committing resources • Evaluate effect of proposed changes on software quality 6. Measurement • Ongoing assessment of software quality • Track quality changes as system evolves • Warn management if software quality appears to be degrading COEN 345 22
  • 23. SQA Activities 7. Record Keeping and Reporting • Collect output and reports of SQA activities • Disseminate reports to software managers • Maintain archive of SQA reports • Maintain log of software development activity (especially testing) to satisfy legal requirements • Maintain institutional memory of the software development effort COEN 345 23
  • 24. SQA Group Who involves quality assurance activities? Software engineers, project managers, customers, sale people, SQA group Engineers involved the quality assurance work: - apply technical methods and measures - conduct formal technical review - perform well-planned software testing The SQA group’s role -> serves as the customer’s in-house representative assist the software engineering team in achieving high-quality The SQA group’s responsibility: - quality assurance planning oversight, record keeping, analysis and reporting The SQA group’s tasks: - Prepare a SQA plan for a project - Participate in the development of the project’s software process description - Review engineering activities to verify compliance with the defined process - Audits designated software work products to verify compliance the defined process - Ensure the deviations in software work and products according to a documented procedure - Records any noncompliance and reports to senior management COEN 345 24
  • 25. What experts say about quality? Two dominant views on software quality: Experts: 1. Conformance and degree of satisfaction to defined Philip B. Crosby specification. W. Edwards Deming Armand V. Feigenbaum 2. The products or services Joseph M. Juran capability to meet customer Walter A. Shewhart expectations – explicitly or implicitly Kaoru Ishikawa stated. COEN 345 25
  • 26. McCall’s Quality Model Correctness Product operation factors Reliability Efficiency Integrity Usability Quality Software Product revision factors Maintainability Flexibility Testability Product transition factors Portability Reusability Interoperability COEN 345 26
  • 27. Quality models Product operation Correctness-Does it do what I want? Reliability -Does it do it accurately all the time? Efficiency -Will it run on my machine as well as it can? Integrity -Is it secure? Usability-Can I run it? Product revision Maintainability-Can I fix it? Flexibility-Can I change it? Testability-Can I test it? Product transition Portability-Will I be able to use on another machine? Reusability-Will I be able to reuse some of the software? Interoperability -Will I be able to interface it with another machine? COEN 345 27
  • 28. Quality Models (cont.)  Product revision includes  maintainability (the effort required to locate and fix a fault in the program within its operating environment),  flexibility (the ease of making changes required by changes in the operating environment) and  testability (the ease of testing the program, to ensure that it is error-free and meets its specification). COEN 345 28
  • 29. Quality Models (cont.)  Product transition is all about  portability (the effort required to transfer a program from one environment to another),  reusability (the ease of reusing software in a different context) and  interoperability (the effort required to couple the system to another system). COEN 345 29
  • 30. Quality Models (cont.)  Quality of product operations depends on  correctness (the extent to which a program fulfils its specification),  reliability (the systems ability not to fail),  efficiency (further categorized into execution efficiency and storage efficiency and generally meaning the use of resources, e.g. processor time, storage),  integrity (the protection of the program from unauthorized access) and  usability (the ease of the software). COEN 345 30
  • 31. Some Quality Standards Development Offer SW Deliverable Development Int. Quality Process Ext. Quality Tools • ISO 12119 • ISO 9126 SQ • Z 67-142 PCTE • Z 67-133-1 • Z67-130 PQL • ISO 14102 AGL • ISO 9241 TTY • IEEE 829 DC • IEEE 1209 CASE Development Development Quality System Activities Process • IEEE 1012 V&V • ISO 12207 • CMM, ... • ISO9001 • Z 67-101 Project • MIL-Std 498 • ISO9000-3 • IEEE 830 Spec. COEN 345 31
  • 32. Software Development Life Cycle  SDLC model (~water fall)  Prototyping model  Spiral Model  Advanced Spiral Model  Object oriented Model COEN 345 32
  • 33. Structured development Project Management Specification Design Coding Test Verification & Validation COEN 345 33
  • 34. Instantiation of standard process Specification Project Management Design Configuration Management Coding Document Management Verification Testing & Validation Reuse & Training, COEN 345 34
  • 35. Process Improvement Specification Project Management Design Metrics Configuration Management Coding Document Management Verification Testing & Validation Reuse , Training, ... COEN 345 35
  • 36. Requirements definition SDLC Models Analysis Design Coding System Tests Installation and Conversion Operation and Maintenance COEN 345 36
  • 37. The prototyping Model Req. determination Prototype Design By the costumer Prototype implementation Prototype Evaluation By customer No Demands for Req. fulfilled? Corrections, Changes and additions Yes System Tests & System Conversion System Operation Acceptance tests and maintenance COEN 345 37
  • 38. Spiral Model  An iterative process, at each iteration, the following activities are performed:  Planning  Risk analysis and resolution  Engineering activities according to the stage of the project: design, coding, testing, installation and release  Customer evaluation, including comments, changes and additional requirements, etc. COEN 345 38
  • 39. Spirale Cycle Planning I. Risk analysis II. 3 Determine Operational objectives, Prototype alternatives, 2 constraints 1 review 3 1 2 Requirements Simulation, models, benchmarks Plan, life cycle Concept of plan Operation Development plan Validation of Detailed requirements design Evaluation by the customer Coding IV. etc. III. COEN 345 Engineering 39 49
  • 40. The advanced Spiral Model  The advanced Spiral model add extra emphasis on communication and negotiation between the customer and the developer  It contains six activities that are carried out in each iteration  Customer’s specification of requirements, comments and change demands  Developer’s planning activities  Developer’s risk analysis and resolution  Developer’s Design activities  Developer’s construction activities pertaining to coding, testing, installation and release  Customer’s evaluation COEN 345 40
  • 41. Requirements definition The object oriented Model Object-oriented analysis Object oriented design Addition to Reusable Lib. Reusability survey Components Of components lib. library Not accepted Component is not available Availability Of comp. in Lib.? Req. definition Analysis and Design Reusable component Is available Coding Customer’s needs Component tests accepted System construction Installation And conversion Development of System tests a new component Operation and maintenance COEN 345 41
  • 42. Factors affecting intensity of QA activities in the development process  The list of quality assurance activities to be applied  For each activity:  Timing  Type of quality assurance activity to be applied  Who performs the activity and resource required (development team, department staff, external QA team members…)  Resources required for removal of defects and introduction of changes COEN 345 42
  • 43. Factors affecting the required intensity of quality assurance activities  Project Factors  Magnitude of the project  Technical complexity and difficulty  Extent of reusable components  Severity of failure outcome if the project fails  Team Factors  Professional Qualification of the team members  Team acquaintance with the project and its experience in the area  Availability of staff members who professionally support the team  Familiarity with the team members ( percentage of new staff members in the team) COEN 345 43