Software Quality Assurance

SQA Workshop Version 1.0 By: B. M. Shahrier Majumder My Profile: http:// www.linkedin.com/in/shahrier

Workshop Contents Quality and Process Concept Quality Models SQA Role Audit System Summery

Training Objectives To prepare participants for effective implementation of SQA role in the organization To provide some practice in the technique used

Logistics Timings Flow of Course (lecture, exercises, Quiz) Course Material

Quality Fit for use Conforms to the statement of requirement

Two Views of Quality Producer view of quality Customer view of quality Quality Assurance closes the gap

Quality Definition Operationally, the word quality refers to products. A product is a quality product if it is defect free. Producer View of Quality: The producer view of quality has these 4 characteristics. Doing the right thing, Doing it the right way, Doing it right the first time and Doing it on time without exceeding cost. Customer View of Quality: Meeting requirements is a producer’s view of quality. This is the view of the organization responsibility for the project and process, and the products and services acquired, developed, and maintained by those processes.

Quality Gurus DR. W. Edwards Deming Philips Corseby DR. Joseph Juran

Total Quality Management A philosophy A set of guiding principle The foundation for a continuous improving organization The application of quantitative methods and human resources To improve processes To satisfy customers, now and later

Definition of a Process A process is a vehicle of communication, specifying the methods used to produce a product or service. It is the set of activities that represent the way work is to be performed. Procedure: the step-by-step method followed to ensure that standards are met.

Why Process are Needed From management perspective, process are needed to: Explain to workers how to perform work tasks Transfer knowledge from more experienced to less experienced workers Assure predictability of work activities so that approximately the same deliverables will be produced with the same resources each time the process is followed Establish a basic set of work tasks that can be continuously improved Provide a means for involving workers in improving quality, productivity and customer satisfaction by having workers define and improve their own work process Free management from their activities associated with “expediting work products” to send more time on activities such as planning, and customer & vendor interaction

Why Process are Needed From worker perspective, process are needed to: Increase the probability that the deliverables produced will be the desired deliverables Put workers in charge of their own destiny because they know the standards by which their work products will be evaluated Enable workers to devote their creativity to improving the business instead of having to develop work processes to build products Enable workers to better plan their workday because of the predictability resulting from work processes

Process Management Process management is a PDCA cycle. Process management processes provide the framework from within which an organization can implement process management on a daily basis. PLAN Process Inventory – 1 Process Mapping – 2 Process Planning – 3 Enables process definition CHECK Process Measurement – 6 Enables process assessment ACT Process Improvement – 7 Enables process improvement DO Process Definition – 4 Process Controls – 5 Enables process execution

Industry Quality Models There are many industry models available against which your organization can establish a baseline. Most commonly used models in the IT industry are: ISO 9001:2000 CMMI

Quality System Elements ISO 9001 Management Responsibility Quality System Contract review Design Control Document & data control Purchasing Control of customer supplied product Product identification & traceability Process control Inspection & testing Control of inspection, measuring & test equipment Inspection & test status Control of non-conforming product Corrective & preventive action Handling, storage, packaging, presentation & delivery Control of quality records Internal quality audits Training Servicing Statistical techniques

ISO 9001:2000 Released in December, 2000 Consistency with PDCA cycle Based on eight quality management principles Customer focus Leadership Involvement of people Process approach System approach to management Continual improvement Factual approach to decision making Mutually beneficial supplier relationships

ISO 9001:2000 Logical grouping of clauses under the following heads: Management Responsibility Resource Management Product realization Measure, Analysis, Improvement

SEI CMMI Capability Maturity Model Integration (CMMI) evaluates software process capability Used for - Where are we today? - Where do we want to be? - How do we get there? (Planning) - Have we reached there? (Measurement)

Some Definitions Software Process A set of activities, methods, practices, and transformations that people use to develop and maintain software and associated products (e.g. plans, design documents, code, test cases, user manual, etc.) Software process capability Describes that range of expected results that can be achieved by following a software process. Software process performance Represents the actual results achieved by following a software process.

Some Definitions Software Process Maturity - The extent to which a specific process is explicitly defined, managed, measured, controlled and effective - Implies a potential for growth in capability and indicates both the richness of an organization’s software process and the consistency with which it is applied in projects throughout the organization. Institutionalization Entails building an infrastructure and a corporate culture that supports the methods, practices, and procedures of the business so that they endure after those who originally defined them have gone.

Some Definitions Maturity Level - A well defined evolutionary plateau towards achieving a mature software process. - Each level provides a layer in the foundation for continuous process improvement. Process Area Identifies a cluster of related activities that, when performed collectively, achieve a set of goals considered important for enhancing process capability

CMMI Levels Level Process Characteristics Initial (1) Process is informal and adhoc Repeatable (2) Project management practices are institutionalized Defined (3) Technical practices are integrated with management practices and institutionalized Managed (4) Product and process quantitatively controlled Optimizing (5) Process improvement institutionalized

Quality Management Setting Quality goals / policy / objectives Building support for Quality Planning Quality Measuring quality Controlling Quality / Poor Quality Improve Quality

Process Orientation of Quality Quality has been defined in many different ways but always to satisfy the ‘customers’ Quality can be measured Quality control detects errors Quality assurance prevent errors Processes determine the quality of the product Product can improve only if process improve continuously Quality is every person’s responsibility It should be imbibed as a pert of day-to-day work

Definitions Quality Control The operational techniques and activities that are used to fulfill requirements for quality. Quality Assurance All those planned and systematic activities implemented within the quality system and demonstrated as needed to provide adequate confidence that an entity will fulfill requirements for quality.

QC vs QA QC QA  Product  Process  Reactive  Proactive  Line function  Staff function  Find defects  Prevent defects

QC vs QA Examples QC QA  Walkthrough  Quality Audit  Testing  Defining process  Inspection  Selection of tools  Checkpoint review  Training

Cost of Quality Failure Costs Project rework Overtime Maintenance costs Lost credibility Providing alternate service Lost management time Complaints, rebates & damage claims Lost assets, opportunity Unrealized savings Appraisal Costs Reviews Inspections Testing Prevention Costs Quality audit Planning quality improvement Quality training Installation - Project selection process - Planning database - Improved programming techniques

Quality of requirements / specification Good planning Use of trained personnel Usage of pre-defined techniques Use of templates, checklists Through review Requirements sign off A good SRS is - Unambiguous, complete, correct, verifiable - Helps customers describe what they want to obtain - Helps supplier understand what the customer wants

Quality of Design Good planning Use trained resources Choice of appropriate model, techniques and tools for design - Top-down vs. Bottom-up approaches selected / mixed to get most suitable approach - Build in the attributes related to reuse of components, product attributes like scalability, interoperability, product performance and so on based on application requirement, clear interfaces Use of standard templates Review checklist to ensure no major aspect is missed out Review of design documents - Various specialists / review focus – e.g. optimization, technical feasibility - Traceability of design to requirements - Ensure consistency between low level design

For Better Code Use coding standards Use proper code samples and templates Plan to ensure common libraries are available Conduct code review Use checklist for review Good planning Train people

For Successful Testing Plan for the test strategy, test cases Review test plans, test conditions, etc. Have independent testing teams Test the units before moving to integration testing Use pre-defined forms for test scripts, test logs, etc. Use processes for testing and defect management

For Successful Post Testing Activities Plan for the acceptance testing in the initial stages of the project Clearly agree on the acceptance criteria with the customer Have clear documentation of the product in the form of: - Installation manual - Maintenance manual - User manual Review the manuals and test before delivery Include exception handling

Quality Pyramid Assure Quality Control Quality Define Quality Objective Function Quality Assurance Measurement Quality Control Procedure Standards Management Policies / Plans

Quality Management System Process Procedures Guidelines Standards Checklists Formats Templates

Value of Documentation Assists for conformity to customer requirements and quality improvement Provides appropriate training Enables repeatability & traceability Provision of objective evidence Evaluate effectiveness of QMS

Audit-Definition Definition: A systematic, independent and documented process for obtaining audit evidence and evaluating it objectively to determine the extent to which audit criteria are fulfilled. Audit Criteria: Set of policies, procedures or requirements used as reference Audit Evidence: Records, statement of facts or other information which are relevant to the audit criteria

Purpose of Audits Management Tool Positive and constructive process Identifies problem areas Increases process compliance Increases process effectiveness

Audits NOT to be used to assign blame Does NOT replace inspection / testing activities Should NOT be used as a means to accept or reject products CANNOT support an ineffective system

Types of Audits First Party Second Party Third Party

First Party Audits To check compliance with QMS To find & correct system shortfalls To identify improvements of QMS To enhance quality awareness To increase cross-department understanding A requirement of ISO 9001

Second Party Audits To evaluate potential suppliers / subcontractors To keep an eye on suppliers / subcontractors To help suppliers improve their QA To improve end products and services To limit costs of external failure

Third Party Audits To provide objective evidence To identify required improvements Part of certification process To provide credibility to claims of quality

The Players Auditor - A person who has the qualifications to perform quality audits Client - A person or organization requesting the audit Auditee - An organization to be audited

The Players in a First Party (Internal) Audit Auditor - An employee (or sometimes a consultant) who is trained as an auditor and is independent of the area audited Client - The senior management of the organization Auditee - The project / department / unit being audited

The Players in a Second Party Audit Auditor - An employee (or sometimes a consultant) who is trained as an auditor and is representing a customer or potential customer Client - The senior management of the customer organization Auditee - The project / department / unit being audited

The Players in a Third Party Audit Auditor - A recognize auditor belonging to a certifying body Client - The senior management of the organization Auditee - The project / department / unit and organization being audited

Basic Purpose Objective Evidence Does the Quality System meet the requirements of the relevant standard or contract? Does the organization do what the QMS requires? Is the QMS effective for the Organization’s business?

Objective Evidence A factual statement that can be verified Not based on opinion or preference Not based on emotion Based on actual observations & statements

Evidence – Quality System Quality Manual referring to procedures Procedures covering the standard being followed (ISO / CMMI) Departmental Handbooks Project proposals / Plans Instructions Policy and objectives Responsibilities and authorities

Evidence – Implementation Records Review records Minutes of meeting Audit reports Testing records Delivery notes Training records

Evidence of Effectiveness Records / results Measurements / metrics Milestone achievement Management review Customer feedback Timely corrective action Customer complaints

Prepare Long Term Audit Plan Typically for the whole year Aspects to plan for: - How many cycles (typically once every 2-3 months) - What units / departments / areas / projects will be covered in every cycle – this would depend on the status and importance of the unit / department and the extent of changes expected

For Every Cycle Review and revise the list of auditee units / departments / projects Nominate a lead auditor and audit team Make initial contact with auditees Finalize audit program

Review & Revise Auditee List Review / revise the list of auditee units / departments / projects based on: - The extent of activities - Changes in structure, personnel, type of work - Findings of previous audit - Proposed changes in the projects

Nominate Audit Team To be done by EPG head or SQA lead or mutually agreed and planned between the PM and SQA Identify “Lead Auditor” for the audit in case of IA across the organization Identify all auditors of the audit - Number of auditors - Assignment to auditees areas Ensure availability of auditors for: - Preparation, interviews, reporting, follow-up (approx 4-5 hours per project / support group) Provide training to untrained auditors Check whether the auditee and auditor are independent Confirm that the auditee and auditor have no “issues” that may impact objectively Set up initial contact between auditor and auditee

Lead Auditor Responsibilities Manage the team Assist in team selection Preparation of program / checklist Quality control over the team’s work Interfacing with auditees management Preparation / submission of audit report Conduct audit interviews

Auditor Responsibilities Communicate audit requirements Be active and efficient Document observations Report results Verify corrective action effectiveness Remain with scope Support other team members

Auditee Responsibilities Inform team members Appoint guides Provide logistical resources Cooperate with auditors Share information, records Agree on non-compliances Propose and implement corrective actions

Finalize Schedule for Audit Cycle Schedule interview of 1-3 hours for each project / department 1-2 auditors to conduct the interviews (new auditors must go in pairs) Scheduling to be completed around two weeks before audit cycle start Circulate and get confirmation from all auditees

Checklist Benefits Ensures coverage is balanced Assists in preparing audit team Help maintain correct pace Provides a record of the audit for future reference Ensure nothing is forgotten!

Checklist Preparation Use checklist of the previous audit as a starting point Study the document QMS, Procedures, guidelines Read relevant section of the Model Prepare separate lists for each project / support function Consider time allocated and key areas

Remember Become fully conversant with the area before preparing / modifying checklists Make separate checklists for different support functions You may have to make different checklists for different project types With more experience you can make smaller checklists or just bullet points Checklist is a tool and should be servant to the auditor – CHECKLIST SHOULD NOT BE ALLOWED TO CONTROL THE AUDITOR Checklists used in one audit can be used as a starting point in the next audit Standard checklists may be included in the QMS after 1-2 cycles

The Opening Meeting Purpose Scheduling Agenda Tips

Purpose Confirms scope and process of audit Put the auditee at ease Create the “right” atmosphere (In external audit) Give the auditors an insight to the management commitment to quality

Scheduling Before the start of audit interviews After the audit schedule is finalized Present in the opening meeting: - Senior Management / MD - EPG / SQA - Lead Auditor for the audit - Other auditors for the audit - Senior-most representatives of all auditee groups (e.g. PMs, Department Heads) - Others who may interested

Agenda Make sure all participants are presents Introduction to the audit team (Senior Manager / MD or Lead Auditor for the audit cycle) Circulation of the attendance record Lead Auditor to explain - Purpose / scope of the audit cycle - The audit interview process - Need for openness - Confidentiality - Documentation of findings - Reporting

Agenda (contd.) Circulate / display audit schedule Discuss any logistics related issues Provide clarifications Invite everyone to closing meeting The Sr. Manager / MD can emphasize - Use the findings will be to improve the process - Need to share information openly

Tips for the Auditors Keep it short - Schedule 30 minutes - Try to finish in 20 minutes Be well prepared Conduct meeting in businesslike manager Keep a record (attendance) Do not let the MD hijack the session

Audit Investigations Approach Interviewing Audit Trail Recording Findings

Approach The auditor must keep control The auditor must manage his / her time Use prepared checklists as a guide Judgment – is there a problem or not The audit team must keep in touch

Objective Evidence Records Document Statements Observations Relevance Significance Existence Accuracy Remember: only objective evidence is permitted

Audit Trail Record the facts Is it on your checklist? Is there time available? Pass to the appropriate Auditor Consult the Lead Auditor Note: if it is important, someone must look at it

Identifying Problems Focus on the key matters Decide whether or not the Auditee is the right person to ask the question Consider if there are further symptoms Where in the process could the root cause lie? Always verify evidence of non-compliance

Purpose of Interview Elaboration Explanation Work status – what really happens? Basis for evidence Understanding Dialogue / rapport Perspective

Starting the Interview Find a suitable location near their workplace Introduce yourself Explain the process “ Assessing the system – not individuals” Be friendly but polite Dialogue / rapport Perspective Interviewing is your main tool

The Interview The auditor must keep control The auditor must manage his/her time Split time between managers and staff Work through the checklist - If no problems – go quickly to next issue - Problems – investigate to get objective evidence & idea of magnitude - No sense digging until something is found

Useful Types of Questions Open (STARTING) Follow up Probing Focusing Closed (ENDING)

Examples of Open Questions Please describe your responsibilities Tell me about …? How does ….? Please explain how ….? Please describe the process ….?

Examples of Probing Questions Where does ….? When did …? What is ….?

Examples of Closed Questions Is this ….? Do you …? Does this ….? Please show me ….?

Remember Interviewing is your main tool Look at the evidence Listen to the auditees Make sure you are asking the right persons Be ready to handle auditee reactions Watch out for auditee reactions Verify details of non-compliance Pass on information to team members Focus on the key matters Take help from other auditors / lead auditor

Non-Compliances Also called - Non-conformities - Non-conformances - Deficiencies - Discrepancies - Deviations

Types of Non-compliances Major non-compliances - A consistent, significant breakdown of the quality system Minor non-compliances - Isolated or one-off failures; localized impact Observations - Warning about potential non-compliances

Recording Non-compliances What Acknowledged by Auditee At the time they are found Using OBJECTIVE evidence - Where, when, who, (how) Non-compliance statements must be - Accurate - Complete - Helpful - Brief Does it pass the ‘so-what’ test? Anticipate the corrective action

Audit Reporting Report Contents Closing Meeting Audit Records

Purpose To consolidate the activities and findings related to the Audit cycle To provide feedback to the audit participants, Senior Management and the auditors To collect all related records and close the Audit

Audit Report - Contents The Audit Cycle reference Date of the Audit Cycle Scope of the Audit Cycle Lead Auditor and other auditors Summer of non-compliances Summery of good practices identified Target dates for closing all non-compliances

Audit Report - Contents Statistics - Total meeting hours - Total areas / project audited - Number of major non-compliance - Number of minor non-compliance - Number of observations - Number of good practices observed - Number of persons in the opening meeting Appendices - Audit Cycle Schedule - List of the attendees in the opening meeting - Non-compliance list / tracking sheet

Closing Meeting Introduction and thank you Purpose / objective / scope Statistics Important findings Follow-up actions Any questions Acknowledgement of report

Audit Records Audit Cycle schedule Opening meeting attendance Audit Report Non-compliance Reports Checklist used Interview notes Closing meeting attendance

Corrective Action Follow-up Identification Implementation Tracking and Closure

Identification of the Non-compliance The Auditor raises the problem The facts The non-compliance The department / project responsible

Corrective Action Proposal Auditee proposes corrective action Root cause analysis Immediate remedial action Long term corrective action Auditor evaluates the proposed corrective actions The organization’s QA functions provides advice to the responsible manager

Corrective Action Implementation Auditee implements agreed corrective action Keeps records of implementation Confirms that there is no other occurrences that need to be corrected Confirms that the probability of similar occurrences are considerably reduced

Verification Performed by Auditor Are there other similar non-compliance? Has the root cause been addressed? Has the likelihood of recurrence been assessed? Have they followed the CA procedure? Is a track of all corrective actions being maintained?

Closure Close non-compliance after verification Raise process improvement proposal if the corrective action is deemed useful on a wider basis

Tracking of Non-compliances EPG to track ALL non-compliances to closure, using some tracking sheet / database

Conclusion Auditor Attributes Purpose of Audit Audit System

Auditors Attributes Positive Pragmatic Professional Prepared Perceptive

Purpose of Audit Compliance to: - Contract, requirements, proposal - Internal quality management system - A quality / process standard (e.g. CMMI) Provide confidence to management and stakeholders Identify process improvements

Audit System Must be planned / scheduled Conducted by trained auditors Finding based on “objective evidence” Actionable findings must be tracked to closure

Software Quality Assurance

More Related Content

What's hot

Viewers also liked

Similar to Software Quality Assurance

More from B M Shahrier Majumder, PMP, CSM

Recently uploaded

Software Quality Assurance