Software Quality Assurance

SQA Workshop Version 1.0 By: B. M. Shahrier Majumder My Profile: http:// www.linkedin.com/in/shahrier

Workshop Contents
Quality and Process Concept
Quality Models
SQA Role
Audit System
Summery

Training Objectives
To prepare participants for effective implementation of SQA role in the organization
To provide some practice in the technique used

Logistics
Timings
Flow of Course (lecture, exercises, Quiz)
Course Material

Let Us Begin!!

Quality & Process Concepts

Quality Fit for use Conforms to the statement of requirement

Two Views of Quality Producer view of quality Customer view of quality Quality Assurance closes the gap

Quality Definition
Operationally, the word quality refers to products. A product is a quality product if it is defect free.
Producer View of Quality: The producer view of quality has these 4 characteristics. Doing the right thing, Doing it the right way, Doing it right the first time and Doing it on time without exceeding cost.
Customer View of Quality: Meeting requirements is a producer’s view of quality. This is the view of the organization responsibility for the project and process, and the products and services acquired, developed, and maintained by those processes.

Quality Gurus
DR. W. Edwards Deming
Philips Corseby
DR. Joseph Juran

Total Quality Management
A philosophy
A set of guiding principle
The foundation for a continuous improving organization
The application of quantitative methods and human resources
To improve processes
To satisfy customers, now and later

Definition of a Process
A process is a vehicle of communication, specifying the methods used to produce a product or service. It is the set of activities that represent the way work is to be performed.
Procedure: the step-by-step method followed to ensure that standards are met.

Why Process are Needed
From management perspective, process are needed to:
Explain to workers how to perform work tasks
Transfer knowledge from more experienced to less experienced workers
Assure predictability of work activities so that approximately the same deliverables will be produced with the same resources each time the process is followed
Establish a basic set of work tasks that can be continuously improved
Provide a means for involving workers in improving quality, productivity and customer satisfaction by having workers define and improve their own work process
Free management from their activities associated with “expediting work products” to send more time on activities such as planning, and customer & vendor interaction

Why Process are Needed
From worker perspective, process are needed to:
Increase the probability that the deliverables produced will be the desired deliverables
Put workers in charge of their own destiny because they know the standards by which their work products will be evaluated
Enable workers to devote their creativity to improving the business instead of having to develop work processes to build products
Enable workers to better plan their workday because of the predictability resulting from work processes

Process Management
Process management is a PDCA cycle. Process management processes provide the framework from within which an organization can implement process management on a daily basis.
PLAN Process Inventory – 1 Process Mapping – 2 Process Planning – 3 Enables process definition CHECK Process Measurement – 6 Enables process assessment ACT Process Improvement – 7 Enables process improvement DO Process Definition – 4 Process Controls – 5 Enables process execution

Quality Models

Industry Quality Models
There are many industry models available against which your organization can establish a baseline. Most commonly used models in the IT industry are:
ISO 9001:2000
CMMI

Quality System Elements ISO 9001
Management Responsibility
Quality System
Contract review
Design Control
Document & data control
Purchasing
Control of customer supplied product
Product identification & traceability
Process control
Inspection & testing
Control of inspection, measuring & test equipment
Inspection & test status
Control of non-conforming product
Corrective & preventive action
Handling, storage, packaging, presentation & delivery
Control of quality records
Internal quality audits
Training
Servicing
Statistical techniques

ISO 9001:2000
Released in December, 2000
Consistency with PDCA cycle
Based on eight quality management principles
Customer focus
Leadership
Involvement of people
Process approach
System approach to management
Continual improvement
Factual approach to decision making
Mutually beneficial supplier relationships

ISO 9001:2000
Logical grouping of clauses under the following heads:
Management Responsibility
Resource Management
Product realization
Measure, Analysis, Improvement

SEI CMMI
Capability Maturity Model Integration (CMMI) evaluates software process capability
Used for
- Where are we today?
- Where do we want to be?
- How do we get there? (Planning)
- Have we reached there? (Measurement)

Some Definitions
Software Process
A set of activities, methods, practices, and transformations that people use to develop and maintain software and associated products (e.g. plans, design documents, code, test cases, user manual, etc.)
Software process capability
Describes that range of expected results that can be achieved by following a software process.
Software process performance
Represents the actual results achieved by following a software process.

Some Definitions
Software Process Maturity
- The extent to which a specific process is explicitly defined, managed, measured, controlled and effective
- Implies a potential for growth in capability and indicates both the richness of an organization’s software process and the consistency with which it is applied in projects throughout the organization.
Institutionalization
Entails building an infrastructure and a corporate culture that supports the methods, practices, and procedures of the business so that they endure after those who originally defined them have gone.

Some Definitions
Maturity Level
- A well defined evolutionary plateau towards achieving a mature software process.
- Each level provides a layer in the foundation for continuous process improvement.
Process Area
Identifies a cluster of related activities that, when performed collectively, achieve a set of goals considered important for enhancing process capability

CMMI Levels
Level Process Characteristics
Initial (1) Process is informal and adhoc
Repeatable (2) Project management practices are institutionalized
Defined (3) Technical practices are integrated with management practices and institutionalized
Managed (4) Product and process quantitatively controlled
Optimizing (5) Process improvement institutionalized

Quality Management
Setting Quality goals / policy / objectives
Building support for Quality
Planning Quality
Measuring quality
Controlling Quality / Poor Quality
Improve Quality

Process Orientation of Quality
Quality has been defined in many different ways but always to satisfy the ‘customers’
Quality can be measured
Quality control detects errors
Quality assurance prevent errors
Processes determine the quality of the product
Product can improve only if process improve continuously
Quality is every person’s responsibility
It should be imbibed as a pert of day-to-day work

Definitions
Quality Control
The operational techniques and activities that are used to fulfill requirements for quality.
Quality Assurance
All those planned and systematic activities implemented within the quality system and demonstrated as needed to provide adequate confidence that an entity will fulfill requirements for quality.

QC vs QA
QC QA
 Product  Process
 Reactive  Proactive
 Line function  Staff function
 Find defects  Prevent defects

QC vs QA Examples
QC QA
 Walkthrough  Quality Audit
 Testing  Defining process
 Inspection  Selection of tools
 Checkpoint review  Training

Cost of Quality
Failure Costs
Project rework
Overtime
Maintenance costs
Lost credibility
Providing alternate service
Lost management time
Complaints, rebates & damage claims
Lost assets, opportunity
Unrealized savings
Appraisal Costs
Reviews
Inspections
Testing
Prevention Costs
Quality audit
Planning quality improvement
Quality training
Installation
- Project selection process
- Planning database
- Improved programming techniques

SQA Role

Quality of requirements / specification
Good planning
Use of trained personnel
Usage of pre-defined techniques
Use of templates, checklists
Through review
Requirements sign off
A good SRS is
- Unambiguous, complete, correct, verifiable
- Helps customers describe what they want to obtain
- Helps supplier understand what the customer wants

Quality of Design
Good planning
Use trained resources
Choice of appropriate model, techniques and tools for design
- Top-down vs. Bottom-up approaches selected / mixed to get most suitable approach
- Build in the attributes related to reuse of components, product attributes like scalability, interoperability, product performance and so on based on application requirement, clear interfaces
Use of standard templates
Review checklist to ensure no major aspect is missed out
Review of design documents
- Various specialists / review focus – e.g. optimization, technical feasibility
- Traceability of design to requirements
- Ensure consistency between low level design

For Better Code
Use coding standards
Use proper code samples and templates
Plan to ensure common libraries are available
Conduct code review
Use checklist for review
Good planning
Train people

For Successful Testing
Plan for the test strategy, test cases
Review test plans, test conditions, etc.
Have independent testing teams
Test the units before moving to integration testing
Use pre-defined forms for test scripts, test logs, etc.
Use processes for testing and defect management

For Successful Post Testing Activities
Plan for the acceptance testing in the initial stages of the project
Clearly agree on the acceptance criteria with the customer
Have clear documentation of the product in the form of:
- Installation manual
- Maintenance manual
- User manual
Review the manuals and test before delivery
Include exception handling

Quality Pyramid Assure Quality Control Quality Define Quality Objective Function Quality Assurance Measurement Quality Control Procedure Standards Management Policies / Plans

Quality Management System
Process
Procedures
Guidelines
Standards
Checklists
Formats
Templates

Value of Documentation
Assists for conformity to customer requirements and quality improvement
Provides appropriate training
Enables repeatability & traceability
Provision of objective evidence
Evaluate effectiveness of QMS

Audit System

Audit-Definition
Definition:
A systematic, independent and documented process for obtaining audit evidence and evaluating it objectively to determine the extent to which audit criteria are fulfilled.
Audit Criteria:
Set of policies, procedures or requirements used as reference
Audit Evidence:
Records, statement of facts or other information which are relevant to the audit criteria

Purpose of Audits
Management Tool
Positive and constructive process
Identifies problem areas
Increases process compliance
Increases process effectiveness

Audits
NOT to be used to assign blame
Does NOT replace inspection / testing activities
Should NOT be used as a means to accept or reject products
CANNOT support an ineffective system

Types of Audits
First Party
Second Party
Third Party

First Party Audits
To check compliance with QMS
To find & correct system shortfalls
To identify improvements of QMS
To enhance quality awareness
To increase cross-department understanding
A requirement of ISO 9001

Second Party Audits
To evaluate potential suppliers / subcontractors
To keep an eye on suppliers / subcontractors
To help suppliers improve their QA
To improve end products and services
To limit costs of external failure

Third Party Audits
To provide objective evidence
To identify required improvements
Part of certification process
To provide credibility to claims of quality

The Players
Auditor
- A person who has the qualifications to perform quality audits
Client
- A person or organization requesting the audit
Auditee
- An organization to be audited

The Players in a First Party (Internal) Audit
Auditor
- An employee (or sometimes a consultant) who is trained as an auditor and is independent of the area audited
Client
- The senior management of the organization
Auditee
- The project / department / unit being audited

The Players in a Second Party Audit
Auditor
- An employee (or sometimes a consultant) who is trained as an auditor and is representing a customer or potential customer
Client
- The senior management of the customer organization
Auditee
- The project / department / unit being audited

The Players in a Third Party Audit
Auditor
- A recognize auditor belonging to a certifying body
Client
- The senior management of the organization
Auditee
- The project / department / unit and organization being audited

Basic Purpose
Objective Evidence
Does the Quality System meet the requirements of the relevant standard or contract?
Does the organization do what the QMS requires?
Is the QMS effective for the Organization’s business?

Objective Evidence
A factual statement that can be verified
Not based on opinion or preference
Not based on emotion
Based on actual observations & statements

Evidence – Quality System
Quality Manual referring to procedures
Procedures covering the standard being followed (ISO / CMMI)
Departmental Handbooks
Project proposals / Plans
Instructions
Policy and objectives
Responsibilities and authorities

Evidence – Implementation Records
Review records
Minutes of meeting
Audit reports
Testing records
Delivery notes
Training records

Evidence of Effectiveness
Records / results
Measurements / metrics
Milestone achievement
Management review
Customer feedback
Timely corrective action
Customer complaints

The Audit System

Prepare Long Term Audit Plan
Typically for the whole year
Aspects to plan for:
- How many cycles (typically once every 2-3 months)
- What units / departments / areas / projects will be covered in every cycle – this would depend on the status and importance of the unit / department and the extent of changes expected

For Every Cycle
Review and revise the list of auditee units / departments / projects
Nominate a lead auditor and audit team
Make initial contact with auditees
Finalize audit program

Review & Revise Auditee List
Review / revise the list of auditee units / departments / projects based on:
- The extent of activities
- Changes in structure, personnel, type of work
- Findings of previous audit
- Proposed changes in the projects

Nominate Audit Team
To be done by EPG head or SQA lead or mutually agreed and planned between the PM and SQA
Identify “Lead Auditor” for the audit in case of IA across the organization
Identify all auditors of the audit
- Number of auditors
- Assignment to auditees areas
Ensure availability of auditors for:
- Preparation, interviews, reporting, follow-up (approx 4-5 hours per project / support group)
Provide training to untrained auditors
Check whether the auditee and auditor are independent
Confirm that the auditee and auditor have no “issues” that may impact objectively
Set up initial contact between auditor and auditee

Lead Auditor Responsibilities
Manage the team
Assist in team selection
Preparation of program / checklist
Quality control over the team’s work
Interfacing with auditees management
Preparation / submission of audit report
Conduct audit interviews

Auditor Responsibilities
Communicate audit requirements
Be active and efficient
Document observations
Report results
Verify corrective action effectiveness
Remain with scope
Support other team members

Auditee Responsibilities
Inform team members
Appoint guides
Provide logistical resources
Cooperate with auditors
Share information, records
Agree on non-compliances
Propose and implement corrective actions

Finalize Schedule for Audit Cycle
Schedule interview of 1-3 hours for each project / department
1-2 auditors to conduct the interviews (new auditors must go in pairs)
Scheduling to be completed around two weeks before audit cycle start
Circulate and get confirmation from all auditees

Checklist Benefits
Ensures coverage is balanced
Assists in preparing audit team
Help maintain correct pace
Provides a record of the audit for future reference
Ensure nothing is forgotten!

Checklist Preparation
Use checklist of the previous audit as a starting point
Study the document QMS, Procedures, guidelines
Read relevant section of the Model
Prepare separate lists for each project / support function
Consider time allocated and key areas

Remember
Become fully conversant with the area before preparing / modifying checklists
Make separate checklists for different support functions
You may have to make different checklists for different project types
With more experience you can make smaller checklists or just bullet points
Checklist is a tool and should be servant to the auditor – CHECKLIST SHOULD NOT BE ALLOWED TO CONTROL THE AUDITOR
Checklists used in one audit can be used as a starting point in the next audit
Standard checklists may be included in the QMS after 1-2 cycles

The Opening Meeting
Purpose
Scheduling
Agenda
Tips

Purpose
Confirms scope and process of audit
Put the auditee at ease
Create the “right” atmosphere
(In external audit) Give the auditors an insight to the management commitment to quality

Scheduling
Before the start of audit interviews
After the audit schedule is finalized
Present in the opening meeting:
- Senior Management / MD
- EPG / SQA
- Lead Auditor for the audit
- Other auditors for the audit
- Senior-most representatives of all auditee groups (e.g. PMs, Department Heads)
- Others who may interested

Agenda
Make sure all participants are presents
Introduction to the audit team (Senior Manager / MD or Lead Auditor for the audit cycle)
Circulation of the attendance record
Lead Auditor to explain
- Purpose / scope of the audit cycle
- The audit interview process
- Need for openness
- Confidentiality
- Documentation of findings
- Reporting

Agenda (contd.)
Circulate / display audit schedule
Discuss any logistics related issues
Provide clarifications
Invite everyone to closing meeting
The Sr. Manager / MD can emphasize
- Use the findings will be to improve the process
- Need to share information openly

Tips for the Auditors
Keep it short
- Schedule 30 minutes
- Try to finish in 20 minutes
Be well prepared
Conduct meeting in businesslike manager
Keep a record (attendance)
Do not let the MD hijack the session

Audit Investigations
Approach
Interviewing
Audit Trail
Recording Findings

Approach
The auditor must keep control
The auditor must manage his / her time
Use prepared checklists as a guide
Judgment – is there a problem or not
The audit team must keep in touch

Objective Evidence Records Document Statements Observations Relevance Significance Existence Accuracy Remember: only objective evidence is permitted

Audit Trail
Record the facts
Is it on your checklist?
Is there time available?
Pass to the appropriate Auditor
Consult the Lead Auditor
Note: if it is important, someone must look at it

Identifying Problems
Focus on the key matters
Decide whether or not the Auditee is the right person to ask the question
Consider if there are further symptoms
Where in the process could the root cause lie?
Always verify evidence of non-compliance

Purpose of Interview
Elaboration
Explanation
Work status – what really happens?
Basis for evidence
Understanding
Dialogue / rapport
Perspective

Starting the Interview
Find a suitable location near their workplace
Introduce yourself
Explain the process
“ Assessing the system – not individuals”
Be friendly but polite
Dialogue / rapport
Perspective
Interviewing is your main tool

The Interview
The auditor must keep control
The auditor must manage his/her time
Split time between managers and staff
Work through the checklist
- If no problems – go quickly to next issue
- Problems – investigate to get objective evidence & idea of magnitude
- No sense digging until something is found

Useful Types of Questions
Open (STARTING)
Follow up
Probing
Focusing
Closed (ENDING)

Examples of Open Questions
Please describe your responsibilities
Tell me about …?
How does ….?
Please explain how ….?
Please describe the process ….?

Examples of Probing Questions
Where does ….?
When did …?
What is ….?

Examples of Closed Questions
Is this ….?
Do you …?
Does this ….?
Please show me ….?

Remember
Interviewing is your main tool
Look at the evidence
Listen to the auditees
Make sure you are asking the right persons
Be ready to handle auditee reactions
Watch out for auditee reactions
Verify details of non-compliance
Pass on information to team members
Focus on the key matters
Take help from other auditors / lead auditor

Non-Compliances
Also called
- Non-conformities
- Non-conformances
- Deficiencies
- Discrepancies
- Deviations

Types of Non-compliances
Major non-compliances
- A consistent, significant breakdown of the quality system
Minor non-compliances
- Isolated or one-off failures; localized impact
Observations
- Warning about potential non-compliances

Recording Non-compliances
What
Acknowledged by Auditee
At the time they are found
Using OBJECTIVE evidence
- Where, when, who, (how)
Non-compliance statements must be
- Accurate
- Complete
- Helpful
- Brief
Does it pass the ‘so-what’ test?
Anticipate the corrective action

Audit Reporting
Report Contents
Closing Meeting
Audit Records

Purpose
To consolidate the activities and findings related to the Audit cycle
To provide feedback to the audit participants, Senior Management and the auditors
To collect all related records and close the Audit

Audit Report - Contents
The Audit Cycle reference
Date of the Audit Cycle
Scope of the Audit Cycle
Lead Auditor and other auditors
Summer of non-compliances
Summery of good practices identified
Target dates for closing all non-compliances

Audit Report - Contents
Statistics
- Total meeting hours
- Total areas / project audited
- Number of major non-compliance
- Number of minor non-compliance
- Number of observations
- Number of good practices observed
- Number of persons in the opening meeting
Appendices
- Audit Cycle Schedule
- List of the attendees in the opening meeting
- Non-compliance list / tracking sheet

Closing Meeting
Introduction and thank you
Purpose / objective / scope
Statistics
Important findings
Follow-up actions
Any questions
Acknowledgement of report

Audit Records
Audit Cycle schedule
Opening meeting attendance
Audit Report
Non-compliance Reports
Checklist used
Interview notes
Closing meeting attendance

Corrective Action Follow-up
Identification
Implementation
Tracking and Closure

Identification of the Non-compliance
The Auditor raises the problem
The facts
The non-compliance
The department / project responsible

Corrective Action Proposal
Auditee proposes corrective action
Root cause analysis
Immediate remedial action
Long term corrective action
Auditor evaluates the proposed corrective actions
The organization’s QA functions provides advice to the responsible manager

Corrective Action Implementation
Auditee implements agreed corrective action
Keeps records of implementation
Confirms that there is no other occurrences that need to be corrected
Confirms that the probability of similar occurrences are considerably reduced

Verification
Performed by Auditor
Are there other similar non-compliance?
Has the root cause been addressed?
Has the likelihood of recurrence been assessed?
Have they followed the CA procedure?
Is a track of all corrective actions being maintained?

Closure
Close non-compliance after verification
Raise process improvement proposal if the corrective action is deemed useful on a wider basis

Tracking of Non-compliances
EPG to track ALL non-compliances to closure, using some tracking sheet / database

Conclusion
Auditor Attributes
Purpose of Audit
Audit System

Auditors Attributes
Positive
Pragmatic
Professional
Prepared
Perceptive

Purpose of Audit
Compliance to:
- Contract, requirements, proposal
- Internal quality management system
- A quality / process standard (e.g. CMMI)
Provide confidence to management and stakeholders
Identify process improvements

Audit System
Must be planned / scheduled
Conducted by trained auditors
Finding based on “objective evidence”
Actionable findings must be tracked to closure

Software Quality Assurance

by B.M. Shahrier Majumder on May 15, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

1 Embed 12

Statistics

Software Quality Assurance — Presentation Transcript