Software Quality Assurance


Published on

Published in: Business
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Software Quality Assurance

  1. 1. SQA Workshop Version 1.0 By: B. M. Shahrier Majumder My Profile: http://
  2. 2. Workshop Contents <ul><li>Quality and Process Concept </li></ul><ul><li>Quality Models </li></ul><ul><li>SQA Role </li></ul><ul><li>Audit System </li></ul><ul><li>Summery </li></ul>
  3. 3. Training Objectives <ul><li>To prepare participants for effective implementation of SQA role in the organization </li></ul><ul><li>To provide some practice in the technique used </li></ul>
  4. 4. Logistics <ul><li>Timings </li></ul><ul><li>Flow of Course (lecture, exercises, Quiz) </li></ul><ul><li>Course Material </li></ul>
  5. 5. <ul><li>Let Us Begin!! </li></ul>
  6. 6. Quality & Process Concepts
  7. 7. Quality Fit for use Conforms to the statement of requirement
  8. 8. Two Views of Quality Producer view of quality Customer view of quality Quality Assurance closes the gap
  9. 9. Quality Definition <ul><li>Operationally, the word quality refers to products. A product is a quality product if it is defect free. </li></ul><ul><li>Producer View of Quality: The producer view of quality has these 4 characteristics. Doing the right thing, Doing it the right way, Doing it right the first time and Doing it on time without exceeding cost. </li></ul><ul><li>Customer View of Quality: Meeting requirements is a producer’s view of quality. This is the view of the organization responsibility for the project and process, and the products and services acquired, developed, and maintained by those processes. </li></ul>
  10. 10. Quality Gurus <ul><li>DR. W. Edwards Deming </li></ul><ul><li>Philips Corseby </li></ul><ul><li>DR. Joseph Juran </li></ul>
  11. 11. Total Quality Management <ul><li>A philosophy </li></ul><ul><li>A set of guiding principle </li></ul><ul><li>The foundation for a continuous improving organization </li></ul><ul><li>The application of quantitative methods and human resources </li></ul><ul><ul><li>To improve processes </li></ul></ul><ul><ul><li>To satisfy customers, now and later </li></ul></ul>
  12. 12. Definition of a Process <ul><li>A process is a vehicle of communication, specifying the methods used to produce a product or service. It is the set of activities that represent the way work is to be performed. </li></ul><ul><li>Procedure: the step-by-step method followed to ensure that standards are met. </li></ul>
  13. 13. Why Process are Needed <ul><li>From management perspective, process are needed to: </li></ul><ul><li>Explain to workers how to perform work tasks </li></ul><ul><li>Transfer knowledge from more experienced to less experienced workers </li></ul><ul><li>Assure predictability of work activities so that approximately the same deliverables will be produced with the same resources each time the process is followed </li></ul><ul><li>Establish a basic set of work tasks that can be continuously improved </li></ul><ul><li>Provide a means for involving workers in improving quality, productivity and customer satisfaction by having workers define and improve their own work process </li></ul><ul><li>Free management from their activities associated with “expediting work products” to send more time on activities such as planning, and customer & vendor interaction </li></ul>
  14. 14. Why Process are Needed <ul><li>From worker perspective, process are needed to: </li></ul><ul><li>Increase the probability that the deliverables produced will be the desired deliverables </li></ul><ul><li>Put workers in charge of their own destiny because they know the standards by which their work products will be evaluated </li></ul><ul><li>Enable workers to devote their creativity to improving the business instead of having to develop work processes to build products </li></ul><ul><li>Enable workers to better plan their workday because of the predictability resulting from work processes </li></ul>
  15. 15. Process Management <ul><li>Process management is a PDCA cycle. Process management processes provide the framework from within which an organization can implement process management on a daily basis. </li></ul>PLAN Process Inventory – 1 Process Mapping – 2 Process Planning – 3 Enables process definition CHECK Process Measurement – 6 Enables process assessment ACT Process Improvement – 7 Enables process improvement DO Process Definition – 4 Process Controls – 5 Enables process execution
  16. 16. Quality Models
  17. 17. Industry Quality Models <ul><li>There are many industry models available against which your organization can establish a baseline. Most commonly used models in the IT industry are: </li></ul><ul><li>ISO 9001:2000 </li></ul><ul><li>CMMI </li></ul>
  18. 18. Quality System Elements ISO 9001 <ul><li>Management Responsibility </li></ul><ul><li>Quality System </li></ul><ul><li>Contract review </li></ul><ul><li>Design Control </li></ul><ul><li>Document & data control </li></ul><ul><li>Purchasing </li></ul><ul><li>Control of customer supplied product </li></ul><ul><li>Product identification & traceability </li></ul><ul><li>Process control </li></ul><ul><li>Inspection & testing </li></ul><ul><li>Control of inspection, measuring & test equipment </li></ul><ul><li>Inspection & test status </li></ul><ul><li>Control of non-conforming product </li></ul><ul><li>Corrective & preventive action </li></ul><ul><li>Handling, storage, packaging, presentation & delivery </li></ul><ul><li>Control of quality records </li></ul><ul><li>Internal quality audits </li></ul><ul><li>Training </li></ul><ul><li>Servicing </li></ul><ul><li>Statistical techniques </li></ul>
  19. 19. ISO 9001:2000 <ul><li>Released in December, 2000 </li></ul><ul><li>Consistency with PDCA cycle </li></ul><ul><li>Based on eight quality management principles </li></ul><ul><ul><li>Customer focus </li></ul></ul><ul><ul><li>Leadership </li></ul></ul><ul><ul><li>Involvement of people </li></ul></ul><ul><ul><li>Process approach </li></ul></ul><ul><ul><li>System approach to management </li></ul></ul><ul><ul><li>Continual improvement </li></ul></ul><ul><ul><li>Factual approach to decision making </li></ul></ul><ul><ul><li>Mutually beneficial supplier relationships </li></ul></ul>
  20. 20. ISO 9001:2000 <ul><li>Logical grouping of clauses under the following heads: </li></ul><ul><ul><li>Management Responsibility </li></ul></ul><ul><ul><li>Resource Management </li></ul></ul><ul><ul><li>Product realization </li></ul></ul><ul><ul><li>Measure, Analysis, Improvement </li></ul></ul>
  21. 21. SEI CMMI <ul><li>Capability Maturity Model Integration (CMMI) evaluates software process capability </li></ul><ul><li>Used for </li></ul><ul><ul><li>- Where are we today? </li></ul></ul><ul><ul><li>- Where do we want to be? </li></ul></ul><ul><ul><li>- How do we get there? (Planning) </li></ul></ul><ul><ul><li>- Have we reached there? (Measurement) </li></ul></ul>
  22. 22. Some Definitions <ul><li>Software Process </li></ul><ul><li>A set of activities, methods, practices, and transformations that people use to develop and maintain software and associated products (e.g. plans, design documents, code, test cases, user manual, etc.) </li></ul><ul><li>Software process capability </li></ul><ul><li>Describes that range of expected results that can be achieved by following a software process. </li></ul><ul><li>Software process performance </li></ul><ul><li>Represents the actual results achieved by following a software process. </li></ul>
  23. 23. Some Definitions <ul><li>Software Process Maturity </li></ul><ul><li>- The extent to which a specific process is explicitly defined, managed, measured, controlled and effective </li></ul><ul><li>- Implies a potential for growth in capability and indicates both the richness of an organization’s software process and the consistency with which it is applied in projects throughout the organization. </li></ul><ul><li>Institutionalization </li></ul><ul><li>Entails building an infrastructure and a corporate culture that supports the methods, practices, and procedures of the business so that they endure after those who originally defined them have gone. </li></ul>
  24. 24. Some Definitions <ul><li>Maturity Level </li></ul><ul><li>- A well defined evolutionary plateau towards achieving a mature software process. </li></ul><ul><li>- Each level provides a layer in the foundation for continuous process improvement. </li></ul><ul><li>Process Area </li></ul><ul><li>Identifies a cluster of related activities that, when performed collectively, achieve a set of goals considered important for enhancing process capability </li></ul>
  25. 25. CMMI Levels <ul><li>Level Process Characteristics </li></ul><ul><li>Initial (1) Process is informal and adhoc </li></ul><ul><li>Repeatable (2) Project management practices are institutionalized </li></ul><ul><li>Defined (3) Technical practices are integrated with management practices and institutionalized </li></ul><ul><li>Managed (4) Product and process quantitatively controlled </li></ul><ul><li>Optimizing (5) Process improvement institutionalized </li></ul>
  26. 26. Quality Management <ul><li>Setting Quality goals / policy / objectives </li></ul><ul><li>Building support for Quality </li></ul><ul><li>Planning Quality </li></ul><ul><li>Measuring quality </li></ul><ul><li>Controlling Quality / Poor Quality </li></ul><ul><li>Improve Quality </li></ul>
  27. 27. Process Orientation of Quality <ul><li>Quality has been defined in many different ways but always to satisfy the ‘customers’ </li></ul><ul><li>Quality can be measured </li></ul><ul><li>Quality control detects errors </li></ul><ul><li>Quality assurance prevent errors </li></ul><ul><li>Processes determine the quality of the product </li></ul><ul><li>Product can improve only if process improve continuously </li></ul><ul><li>Quality is every person’s responsibility </li></ul><ul><li>It should be imbibed as a pert of day-to-day work </li></ul>
  28. 28. Definitions <ul><li>Quality Control </li></ul><ul><li>The operational techniques and activities that are used to fulfill requirements for quality. </li></ul><ul><li>Quality Assurance </li></ul><ul><li>All those planned and systematic activities implemented within the quality system and demonstrated as needed to provide adequate confidence that an entity will fulfill requirements for quality. </li></ul>
  29. 29. QC vs QA <ul><li>QC QA </li></ul><ul><li> Product  Process </li></ul><ul><li> Reactive  Proactive </li></ul><ul><li> Line function  Staff function </li></ul><ul><li> Find defects  Prevent defects </li></ul>
  30. 30. QC vs QA Examples <ul><li>QC QA </li></ul><ul><li> Walkthrough  Quality Audit </li></ul><ul><li> Testing  Defining process </li></ul><ul><li> Inspection  Selection of tools </li></ul><ul><li> Checkpoint review  Training </li></ul>
  31. 31. Cost of Quality <ul><li>Failure Costs </li></ul><ul><li>Project rework </li></ul><ul><li>Overtime </li></ul><ul><li>Maintenance costs </li></ul><ul><li>Lost credibility </li></ul><ul><li>Providing alternate service </li></ul><ul><li>Lost management time </li></ul><ul><li>Complaints, rebates & damage claims </li></ul><ul><li>Lost assets, opportunity </li></ul><ul><li>Unrealized savings </li></ul><ul><li>Appraisal Costs </li></ul><ul><li>Reviews </li></ul><ul><li>Inspections </li></ul><ul><li>Testing </li></ul><ul><li>Prevention Costs </li></ul><ul><li>Quality audit </li></ul><ul><li>Planning quality improvement </li></ul><ul><li>Quality training </li></ul><ul><li>Installation </li></ul><ul><li>- Project selection process </li></ul><ul><li>- Planning database </li></ul><ul><li>- Improved programming techniques </li></ul>
  32. 32. SQA Role
  33. 33. Quality of requirements / specification <ul><li>Good planning </li></ul><ul><li>Use of trained personnel </li></ul><ul><li>Usage of pre-defined techniques </li></ul><ul><li>Use of templates, checklists </li></ul><ul><li>Through review </li></ul><ul><li>Requirements sign off </li></ul><ul><li>A good SRS is </li></ul><ul><li>- Unambiguous, complete, correct, verifiable </li></ul><ul><li>- Helps customers describe what they want to obtain </li></ul><ul><li>- Helps supplier understand what the customer wants </li></ul>
  34. 34. Quality of Design <ul><li>Good planning </li></ul><ul><li>Use trained resources </li></ul><ul><li>Choice of appropriate model, techniques and tools for design </li></ul><ul><li>- Top-down vs. Bottom-up approaches selected / mixed to get most suitable approach </li></ul><ul><li>- Build in the attributes related to reuse of components, product attributes like scalability, interoperability, product performance and so on based on application requirement, clear interfaces </li></ul><ul><li>Use of standard templates </li></ul><ul><li>Review checklist to ensure no major aspect is missed out </li></ul><ul><li>Review of design documents </li></ul><ul><li>- Various specialists / review focus – e.g. optimization, technical feasibility </li></ul><ul><li>- Traceability of design to requirements </li></ul><ul><li>- Ensure consistency between low level design </li></ul>
  35. 35. For Better Code <ul><li>Use coding standards </li></ul><ul><li>Use proper code samples and templates </li></ul><ul><li>Plan to ensure common libraries are available </li></ul><ul><li>Conduct code review </li></ul><ul><li>Use checklist for review </li></ul><ul><li>Good planning </li></ul><ul><li>Train people </li></ul>
  36. 36. For Successful Testing <ul><li>Plan for the test strategy, test cases </li></ul><ul><li>Review test plans, test conditions, etc. </li></ul><ul><li>Have independent testing teams </li></ul><ul><li>Test the units before moving to integration testing </li></ul><ul><li>Use pre-defined forms for test scripts, test logs, etc. </li></ul><ul><li>Use processes for testing and defect management </li></ul>
  37. 37. For Successful Post Testing Activities <ul><li>Plan for the acceptance testing in the initial stages of the project </li></ul><ul><li>Clearly agree on the acceptance criteria with the customer </li></ul><ul><li>Have clear documentation of the product in the form of: </li></ul><ul><li>- Installation manual </li></ul><ul><li>- Maintenance manual </li></ul><ul><li>- User manual </li></ul><ul><li>Review the manuals and test before delivery </li></ul><ul><li>Include exception handling </li></ul>
  38. 38. Quality Pyramid Assure Quality Control Quality Define Quality Objective Function Quality Assurance Measurement Quality Control Procedure Standards Management Policies / Plans
  39. 39. Quality Management System <ul><li>Process </li></ul><ul><li>Procedures </li></ul><ul><li>Guidelines </li></ul><ul><li>Standards </li></ul><ul><li>Checklists </li></ul><ul><li>Formats </li></ul><ul><li>Templates </li></ul>
  40. 40. Value of Documentation <ul><li>Assists for conformity to customer requirements and quality improvement </li></ul><ul><li>Provides appropriate training </li></ul><ul><li>Enables repeatability & traceability </li></ul><ul><li>Provision of objective evidence </li></ul><ul><li>Evaluate effectiveness of QMS </li></ul>
  41. 41. Audit System
  42. 42. Audit-Definition <ul><li>Definition: </li></ul><ul><li>A systematic, independent and documented process for obtaining audit evidence and evaluating it objectively to determine the extent to which audit criteria are fulfilled. </li></ul><ul><li>Audit Criteria: </li></ul><ul><li>Set of policies, procedures or requirements used as reference </li></ul><ul><li>Audit Evidence: </li></ul><ul><li>Records, statement of facts or other information which are relevant to the audit criteria </li></ul>
  43. 43. Purpose of Audits <ul><li>Management Tool </li></ul><ul><li>Positive and constructive process </li></ul><ul><li>Identifies problem areas </li></ul><ul><li>Increases process compliance </li></ul><ul><li>Increases process effectiveness </li></ul>
  44. 44. Audits <ul><li>NOT to be used to assign blame </li></ul><ul><li>Does NOT replace inspection / testing activities </li></ul><ul><li>Should NOT be used as a means to accept or reject products </li></ul><ul><li>CANNOT support an ineffective system </li></ul>
  45. 45. Types of Audits <ul><li>First Party </li></ul><ul><li>Second Party </li></ul><ul><li>Third Party </li></ul>
  46. 46. First Party Audits <ul><li>To check compliance with QMS </li></ul><ul><li>To find & correct system shortfalls </li></ul><ul><li>To identify improvements of QMS </li></ul><ul><li>To enhance quality awareness </li></ul><ul><li>To increase cross-department understanding </li></ul><ul><li>A requirement of ISO 9001 </li></ul>
  47. 47. Second Party Audits <ul><li>To evaluate potential suppliers / subcontractors </li></ul><ul><li>To keep an eye on suppliers / subcontractors </li></ul><ul><li>To help suppliers improve their QA </li></ul><ul><li>To improve end products and services </li></ul><ul><li>To limit costs of external failure </li></ul>
  48. 48. Third Party Audits <ul><li>To provide objective evidence </li></ul><ul><li>To identify required improvements </li></ul><ul><li>Part of certification process </li></ul><ul><li>To provide credibility to claims of quality </li></ul>
  49. 49. The Players <ul><li>Auditor </li></ul><ul><li>- A person who has the qualifications to perform quality audits </li></ul><ul><li>Client </li></ul><ul><li>- A person or organization requesting the audit </li></ul><ul><li>Auditee </li></ul><ul><li>- An organization to be audited </li></ul>
  50. 50. The Players in a First Party (Internal) Audit <ul><li>Auditor </li></ul><ul><li>- An employee (or sometimes a consultant) who is trained as an auditor and is independent of the area audited </li></ul><ul><li>Client </li></ul><ul><li>- The senior management of the organization </li></ul><ul><li>Auditee </li></ul><ul><li>- The project / department / unit being audited </li></ul>
  51. 51. The Players in a Second Party Audit <ul><li>Auditor </li></ul><ul><li>- An employee (or sometimes a consultant) who is trained as an auditor and is representing a customer or potential customer </li></ul><ul><li>Client </li></ul><ul><li>- The senior management of the customer organization </li></ul><ul><li>Auditee </li></ul><ul><li>- The project / department / unit being audited </li></ul>
  52. 52. The Players in a Third Party Audit <ul><li>Auditor </li></ul><ul><li>- A recognize auditor belonging to a certifying body </li></ul><ul><li>Client </li></ul><ul><li>- The senior management of the organization </li></ul><ul><li>Auditee </li></ul><ul><li>- The project / department / unit and organization being audited </li></ul>
  53. 53. Basic Purpose <ul><li>Objective Evidence </li></ul><ul><li>Does the Quality System meet the requirements of the relevant standard or contract? </li></ul><ul><li>Does the organization do what the QMS requires? </li></ul><ul><li>Is the QMS effective for the Organization’s business? </li></ul>
  54. 54. Objective Evidence <ul><li>A factual statement that can be verified </li></ul><ul><li>Not based on opinion or preference </li></ul><ul><li>Not based on emotion </li></ul><ul><li>Based on actual observations & statements </li></ul>
  55. 55. Evidence – Quality System <ul><li>Quality Manual referring to procedures </li></ul><ul><li>Procedures covering the standard being followed (ISO / CMMI) </li></ul><ul><li>Departmental Handbooks </li></ul><ul><li>Project proposals / Plans </li></ul><ul><li>Instructions </li></ul><ul><li>Policy and objectives </li></ul><ul><li>Responsibilities and authorities </li></ul>
  56. 56. Evidence – Implementation Records <ul><li>Review records </li></ul><ul><li>Minutes of meeting </li></ul><ul><li>Audit reports </li></ul><ul><li>Testing records </li></ul><ul><li>Delivery notes </li></ul><ul><li>Training records </li></ul>
  57. 57. Evidence of Effectiveness <ul><li>Records / results </li></ul><ul><li>Measurements / metrics </li></ul><ul><li>Milestone achievement </li></ul><ul><li>Management review </li></ul><ul><li>Customer feedback </li></ul><ul><li>Timely corrective action </li></ul><ul><li>Customer complaints </li></ul>
  58. 58. The Audit System
  59. 59. Prepare Long Term Audit Plan <ul><li>Typically for the whole year </li></ul><ul><li>Aspects to plan for: </li></ul><ul><li>- How many cycles (typically once every 2-3 months) </li></ul><ul><li>- What units / departments / areas / projects will be covered in every cycle – this would depend on the status and importance of the unit / department and the extent of changes expected </li></ul>
  60. 60. For Every Cycle <ul><li>Review and revise the list of auditee units / departments / projects </li></ul><ul><li>Nominate a lead auditor and audit team </li></ul><ul><li>Make initial contact with auditees </li></ul><ul><li>Finalize audit program </li></ul>
  61. 61. Review & Revise Auditee List <ul><li>Review / revise the list of auditee units / departments / projects based on: </li></ul><ul><li>- The extent of activities </li></ul><ul><li>- Changes in structure, personnel, type of work </li></ul><ul><li>- Findings of previous audit </li></ul><ul><li>- Proposed changes in the projects </li></ul>
  62. 62. Nominate Audit Team <ul><li>To be done by EPG head or SQA lead or mutually agreed and planned between the PM and SQA </li></ul><ul><li>Identify “Lead Auditor” for the audit in case of IA across the organization </li></ul><ul><li>Identify all auditors of the audit </li></ul><ul><li>- Number of auditors </li></ul><ul><li>- Assignment to auditees areas </li></ul><ul><li>Ensure availability of auditors for: </li></ul><ul><li>- Preparation, interviews, reporting, follow-up (approx 4-5 hours per project / support group) </li></ul><ul><li>Provide training to untrained auditors </li></ul><ul><li>Check whether the auditee and auditor are independent </li></ul><ul><li>Confirm that the auditee and auditor have no “issues” that may impact objectively </li></ul><ul><li>Set up initial contact between auditor and auditee </li></ul>
  63. 63. Lead Auditor Responsibilities <ul><li>Manage the team </li></ul><ul><li>Assist in team selection </li></ul><ul><li>Preparation of program / checklist </li></ul><ul><li>Quality control over the team’s work </li></ul><ul><li>Interfacing with auditees management </li></ul><ul><li>Preparation / submission of audit report </li></ul><ul><li>Conduct audit interviews </li></ul>
  64. 64. Auditor Responsibilities <ul><li>Communicate audit requirements </li></ul><ul><li>Be active and efficient </li></ul><ul><li>Document observations </li></ul><ul><li>Report results </li></ul><ul><li>Verify corrective action effectiveness </li></ul><ul><li>Remain with scope </li></ul><ul><li>Support other team members </li></ul>
  65. 65. Auditee Responsibilities <ul><li>Inform team members </li></ul><ul><li>Appoint guides </li></ul><ul><li>Provide logistical resources </li></ul><ul><li>Cooperate with auditors </li></ul><ul><li>Share information, records </li></ul><ul><li>Agree on non-compliances </li></ul><ul><li>Propose and implement corrective actions </li></ul>
  66. 66. Finalize Schedule for Audit Cycle <ul><li>Schedule interview of 1-3 hours for each project / department </li></ul><ul><li>1-2 auditors to conduct the interviews (new auditors must go in pairs) </li></ul><ul><li>Scheduling to be completed around two weeks before audit cycle start </li></ul><ul><li>Circulate and get confirmation from all auditees </li></ul>
  67. 67. Checklist Benefits <ul><li>Ensures coverage is balanced </li></ul><ul><li>Assists in preparing audit team </li></ul><ul><li>Help maintain correct pace </li></ul><ul><li>Provides a record of the audit for future reference </li></ul><ul><li>Ensure nothing is forgotten! </li></ul>
  68. 68. Checklist Preparation <ul><li>Use checklist of the previous audit as a starting point </li></ul><ul><li>Study the document QMS, Procedures, guidelines </li></ul><ul><li>Read relevant section of the Model </li></ul><ul><li>Prepare separate lists for each project / support function </li></ul><ul><li>Consider time allocated and key areas </li></ul>
  69. 69. Remember <ul><li>Become fully conversant with the area before preparing / modifying checklists </li></ul><ul><li>Make separate checklists for different support functions </li></ul><ul><li>You may have to make different checklists for different project types </li></ul><ul><li>With more experience you can make smaller checklists or just bullet points </li></ul><ul><li>Checklist is a tool and should be servant to the auditor – CHECKLIST SHOULD NOT BE ALLOWED TO CONTROL THE AUDITOR </li></ul><ul><li>Checklists used in one audit can be used as a starting point in the next audit </li></ul><ul><li>Standard checklists may be included in the QMS after 1-2 cycles </li></ul>
  70. 70. The Opening Meeting <ul><li>Purpose </li></ul><ul><li>Scheduling </li></ul><ul><li>Agenda </li></ul><ul><li>Tips </li></ul>
  71. 71. Purpose <ul><li>Confirms scope and process of audit </li></ul><ul><li>Put the auditee at ease </li></ul><ul><li>Create the “right” atmosphere </li></ul><ul><li>(In external audit) Give the auditors an insight to the management commitment to quality </li></ul>
  72. 72. Scheduling <ul><li>Before the start of audit interviews </li></ul><ul><li>After the audit schedule is finalized </li></ul><ul><li>Present in the opening meeting: </li></ul><ul><li>- Senior Management / MD </li></ul><ul><li>- EPG / SQA </li></ul><ul><li>- Lead Auditor for the audit </li></ul><ul><li>- Other auditors for the audit </li></ul><ul><li>- Senior-most representatives of all auditee groups (e.g. PMs, Department Heads) </li></ul><ul><li>- Others who may interested </li></ul>
  73. 73. Agenda <ul><li>Make sure all participants are presents </li></ul><ul><li>Introduction to the audit team (Senior Manager / MD or Lead Auditor for the audit cycle) </li></ul><ul><li>Circulation of the attendance record </li></ul><ul><li>Lead Auditor to explain </li></ul><ul><li>- Purpose / scope of the audit cycle </li></ul><ul><li>- The audit interview process </li></ul><ul><li>- Need for openness </li></ul><ul><li>- Confidentiality </li></ul><ul><li>- Documentation of findings </li></ul><ul><li>- Reporting </li></ul>
  74. 74. Agenda (contd.) <ul><li>Circulate / display audit schedule </li></ul><ul><li>Discuss any logistics related issues </li></ul><ul><li>Provide clarifications </li></ul><ul><li>Invite everyone to closing meeting </li></ul><ul><li>The Sr. Manager / MD can emphasize </li></ul><ul><li>- Use the findings will be to improve the process </li></ul><ul><li>- Need to share information openly </li></ul>
  75. 75. Tips for the Auditors <ul><li>Keep it short </li></ul><ul><li>- Schedule 30 minutes </li></ul><ul><li>- Try to finish in 20 minutes </li></ul><ul><li>Be well prepared </li></ul><ul><li>Conduct meeting in businesslike manager </li></ul><ul><li>Keep a record (attendance) </li></ul><ul><li>Do not let the MD hijack the session </li></ul>
  76. 76. Audit Investigations <ul><li>Approach </li></ul><ul><li>Interviewing </li></ul><ul><li>Audit Trail </li></ul><ul><li>Recording Findings </li></ul>
  77. 77. Approach <ul><li>The auditor must keep control </li></ul><ul><li>The auditor must manage his / her time </li></ul><ul><li>Use prepared checklists as a guide </li></ul><ul><li>Judgment – is there a problem or not </li></ul><ul><li>The audit team must keep in touch </li></ul>
  78. 78. Objective Evidence Records Document Statements Observations Relevance Significance Existence Accuracy Remember: only objective evidence is permitted
  79. 79. Audit Trail <ul><li>Record the facts </li></ul><ul><li>Is it on your checklist? </li></ul><ul><li>Is there time available? </li></ul><ul><li>Pass to the appropriate Auditor </li></ul><ul><li>Consult the Lead Auditor </li></ul>Note: if it is important, someone must look at it
  80. 80. Identifying Problems <ul><li>Focus on the key matters </li></ul><ul><li>Decide whether or not the Auditee is the right person to ask the question </li></ul><ul><li>Consider if there are further symptoms </li></ul><ul><li>Where in the process could the root cause lie? </li></ul><ul><li>Always verify evidence of non-compliance </li></ul>
  81. 81. Purpose of Interview <ul><li>Elaboration </li></ul><ul><li>Explanation </li></ul><ul><li>Work status – what really happens? </li></ul><ul><li>Basis for evidence </li></ul><ul><li>Understanding </li></ul><ul><li>Dialogue / rapport </li></ul><ul><li>Perspective </li></ul>
  82. 82. Starting the Interview <ul><li>Find a suitable location near their workplace </li></ul><ul><li>Introduce yourself </li></ul><ul><li>Explain the process </li></ul><ul><li>“ Assessing the system – not individuals” </li></ul><ul><li>Be friendly but polite </li></ul><ul><li>Dialogue / rapport </li></ul><ul><li>Perspective </li></ul>Interviewing is your main tool
  83. 83. The Interview <ul><li>The auditor must keep control </li></ul><ul><li>The auditor must manage his/her time </li></ul><ul><li>Split time between managers and staff </li></ul><ul><li>Work through the checklist </li></ul><ul><li>- If no problems – go quickly to next issue </li></ul><ul><li>- Problems – investigate to get objective evidence & idea of magnitude </li></ul><ul><li>- No sense digging until something is found </li></ul>
  84. 84. Useful Types of Questions <ul><li>Open (STARTING) </li></ul><ul><li>Follow up </li></ul><ul><li>Probing </li></ul><ul><li>Focusing </li></ul><ul><li>Closed (ENDING) </li></ul>
  85. 85. Examples of Open Questions <ul><li>Please describe your responsibilities </li></ul><ul><li>Tell me about …? </li></ul><ul><li>How does ….? </li></ul><ul><li>Please explain how ….? </li></ul><ul><li>Please describe the process ….? </li></ul>
  86. 86. Examples of Probing Questions <ul><li>Where does ….? </li></ul><ul><li>When did …? </li></ul><ul><li>What is ….? </li></ul>
  87. 87. Examples of Closed Questions <ul><li>Is this ….? </li></ul><ul><li>Do you …? </li></ul><ul><li>Does this ….? </li></ul><ul><li>Please show me ….? </li></ul>
  88. 88. Remember <ul><li>Interviewing is your main tool </li></ul><ul><li>Look at the evidence </li></ul><ul><li>Listen to the auditees </li></ul><ul><li>Make sure you are asking the right persons </li></ul><ul><li>Be ready to handle auditee reactions </li></ul><ul><li>Watch out for auditee reactions </li></ul><ul><li>Verify details of non-compliance </li></ul><ul><li>Pass on information to team members </li></ul><ul><li>Focus on the key matters </li></ul><ul><li>Take help from other auditors / lead auditor </li></ul>
  89. 89. Non-Compliances <ul><li>Also called </li></ul><ul><li>- Non-conformities </li></ul><ul><li>- Non-conformances </li></ul><ul><li>- Deficiencies </li></ul><ul><li>- Discrepancies </li></ul><ul><li>- Deviations </li></ul>
  90. 90. Types of Non-compliances <ul><li>Major non-compliances </li></ul><ul><li>- A consistent, significant breakdown of the quality system </li></ul><ul><li>Minor non-compliances </li></ul><ul><li>- Isolated or one-off failures; localized impact </li></ul><ul><li>Observations </li></ul><ul><li>- Warning about potential non-compliances </li></ul>
  91. 91. Recording Non-compliances <ul><li>What </li></ul><ul><li>Acknowledged by Auditee </li></ul><ul><li>At the time they are found </li></ul><ul><li>Using OBJECTIVE evidence </li></ul><ul><li>- Where, when, who, (how) </li></ul><ul><li>Non-compliance statements must be </li></ul><ul><li>- Accurate </li></ul><ul><li>- Complete </li></ul><ul><li>- Helpful </li></ul><ul><li>- Brief </li></ul><ul><li>Does it pass the ‘so-what’ test? </li></ul><ul><li>Anticipate the corrective action </li></ul>
  92. 92. Audit Reporting <ul><li>Report Contents </li></ul><ul><li>Closing Meeting </li></ul><ul><li>Audit Records </li></ul>
  93. 93. Purpose <ul><li>To consolidate the activities and findings related to the Audit cycle </li></ul><ul><li>To provide feedback to the audit participants, Senior Management and the auditors </li></ul><ul><li>To collect all related records and close the Audit </li></ul>
  94. 94. Audit Report - Contents <ul><li>The Audit Cycle reference </li></ul><ul><li>Date of the Audit Cycle </li></ul><ul><li>Scope of the Audit Cycle </li></ul><ul><li>Lead Auditor and other auditors </li></ul><ul><li>Summer of non-compliances </li></ul><ul><li>Summery of good practices identified </li></ul><ul><li>Target dates for closing all non-compliances </li></ul>
  95. 95. Audit Report - Contents <ul><li>Statistics </li></ul><ul><li>- Total meeting hours </li></ul><ul><li>- Total areas / project audited </li></ul><ul><li>- Number of major non-compliance </li></ul><ul><li>- Number of minor non-compliance </li></ul><ul><li>- Number of observations </li></ul><ul><li>- Number of good practices observed </li></ul><ul><li>- Number of persons in the opening meeting </li></ul><ul><li>Appendices </li></ul><ul><li>- Audit Cycle Schedule </li></ul><ul><li>- List of the attendees in the opening meeting </li></ul><ul><li>- Non-compliance list / tracking sheet </li></ul>
  96. 96. Closing Meeting <ul><li>Introduction and thank you </li></ul><ul><li>Purpose / objective / scope </li></ul><ul><li>Statistics </li></ul><ul><li>Important findings </li></ul><ul><li>Follow-up actions </li></ul><ul><li>Any questions </li></ul><ul><li>Acknowledgement of report </li></ul>
  97. 97. Audit Records <ul><li>Audit Cycle schedule </li></ul><ul><li>Opening meeting attendance </li></ul><ul><li>Audit Report </li></ul><ul><li>Non-compliance Reports </li></ul><ul><li>Checklist used </li></ul><ul><li>Interview notes </li></ul><ul><li>Closing meeting attendance </li></ul>
  98. 98. Corrective Action Follow-up <ul><li>Identification </li></ul><ul><li>Implementation </li></ul><ul><li>Tracking and Closure </li></ul>
  99. 99. Identification of the Non-compliance <ul><li>The Auditor raises the problem </li></ul><ul><li>The facts </li></ul><ul><li>The non-compliance </li></ul><ul><li>The department / project responsible </li></ul>
  100. 100. Corrective Action Proposal <ul><li>Auditee proposes corrective action </li></ul><ul><li>Root cause analysis </li></ul><ul><li>Immediate remedial action </li></ul><ul><li>Long term corrective action </li></ul><ul><li>Auditor evaluates the proposed corrective actions </li></ul>The organization’s QA functions provides advice to the responsible manager
  101. 101. Corrective Action Implementation <ul><li>Auditee implements agreed corrective action </li></ul><ul><li>Keeps records of implementation </li></ul><ul><li>Confirms that there is no other occurrences that need to be corrected </li></ul><ul><li>Confirms that the probability of similar occurrences are considerably reduced </li></ul>
  102. 102. Verification <ul><li>Performed by Auditor </li></ul><ul><li>Are there other similar non-compliance? </li></ul><ul><li>Has the root cause been addressed? </li></ul><ul><li>Has the likelihood of recurrence been assessed? </li></ul><ul><li>Have they followed the CA procedure? </li></ul><ul><li>Is a track of all corrective actions being maintained? </li></ul>
  103. 103. Closure <ul><li>Close non-compliance after verification </li></ul><ul><li>Raise process improvement proposal if the corrective action is deemed useful on a wider basis </li></ul>
  104. 104. Tracking of Non-compliances <ul><li>EPG to track ALL non-compliances to closure, using some tracking sheet / database </li></ul>
  105. 105. Conclusion <ul><li>Auditor Attributes </li></ul><ul><li>Purpose of Audit </li></ul><ul><li>Audit System </li></ul>
  106. 106. Auditors Attributes <ul><li>Positive </li></ul><ul><li>Pragmatic </li></ul><ul><li>Professional </li></ul><ul><li>Prepared </li></ul><ul><li>Perceptive </li></ul>
  107. 107. Purpose of Audit <ul><li>Compliance to: </li></ul><ul><li>- Contract, requirements, proposal </li></ul><ul><li>- Internal quality management system </li></ul><ul><li>- A quality / process standard (e.g. CMMI) </li></ul><ul><li>Provide confidence to management and stakeholders </li></ul><ul><li>Identify process improvements </li></ul>
  108. 108. Audit System <ul><li>Must be planned / scheduled </li></ul><ul><li>Conducted by trained auditors </li></ul><ul><li>Finding based on “objective evidence” </li></ul><ul><li>Actionable findings must be tracked to closure </li></ul>