Pharmaceutical Auditing and Inspections Professor Peivand Pirouzi 2010

Pharmaceutical Auditing and
Inspections
One day Corporate Training
Workshop
Professor Peivand Pirouzi
Toronto, Canada
2010

LESSON 01
• The key topics covered in this lesson are as follows:
• Introduction
• Audit definition
• Why Audit
• Three forms of audit
• Overview of the Audit System
• Audit team role and responsibilities

What is an Audit?
• The word itself comes from the Latin root “to
hear”, referring to the hearing of oral evidence of a
captain or cargo master shouting out the contents
of a ship to auditors who were employed by the
king to verify stated conditions.
•The auditor was responsible for seeing that the
cargo was properly recorded so appropriate taxes
would be paid.

What is an Audit?
• Webster’s New World Dictionary defines audit as “A regular
examination and checking of accounts or financial records; a
settlement or adjustment of accounts; or a final statement of
account”.
• structured around financial matters
• limited application

What is an Audit?
• The American Society for Quality Control defines a quality audit as
• “A systematic evaluation of the acts and decisions by people with
respect to quality, in order to independently verify or evaluate and
report compliance to the operational requirements of the quality
program or the specification or contract requirements of the product
or service”

What is an Audit?
•FDA Definition-
• “an established, systematic, independent, examination of
a manufacturer’s quality system that is performed at
defined intervals and at sufficient frequency to ensure
that both quality system activities and the results of such
activities comply with specified quality system
procedures, that these procedures are implemented
effectively, and that these procedures are suitable to
achieve quality system objectives.” (U.S FDA 820.3
(07/95))

What is an Audit?
• HPFBI definition- self inspection-
• “the purpose of self-inspection is to evaluate the fabricator’s,
packager’s/labeler’s, distributor’s, wholesaler’s or importer’s compliance
with GMP in all aspects of production and quality control. The self-
inspection programme is designed to detect any shortcomings in the
implementation of GMP and to recommend the necessary corrective
actions” ( Canadian GMP C.02.012(1)(b))

What is an Audit?
• ISO definition-
• “a systematic and independent examination to determine whether quality
activities and related results comply with planned arrangements, and
whether these arrangements are implemented effectively and are suitable to
achieve objectives.”

What is an Audit?
•Summary definition- a documented, systematic
examination, periodically done by independent,
qualified personnel to determine:
• What the firm does to develop and produce a quality
product;
• If the activities are scientifically sound, rationally based
and properly defined;
• If the activities are integrated into a system;
• If the activities are implemented and documented; and
• If the activities are effective

Why Audit?
• Survival- in any competitive situation, business will go to the more
efficient provider. If you don’t know how well (or poorly) your
organization is performing, someone will take that business away.
• Regulations imposed on your operations
• Identify and resolve your issues before they get out of hand

Three Forms of Quality Audits
•Product audit: a detailed reexamination/ retest
of the FP by the purchaser prior to acceptance.
Product audit has a limited scope.
•Process audit: examines a selected portion of the
work effort
•System audit: examines all or part of the
management control elements for conformance
and effectiveness.

What is an Audit?
Key words:
• Systematic – in application
• People – with special skills and training
• Independently – prevent bias

General and Specific Benefits of a
Quality Audit
• A: General benefits of quality audits
• input for management decision
• information regarding potential or actual risks
• identification of area of opportunity
• identification of cost reduction area
• verification of compliance

Quality Audit
•B: Specific benefits of quality audits:
•System audit:
• Increase competitiveness
• Reduced market barriers
• Decreased costs of multi-customer audits
•Process audits:
• Verification of appropriate control for input, action, and
output.
• Assurance of a better or more consistent final product or
service.

Quality Audit
• Product audit:
• Elimination of non conformities
• Reduced costs and wastes
• Increased confidence between supplier and customer

•Performed on selected portion of something.
•Requires some sort of requirements, specification
or other such measuring criteria
•All performed by someone other than the
performer of the activity under examination. This
gives an audit a certain degree of independence
Common Audit Traits

Overview of the audit system
Conduct
audit
Analysis Report
Follow-up,
closure
Preparation

• Preparation:
• The activities to be performed should be planned and defined before they
happen
• Responsibilities must be set so that accountability and ownership of resulting
performance is established
• Define the need of the customer
• All this becomes the requirements base against which quality is measured.

• Performance:
• Should proceed as planned
• Records must be kept so that measurements can take place
• Those performing the tasks should be given the proper tools and training to
accomplish the job as specified

• Analysis:
• The success (or failure) of an activity needs to be measured against
some accepted standards

• Report:
• A tool for communicating the outcome of an audit and an evidence that the
audit took place
• Follow-up/ closure:
• Problems must be corrected and the process improved.
Regardless of the goods and services produced all management system
includes these activities.

Definitions:
• Auditor—a qualified person responsible for performing quality audit.
• Auditee- an organization to be audited.
• Client- a person or organization requesting the audit.

Classification of audits:
•Audits can be classified as either internal or
external, depending upon the relation ship between
the client, the auditor, and the auditee.
•Internal and external audits vary in scope and
depth. Many audit process such as decisions about
the audit team, the level of preparation for the
audit, etc) are influenced by whether the audit is
internal or external

Internal Audits:
• Performed on an organization by auditors who are employed or hired
by the organization
• Auditors must be independent of the area being audited (e.g. do not
report to the head of the area under review)
• Internal auditors may be part of a separate audit department

Internal Audits:
• Internal auditors can be borrowed from different departments within
the organization and brought together for the purpose of the audit
• An internal audit measures an organization’s strength and weakness
against business goals, internal procedure, and external standards
(GMP, FDA & HPFBI guidelines)

External Audits:
• Purpose:
An Audit of suppliers and sub-suppliers intended to determine
whether the supplier or sub-supplier will be delivering what it
promises.

Challenges That Auditors Face
1) Observation fear
• Auditors are forced to be politically correct and this fear
factor results in auditors modifying their approach to
minimize such risks and in the process perform audits that
are not effective.
2) Management intimidation:
• Who are you telling me that I am out of compliance?
• Are you crazy? That would cost millions
• If it is a problem, how come the FDA didn’t write it up the
last time they were here?

3) Fear of not raising an issue
• Not only can there be a fear of raising an issue, but also a fear of
not raising an issue.
“ Shoot the messenger syndrome”
• If an external audit or inspection yields observations,
management will ask, “why didn’t the audit function find this?”
• If the auditor cannot show that an issue was identified in an audit,
the auditor could become the focus of the company’s corrective
action instead of the non-conformance itself.

4)In a paranoid auditing environment “The auditors
out to get us” any observation can place an auditor
into a “pick your poison” situation i.e.
Don’t write the observation, and hope nobody else
ever uncovers the non-compliance situation and
that you knew about it
• OR - Hold your breath and write the observation,
but only after checking to see that your mortgage
has been paid in full

Challenges That Auditors Face :
• How do you get out the paranoia situation?
• 1) Obtain management commitment to find the truth
• 2) Explain to management the need to them to be receptive to
observations. It is for their own protection. FDA not only
prosecutes companies, but also individual executives.
• 3)Allocate funding for a qualified independent person to
referee questionable compliance observations.
• 4)The audit function is viewed with mistrust and fears as a
police operation or as “nitpickers” or some alienation (a “we”
vs. “them” approach)
• 5)These issues need to be resolved within the auditing
profession; in addition, management needs to be aware of
these issues and take steps to overcome or address them.

Auditor Qualification
• Auditors must be trained and qualified to perform their examination
and analyses.
• There are two levels of qualifications: auditors and lead auditors.
• Both levels should be supported by written qualification records
• Training is based on the roles they play in an audit

Audit Team Role and Responsibilities:
• A) Lead auditor:
• Organize and direct audit
• Report audit results
• Evaluates resulting corrective action
• Must have demonstrated ability to extract information, analyze that
information, and report the
• Result in a meaningful fashion
• Ability to communicate effectively both orally and in writing
• The ability to present complex issues to an often hostile audience
• The ability to receive information from others. This requires the ability to read
and listen effectively.
• Experience
• Formal training (commercial courses or in-house training program)

• B) Auditor:
• Participate in the audit process
• Typically technical specialists, new auditors-in- training
• Audit under the supervision of the lead auditor

•C) Escort:
• An escort usually fulfills the following roles:
• Serves as a guide for he auditor
• Expedites auditor requests for records, documents, etc.
• Makes personal introductions and facilitates clarification
as needed. And escort, for example, helps an auditor
rephrase a question by substituting terminology and
incorporating local Jargon.
• May confirm or deny the presence of a discrepancy or
non conformity before moving on to the next area to be
audited

•C) Escort:
• Ensures that the audit team complies with organizational
rules, indicating safety rules.
• Acts as a management observer and, during daily briefing
with management, provides an overview at the auditor’s
observations, finding, and conclusions.
• Overall, an escort assists and points the auditor in the
right direction. During the interviews, the escort should
be an observer and should respond to questions only
when asked by the auditor.

Lesson 02
•The topics covered in this lesson are as follows:
• A: Code of Ethics
• B: Audit preparation (Audit definition and Plan)
Identification of Authority (Internal and External)
Determination of Audit Purpose
Determination of Audit Type and Scope
Determination of Resources Required
Team Selection and Identification of Roles
Requirements Against Which to Audit

Code of Ethics
• Generally accepted standards of professional conduct.
• A formal code of ethics provides a benchmark against which an
auditee or a client can:
1) measure an auditor’s activities
2) establish an auditor’s independence
3) recognize potential conflicts of interest

Code of Ethics
• 1. Avoid conflict of interest
• - An auditor should examine and reconcile possible conflicts of interest
before an audit begins.
• - If one arises during the course of an audit, the auditor should alert
management and potentially withdraw from the audit.

Code of Ethics - How do we test for bias?
1.Do you have a relative or good friend in the auditee organization?
2.Are you personally involved with someone in the auditee organization?
3.Did you help develop and/or implement the quality system under review?
4.Have you been the sole auditor for the auditee organization a number of
times in the past?
5.Do you have a major financial investment in the auditee organization or
one of its major competitor?
If an auditor answer “yes” to any of the questions, the auditor’s results may
be biased.
If an auditor answers “no” to all the questions, the auditor may have an
acceptable level of independence

Code of Ethics
• 2. Maintain confidentiality
- Quality auditors are exposed to proprietary information
and/ or intellectual property.
- This information should not be shared with other parties
based on legal or ethical standards.

Code of Ethics - Confidentiality
i) Auditors may share audit experiences with one another
as a learning tool but they should not reveal auditee identity
and proprietary information during such discussion when
outside a given organization.
ii) Use confidentiality agreement- a legal document that
states that the auditor will not discover any proprietary
information gained during the course of audit.
Iii) Use flexible auditing techniques. e.g. review a logbook
with other client’s information. Ask the auditee to cover
confidential information.

Code of Ethics
3. Exercise due care- the auditors conclusions would
be similar to those of another auditor under the
same or similar circumstances.
4. Avoid gossip- within the auditing organization, and
auditor should refrain from speaking negatively
about the organization itself or previous auditee.
And auditor should avoid making negative comment
about other interviewees or other auditors during an
audit.

Code of Ethics
5. Follow ethical billing practices
- An auditor should bill fairly and accurately and never
accept payment for the same work form two sources.
6. Report illegal, unsafe, or unethical practices.
7.Diplomacy
- An auditor, should display patience, tact, and
consideration throughout the audit process

Code of Ethics
8. Independence
- An auditor should be free from bias and influences that
affect objectivity. Remember, one of the challenges that
auditors may face in some organization is fear of
observation.
9. Disclose evidence of wrongdoing
- Illegal, unethical, or unsafe activity discovered during the
course of the audit must be reported.
10. Manage audit record disclosure
- procedures for accessing archived audit records must be
in place.

Code of Ethics - Attorney-client privilege
• This means that an attorney cannot be examined regarding
communication made by the client to the attorney in the course of
professional duty. Auditors are ethically bound by this idea.
• Allows auditors to seek an alternate opinion when evaluating
information
Limitation of Attorney-client Privilege:
• Intentions of a client to commit a crime
• Information required to prevent a crime
• Information to rectify the consequences of a client’s criminal or
fraudulent act
• Information to defend against an accusation of wrongful conduct
• Information with the consent of the client

First and Third- Party audits
Common Classifications of Quality Audits
Quality Audits
Internal audits External audits
or First-party Audits (Self inspections) or Third-party audits for
Regulations /Recognition /Registration
NOTE: A second party performs third party audits

•Auditors are most often employees of the
organization being audited, but they must have no
vested interest in the audit results.
•An organization may:
• Employ full-time auditors
• Train employees from other areas
• Hire a consultant

•Benefits of first party audit:
•Provide management with assurance that
adequate procedures are in place and are being
implemented.
•Identify root cause
•Take corrective action to solve existing problems
•Prevent further issues from developing.

• An External Audit (3rd party) is performed on a supplier by a
customer or by a contracted consulting organization on
behalf of a customer. Client and the auditee either have an
existing relationship or anticipate establishing one.
• A Supplier Survey of a potential supplier usually occurs to
evaluate the capabilities of potential suppliers and
determine if the proper systems are in place to deliver a
quality product before a contract is awarded.
• Establishing the scope involves but not limited to an initial
review of the following: supplier annual report, information
from colleagues in other companies

• An organization may perform a quality survey on a potential
supplier with regard to the following:
• facilities,
• resources,
• technical capabilities,
• personnel,
• past performance,
• entire quality system.
• Once a contract is signed, a Third-party quality audit can
validate and verify contractual relationships and/or
document ongoing requirements of the contract.

•Benefits of a Third-party audit:
•Help to eliminate the shipping of nonconformities
•Help both the supplier and the customer reduce
waste and cost
•Build confidence between both partners
•Produce a better output by verifying appropriate
controls for input.

•A further benefit of third-party audit is
confirming on-going compliance. It answers
the question: does the auditee meet
requirements of a given standard, law, code
or regulation?
•Third-party audits do not usually measure or
assess effectiveness merely compliance with
agreed to standards

•Compliance Third-party audits may be performed
for the following reasons:
• Regulation- audits performed in regulated industries
provide assurance of safety to the public
• Recognition- may apply for awards such as the Deming
Prize - the Canadian award for excellence
• Registration- the registration process is when an
independent third-party organization audits a supplier
organization against the requirements of a standard for
which the organization wants to be registered.
Maintenance of the registration normally requires
continuing surveillance.

•Benefits of third-party audits- in addition to
ensuring that the auditee meets regulatory
requirements, achieve recognition, or establishes
registration, third-party audits may provide the
following benefits for an auditee:
• Added assurance to customers
• Increase competitiveness
• Reduce market barriers
• Decrease costs of multi-customer audits.

Audit Preparation
A: Audit Definition and Plan
• Planning and preparation are critical to the success of a
quality audit.
• Proper preparation gets a quality audit off to a good
start.
• A quality audit is defined by an audit plan:
• An audit plan is an approved document (by client – the
one wanting the audit) prepared by the lead auditor, and
communicated with the auditor and auditee.
• Audit plan should be flexible in order to permit changes
in emphasis based on information gathered during the
audit and to permit efficient use of resources.

Audit Preparation
• Allows the auditor and the auditee to maximize time during
the quality audit-
• Allows the auditor to make the most efficient use of time
during the audit process.
• Minimizes the time an auditee needs to take away from the
daily work routine to devote to the audit.
• Establishes the credibility of the auditor and the auditee
• An auditor being prepared for an audit shows an auditee that
the auditor means business. Likewise, thorough preparation by
the auditee shows the auditor that the organization
understands its requirements and takes such obligations
seriously.

Audit Preparation
•How much time is required to prepare for an
audit?
•The specific amount of planning and preparation
required is greatly influenced by:
• Number of auditors and auditees
• First or Third party – (Third-party is more resource demanding)

Audit Preparation
• General steps for quality audit preparation (source: Quality audits for improved
performance by Dennis R. Arter):
1. Define the purpose of the audit
2.Define the scope of the audit
3. Determine the audit resources to be used
4. Identify the authority of the audit
5. Identify the performance standards to be used
6. Develop a technical understanding of the processes to be used
7. Contact those to be audited
8. Perform an initial evaluation of lower-tire documents to higher
level requirements
9. Develop written checklists of the data needs
• Remember checklists guide action not control it

Audit Preparation
• An Audit plan should include: (source: the quality audit hand book, edited by J. R. Russell
2nd Edition Feb. 2000)
• tentative date and place of the audit
• auditee and organizational units to be audited
• names of audit team members
• scope and purpose of the audit
• standards being audited against
• overall schedule of the audit
• expected (maximum) duration of the audit
• expected date of issue of the audit report
• Additional information on the audit plan:
• -language of the audit
• -Confidentiality requirement audit report distribution and the expected date
of issue.

Audit Preparation
• AUDIT PURPOSE (the “why” of the audit)
• First party audit-
• provides management with the assurance that organizational goals and strategies
are being met and that the audited area is in compliance with identified standards.
• Third party audit-
• to ensure that quality systems and proper capabilities exist. Provides the customer with
confidence in the quality of the products or services being delivered.
and
• Third-Party audits to determine the auditee’s compliance with agreed upon criteria. In a
regulated environment the audit may be a government requirement.
• To determine the purpose of the audit ask yourself:” what do your customers want
to achieve with the audit?”.
• Once you have determined the purpose of the audit, write it down. Normally, the
purpose statement is one sentence long.

Audit Preparation
• AUDIT SCOPE (the what of the audit)
• Your next step in the preparation phase is to establish the scope of the audit.
The scope established a perimeter around the area to be audited and identifies
the items, groups, and activities to be examined.
• Maintaining the audit scope may be challenging. During the course of audit,
additional areas requiring examination can arise that are outside of the original
scope. The team must ask itself whether the concern is important enough to
pursue immediately, or whether it can wait for a separate examination at a later
date.
• Generally, it is best to stick to the original scope, regardless of what develops.
The auditors are perceived as more credible when they stick to the rules.
• If serious issues arise Scope may need to be modified to ensure safety and
compliance

Audit Preparation
•Consideration in the audit team selection:
• Experience of the auditors- auditors must be
knowledgeable of the process to be audited. They should
know something about the product line and who the
customers are
• Audit type, objective, and scope
• Budgetary constraints- only so many vendors can be
examined when travel funds are limited.
• Time frame in which the audit is to be completed.
• Amount and complexity of work to be performed

Audit Preparation
•Consideration in the audit team selection:
• Ensuring the audit team is objective and unbiased
• Size of the audit team
• People who have the necessary skills, knowledge, and
personal attributes to get the assignment done.
• Use of the specialists- you cannot be the jack of all trades.
• The lead auditor expertise and experience and/ or
proficiency to perform required activities.

Audit Preparation
• AUDIT AUTHORITY
• The next step in the preparation phase is to determine the authority
for the audit.
• You need to make sure that you are allowed and welcome into the
plant.

Audit Preparation
• PERFORMANCE STANDARDS
• Standards are the norms or criteria against which the performance
of an activity is measured. These come from variety of sources:
• Conformance standards- are used for external quality assurance
to provide confidence to the customer that the company’s quality
system will provide a satisfactory product of service
• Guidance standards- used for internal quality assurance to
provide confidence to management that the intended quality is
being achieved.
• Regulatory standards- come from regulatory agencies and are
enforceable by law.
• Consensus standards- reflects agreement among professionals
regarding good practices

LESSON 03 – Audit Preparation (Continued)
•The key topics covered in this lesson are as
follows:
•Standards and quality audit
•Audit design
• Strategies for collecting data
• Audit data collection plan
• Audit sampling plan
• Audit logistics planning

Standards and Quality Audit
• There are four levels of performance standards:
• Policy Standards. Policy documents include corporate policy
statements, international and national quality system
standards, regulatory standards, and business sector standards.
• Transition Documents. Transition documents are often called
manuals. They might exist for different levels in an
organization-sections, departments, or divisions.
• Procedure documents. Step-by-step requirements are included
in procedure documents. Such documents must be clear,
correct and effective since they provide direction to a trained
individual on performance of a job.
• Detailed support documents. Detail documents may include
drawings, work instructions, purchase orders, product
specifications and inspection plans. They explain specific tasks.

• Each of the four levels of documents provides direction on
an activity to be accomplished
• Each level of documents provides a written description of an
action to be accomplished
• Each of the performance levels represents the promises
made by the audited party
• Auditors use these documented performance standards to
audit against.
• The larger system audits generally examine the top part of
the pyramid (level 1-3); the smaller process and product
audits examine the bottom of the pyramid (levels 2-4)
• Auditors should examine more than one performance level

•Reference documents to audit against also include:
• Contracts
• Specifications
• Policies
• Quality award criteria
Note: in the overall model of auditing, the four
levels of performance standards represent controls

• 1) Contracts-
• oftentimes a contract contains the requirements for an audit.
An audit verifies that the supplier is meeting identified
customer requirements
• 2) Specifications-
• a grouping of specific parameters that are required to ensure
the success of a product to performs as design. Specifications
are used in:
• Product audits, to compare against specified criteria
• Process audits as reference, to confirm existence and possibly
to spot check
• System audits as a reference, to confirm existence and possibly
to spot check.

• 3) Policies- provide an organization with basic control, vision,
and guidance documents. Policies become the basis for an
organization’s quality system.
• 4) Quality award criteria- organizations that are trying to
achieve continuous improvement and reach world-class
levels of quality do not focus solely on compliance with
certain standards. Organizations are, instead, assessing
their operations against specific quality awards criteria on
how system needs to be structured to achieve total
customer satisfactions.

Specifications in Auditing
• There are six types of specifications involved in auditing
• Product specifications. It defines what is required for a product
to perform as expected by the consumer.
• Process specifications. Defines parameters of the
manufacturing process that must be controlled in order to
produce a product.
• Analytical specifications. Defines analytical methodologies
used to measure a required level of accuracy.
• Raw material specifications. Defines what is acceptable as raw
material entering a manufacturing process.
• Quality management specifications. Defines management
practices governing products produced.
• Packaging Material Specifications. Define how quality is
protected during storage and transport.

AUDIT DESIGN
• Strategies in collecting audit data:
• 1.Tracing techniques
• 2. Discovery method
• 3. Department method
• 4. Element method

AUDIT DESIGN
• 1) Tracing techniques:
• Follows the chronological process of something.
• Tracing is a common and effective method of collecting objective data during
an audit.
• Often time a flowchart is used as a road map for tracing activities. Auditor
may use forward or backward tracing technique.

AUDIT DESIGN
• A: Forward tracing techniques
• An auditor starts at the beginning or middle of a process and
traces it forward. E.g. customer complaint. The auditor would
follow the path of a consumer complaint through a complaint-
handling system form start to end.
• B: Backward tracing technique
• An auditor starts at the middle or end of a process and traces it
backward.
• Tracing techniques:
• are commonly used in Third-party audit
• are logical and allow preplanning
• but can be inflexible are dependent upon people’s availability
depend on accuracy of how the records were generated

AUDIT DESIGN
•2) Discovery method:
•An auditor examines actions that are of interest or
in whatever order he or she chooses, literally in a
random manner. (i.e. random auditing)
•It minimizes opportunity for cover-ups because the
auditee cannot anticipate what the auditor intends
to look at
•This method tends to investigate what is actually
taking place at the time of the audit and, therefore,
reflects current procedures.

AUDIT DESIGN
•2) Discovery method:
• Requires an experienced auditor
• An auditor can potentially miss examining an important
action. Therefore, cannot draw conclusion based on this
type of data alone.
• Used in third party audits and the favoured method by
regulatory inspectors
• Is very flexible
• Identifies symptoms and trends
• e.g. walk around a production floor looking for data in an
unbiased way.

AUDIT DESIGN
• 3) Department method
• Looks at the operations of one department
• Beneficial when the entity being audited is small
• Used for first-party (self inspection) audits
• May lose sight of “big picture”
• Example: audit the purchasing department

AUDIT DESIGN
• 4) Element method:
• Examines one element to determine how it affects the entire system.
• Example: How the policy has been communicated, implemented, and
understood in departments through the organization and whether objectives
and targets are being met. E.g. how a corrective action flows through an
organization.
• Can become cumbersome.

Data Collection Planning
• What is data?
• Data can be defined as observations of fact. Data can be
qualitative or quantitative:
• Qualitative data- relate to the nature of a single
characteristic observations, regardless of frequency.
• E.g. an observation of an employee not wearing safety glasses
• Quantitative data-relates to a count or measurement
• e.g. the number of complaints a complaint-handling system
receives in a specified period

• Original Data – “the first time recorded, however recorded,
on whatever recorded”
• Concurrent data – recorded at the time of the event
• Prospective data – recorded before the event
• Retrospective data – recorded from memory (historical)

•What is a data collection plan?
• Things that you want to touch and see during the audit.
These “things” creates objective evidence and data. For
example: records, procedures, completed document pack,
charts, graphs, equipments, timing of things
completed,…..

• Rational for a data collection plan
• Similar to an audit strategy, a data collection plan contributes to
the satisfactory completion of an audit.
• A data collection plan helps to ensure that necessary objective
evidence is collected to verify that the auditee has adequate
controls to meet the requirements of the audit.
• Auditors typically include the things they want to verify and
see in a checklist. To be effective, an auditor must:
• Understand different data collection methods
• Understand basic statistical methods
• Be aware of the functions and limitations of different auditing
tool

•Sampling Plan
• Sampling for an audit is the process of selecting a number
of items, units, or people in such a way that they
represent the larger group from which they are selected.
• The purpose of sampling is to gain information concerning
an entire population

•The auditor should quantify the number of
instruments or records to be evaluated and from
what areas) they are to be obtained. Reviewing
instruments or documents from variety of areas,
although not statistically sound, will identify if a
problem is systemic or focused in certain area.
• In quality audits, for example, sampling may call for one
out of every seven files to be examined or one out of
every twelve sales phone calls to be monitored or
recorded.

•During the sampling process, careful attention must
be given to the following:
• Sample size
• Sample amount
• Location of samples
• Sampling error
•Methods of taking samples:
• from the total population or universe
• from subgroups called strata

• To infer statistical significance from any sample taken, the following
must both be true:
• 1) The population from which the sample is drawn must be
homogeneous.
• Bad parts must not be secreted in one part of the population
• One load from one production setup must not be compared to another load
from another production setup.
• 2) The sample must be random *
• All items have the same chance of being included; there is no plan to skip items
or focus on a few particular items
• Randomization methods can be as simple as pulling names out of a
hat or can be more precise, such as taking every name at a particular
interval.
* Random sampling must be from a population that is homogeneous
mixed populations need skewed sampling.

•Generally speaking, during a quality audit, the area
sampled should be selected because of its
importance.
• Auditors should avoid sampling areas that are
generally insignificant and should concentrate on
areas where the potential impact or severity of a
failure is greater.

Important Considerations in Data
Gathering
• 1) Avoid sampling bias
Be sure to consider all transactions that have been removed from the
file, from storage, or from view
Do not preview transactions for the purpose of selecting only those that
appear to be unacceptable or acceptable
Do not be tempted to ignore those transactions that are inconvenient or
difficult to evaluate
Resist any suggestion the escorts might make to suggest or guide sample
selection so as to limit randomness is the sample selection
• 2) Understand the limitation of statistically sound sampling
Statistically sound sampling is usually not practical during a process or
system quality audit.
Know if the data provided represents the population appropriately
Quantify the number of examples evaluated and the areas they were
taken from

Gathering
3) Understand the risks inherent in sampling
 Because evaluations made during a quality audit may be based on a
relatively small sample, the reality is that a larger sampling risk
exists.
 The auditors must understand the purpose and risks of sampling
and know what is considered appropriate for specific uses.
4) Sampling for an audit is not always statistically valid but it has to be
well-selected and representative in order to identify if a problem is
systemic of focused in certain area.

Gathering
5) Lead auditor must select the appropriate sampling plan, keeping in
mind that certain techniques are more appropriate for certain
situations and that each of the techniques do not give the same level
of assurance as to how representative the sample is of the
population.
6) Formal statistical sampling might be desirable in product audits for
compliance issue or for process audits if the results are to be used to
change the process design.

Logistics Planning
• Why is it important?
• Auditing can be stressful for both the auditor and the auditee. A
preparation phase that is properly planned and communicated
ultimately lessens the stress created by the audit and provides a
smoother transition to the on-site performance. Logistics planning is an
important part of ensuring such a smooth transition.
• During the planning phase, the lead auditor needs to think through each
step of the audit and answer the question: what will be required for the
audit team to get the job done?

Logistics Planning
• The lead auditor needs to consider items such as but not necessarily limited
to:
Time allowance for auditor and auditee meetings
Facilities for the safe storage of any confidential records
Access to telephone to support team communication and
communication between the lead auditor and the auditee
Distance between areas to be visited. Complexity of getting the audit
team together if great distances are involved.
Travel arrangements
Necessary equipment to take to the auditee site 9e.g. notebook
computers, cellular phone)
Expense and economics of travel arrangement.

LESSON 04 - Audit Preparation (cont’d)
• The key topics covered in this lesson are as follows:
• Document Review and Preparation
 Describe audit-related document review
 Discuss how an auditor can use an auditee’s history
 Describe how to prepare audit checklists
• Communication and distribution of audit plan
 Describe how the audit plan is communicated
 Discuss who receives the audit plan

Document review and preparation
•Audit related document review
• Reviewing audit documents before the audit helps the
auditor prepare working papers and determine where to
concentrate effort during the performance stage.
• In order to adequately assess the compliance of a
product, process, or system against the specified
requirement, it is important that the auditor review the
quality – related document and policies.
• Quality documentation can be required in advance of the
audit by contacting the auditee.

• Audit related document review
• If after reviewing the quality documentation, the auditor
determines that the auditee’s quality system is not adequate
for meeting the audit requirement further resources should
not be expected on the audit until such concerns is resolved.
• What documents to review?
• Specifically, documentation must include all the information
necessary to show the auditee’s intent of meeting higher-level
performance standards or specifications

• A variety of documents are appropriate for an auditor to review,
such as:
• Quality manuals ** * Available from Third Party
• *Procedures ** Generally only kept on-site and
not sent to
• *Detail instructions
• Blueprints and drawings **
• *Specifications
• *Certificate-of- compliance forms
• Preventative maintenance plans **
• Quality inspection plans **
• *Purchase order forms
• Site Reference File **
• Normally the information contained in all the documents must be current. It
must also be accurate and appropriate.

•Forms of documentation:
• Documentation can be in any format that can be
recorded, retrieved, and reviewed.
• This includes but is not limited to:
• Production records as hard copy
• Machine printouts
• Charts & graphs
• Electronic records
• Certificates of Analysis
• Specifications
• Investigation reports

• Who reviews the documentation?
• The lead auditor, of course, must examine all of the quality-related
documentation.
• If the audit involves a team, the lead auditor needs to decide what specific
information each team member should review as well.
• Auditee’s Performance History Review
• In addition to quality related documentation, it is important to know the
history of an auditee and how successful implementation of quality systems
has been to date.
• An auditor can gain such knowledge by reviewing past audit records as well
as product/ service quality data. (Applies to Internal and External Audits)
• The auditors should also note any strength that was identified. An objective of
the current audit should be to verify that these strengths have been sustained.
The auditors should also note any strength that was identified. An objective of
the current audit should be to verify that these strengths have been sustained.

• Product/service quality data:
• Reviewing data that are indicators of the product’s or service’s
fitness for use can be gained from review of items such as:
• Reports of any nonconformities – follow plans should be
considered
• Problem histories – adequacy and implementation to be verified
• Inspection and field reports – if available the issues raised need
follow-up
• Product performance records and warranty claims. – Complaints,
ADR’s and the Annual Product Review
• Quality documents can be used to develop checklists and data
collection plan

•Preparation of Audit Checklists
•Checklists are developed during the planning phase
of an audit (after review of quality-related
documentation). But the benefits that checklists
yield carry through the duration of the audit and
beyond.
•Checklists are very important for less experienced
auditors or support staff. They ensure important
issues are not missed

Checklists are developed during the
planning phase of an audit (after review of
quality-related documentation). But the
benefits that checklists yield carry through
the duration of the audit and beyond.
• Checklists help to ensure that:
1. essential items will not be forgotten
2. objective evidence will be collected in a
systematic manner during the audit

• Checklists are very important for less experienced auditors
or support staff. They ensure important issues are not
missed
• Checklists are the primary tool for giving order to quality
audits. As auditors, you will be expected to examine all of
the selected control areas identified from the various
performance standards chosen for your audit.
• Additionally, a method is needed for organizing all the
documents and working papers which together will form the
final records of audit.
• A checklist can meet both of these needs.

• Checklist purpose:
• Checklists are used to gather data and to guide audit team members to
ensure that the scope of the audit is covered.
• A checklist typically includes places to answer questions and/or areas for
taking notes.

•Checklist benefits:
• Objective evidence that the audit was performed
• The essence for the closing meeting and audit report
• Documentation that all appropriate aspects of the quality
program were verified.
• Order and organization to the quality audit process
• An information base for planning future quality audits

• Checklists must be tailored to the audit.
• The specific format and construction of a checklist is influenced by:
• the scope of the audit and
• the preference of the auditor (ISO Guidelines for Checklists are a good
reference)

• Content of a checklist:
• A) Yes/No question that references a specific requirement
Remember to provide for NA
• B) Open ended interview questions that test the limits of a
procedure
• C) statements providing additional instructions for the auditor
• D) space for recording the result of examination including
those people the auditor spoke with . Actual data is best if in a
bound note book referenced to the checklist
• E) a plan for collecting a specific evidence needed to answer
certain checklist questions. What do you wish to look at? How
many items do you wish to sample? These are the type of
questions that should be addressed in the checklist.

• Bottom line, all checklists should be supported by requirements:
• In general, with checklists used for supplier surveys, department audits, or special
processes (e.g. calibration), the requirements should be related to each question
• For specially tailored audits, the auditor should identify the requirement for each
question in the audit form.
• Before an audit begins, the auditor should review all checklist questions to ensure an
understanding of how each question relates to the requirement. Checklists that are
supported by requirements help the auditor fully prepare foe an audit and also
provide the auditor with confidence should any of the audit results be challenged.
• Checklists should allow an auditor to reconstruct an interview or a record reviewed.

• Checklist questions may be developed by the lead auditor, or they
may involve a collaborative effort.
• REMEMBER: Checklists are a valuable tool for quality audits. But they
are not intended to be limiting.

Document review and preparation-
Checklist Review
•When finished with the checklist development,
each auditor should submit his or her portion to
peer review.
• This serves as a check of thoroughness, proper
logic construction, and absence of bias.
• Any qualified reviewer will do. This may be lead
auditor, another team member, or the manager of
the auditing section.
• The purpose of this review should not be to
approve your checklist, but rather, subject it to
critical examination of content.

• Issues for Standardized checklists:
• They fail to identify some special features of a particular
control system that may be crucial to success.
• They may address only a few of the performance criteria
• They allow audit to proceed without really adequate
preparation and thought
• The use of standard checklist by themselves is not
recommended
• They can provide the auditors with a bank of potential
questions upon which to draw. This approach is encouraged.

Communication of the audit plan
•The audit plan should be approved by the client and
communicated to the auditors and the auditee.
•If the auditee objects to any provisions in the audit
plan, such objections should immediately be made
known to the lead auditor. They should be resolved
between the lead auditor and the auditee and, if
necessary, the client before executing the audit.

•Notification of the audit plan
• Once the audit plan is finalized, written conformation of
the impending audit date should be sent to the auditee.
• Auditee and members of the audit team receive copies of
the audit plan
• Written notification to the auditee should be addressed
to the senior person in charge of the area to be audited:
• For Internal audits- area manager (notification is done
through memo)
• For External audits- plant manager or president
(notification is through a formal letter)

• Who provides notification?
• Generally, the written notification should be signed by the client, not the
auditor.
• Having the client sign the letter gives the client ownership for the quality
audit and keeps the client actively involved in the audit process.
• In external audits with contractual matters, the responsible buyer or
purchasing agent may sign the notification.

•When notification is provided?
• Provide notification at least 30-days before on-site
fieldwork begins.
• It forces the auditor to be better prepared for the on-site
work and it allows the auditee enough time to prepare.
NOTE: This gives the auditee time to obscure your review
so Trust becomes a factor in how you give notice.
•Once communication and notification have been
provided by the auditor, the performance phase of
a quality audit can start.

Lesson 5 – Conducting the Audit
• TOPICS
• Audit Management
• How to effectively manage the audit team through team meetings
• Describe when and how the auditor must brief the auditee on audit status
• Review how any changes in the audit plan should be handled
• Opening Meeting
• Presentation and review of the audit plan
• Handling of auditee concerns
• Data Collection
• Examining and verifying documents and records
• Elements of quality interviews
• Elements of quality audit physical examination
• How to effectively observe work activities.

Audit Management
•The Importance of Team Meetings
• Facilitates open communication among team members
• Provide audit team members with the opportunities to
compare notes
• They help to clarify the information gathered to date
• They identify any issues and/or problems that need to be
addressed

Audit Management
•Frequency and timing of audit team meetings:
• Audit professionals recommend daily meetings in team
audit situations.
• Team meetings are sometimes called daily debriefing.
• Timing of an audit team meeting is a team preference.
• The timing of the meeting is less important than the
topics discussed.
• Daily team meetings, while generally informal, should be
held in a quite area of the auditee’s facility or a
conference room so that team can openly discuss audit
business.

Audit Management
• Purpose of audit team meetings
• Daily team meetings typically involve team members in the following
activities:
a) Sharing of facts formulation of tentative conclusions
b) Discussion of issues and problems
c) Discussion of necessary adjustments to the plan for the next day’s
activities
d) Ongoing development of the audit report

Audit Management – Team Meetings
• Sharing and discussing data gathered during an audit on a daily basis
enhances the audit process.
• Team members can corroborate facts and possible areas requiring
further investigation. Members can share perceptions about data
collection methods that worked well and activities that presented
challenges.
• Team strengths and areas for improvement can be discussed. In addition
data discussed at daily team meetings lead to tentative conclusions for
the audit report.
• Facts collected during the day should be organized and sorted so that
they support the tentative conclusion, both positive and negative.
• When evidence is insufficient or missing in support of these conclusions,
plans should be made to fill in the gap.

Audit Management – Team meetings
• Overall, daily team meetings answer questions such as but not
limited to:
Are data collected sufficient to reach conclusions?
Is there a need for data to be collected from additional sites/
sources?
Are additional interviews necessary?
Are additional checklist questions necessary?
Should additional records be reviewed?
Are there administrative issues that need to be addressed?
Does the audit seem to be on track to accomplishing its
objectives?

• In some instances, the daily team meeting may identify areas that
need less attention than originally thought.
• Based upon the audit team meetings, the lead auditor may adjust
the auditor’s work assignments.
• If any minor changes are necessary, the lead auditor should at
least consult with the auditee management representative to
ensure that the change will not disrupt the auditee’s operation
and that appropriate escorts and/or personnel will be available.
• The client must be involved in any changes that impact the audit
purpose and scope.

• Audit team communication between meetings
• Between daily meetings, audit team members are “on their
own” in the event of a problem or uncertainty about how to
proceed, team members should contact the lead auditor for
clarification.
• How audit team meetings contribute to the audit process
• Daily team meetings keep the audit process on track and help
to ensure that sufficient objective evidence is collected to fulfill
the purpose of the audit.

• In summary, daily team meetings help the audit team:
Identify possible areas for findings
Determine how individual findings relate to other
areas
Plan for the next day of the audit
Prepare for daily meetings with the auditee

Audit Management- Communication of the
audit status
•Importance of auditee briefings
• If the goal of the improved performance (of auditee) is to be attained, it
is important that there be NO SURPRISES on all levels.
• It is a good practice for the auditor to discuss areas during the audit with
the escort in an effort to keep the communication lines open
• Daily meetings help to ensure that the auditor and the auditee are in
sync throughout the audit process.

audit status
•Frequency and timing of auditee briefings
• A short meeting at the beginning or end of each day
• KEEP IT BRIEF: Approximately five to ten minutes
•Participants in auditee briefings.
1) The designated auditee escort should attend. If,
however, the auditee does not wish a daily meeting, it is
incumbent upon the auditor to inform the auditee escort
or area supervisor of findings in an area.
2) Audit team members but the lead auditor remains in
charge

audit status
• If a daily meeting is held, the lead auditor typically will explain:
• What has been examined
• What will be looked t next
• Where potential problems exist

audit status
• Purpose of auditee briefings
The auditor can communicate potential findings up the
chain of command so that auditee management is made
aware of problems before the closing meeting
The auditee can confirm or deny that a problem exists. If
an auditee denies a problem, they can offer more data to
the auditor to refute the findings
Maintain openness and communication while the audit is in
progress
Provide the auditee with the opportunity to close a
legitimate finding before the end of audit.

audit status
• If an auditor finding is wrong because of incorrect or
insufficient information, additional facts from the auditee
about the item can prevent it from appearing in the final
report.
• If the auditee truly has a problem, additional facts and/or
investigation by the auditor can help to reinforce the finding
• The auditor may also discuss checklist areas completed and to be
examined (or revised) the next day at a daily briefing.

audit status
• Audit Plan Changes (e.g. schedule, Priorities)
• The audit plan should be designed to be flexible in order to
permit changes in emphasis based on information gathered
during the audit and to permit effective use of resources.
• The on-site management is the responsibility of the lead
auditor, rarely does the client need to be consulted.
• In extreme circumstances, if the objectives appear to be
unobtainable, the audit may be stopped with concurrence from
the client.

audit status
• Audit Plan Changes (e.g. schedule, Priorities)
• Whether a change to the audit plan is minor or significant, it
is the lead auditor’s responsibility to communicate the
nature of the changes as well as any ramifications to the
client and the auditee.
• If, for example, the number of quality documents reviewed is
increased or decreased, the audit scope, schedule ,and resources
required may be impacted.
• In rare cases, the lead auditor may discontinue the audit:
• if the auditee is not prepared or
• if the audit schedule is inappropriate for some reason and the auditor
has not been advised.
• (e.g. a major emergency has occurred or an employee has gone on
vacation and these is no one present to facilitate the audit).

Audit Management- Opening meeting
• The Opening Meeting
• Conducted before the start of on-site auditing activities, often shortly after
the auditor arrives at the audit site.
• Presents and reviews the audit plan
• Formality of this meeting depends on the audit scope but it should not
exceed more than one hour.

Audit Management- Opening Meeting
• Who should attend the opening meeting?
All members of the audit team
Representative(s) of the auditee organization usually from a senior
management or the manager of the area to be audited
The appointed escorts from the auditee organization.
• (The level of people representing the auditee organization at the opening
meeting may be an indication of the level of interest management has in the
audit process)
• Clients rarely attend often the lead auditor acts on behalf of the client.
• The lead auditor should take charge of the opening meeting.
• This action puts the lead auditor in a position to politely cut off any time-
wasting tactics that the auditee may introduce.
• Make a record of the attendees.

• Purpose of opening meeting:
Introduce the members of the audit team to the auditee management
Review the scope and objectives of the audit
Provide a short summary of the methods and procedures to be used to
conduct the audit
Establish the official communication links between the audit team and the
auditee.
Confirm that the resources and facilities needed by the audit team are
available
Confirm the time and date for the closing meeting of the audit team an the
auditee’s senior management
Clarify any unclear details of the audit plan
Gain agreement on when the final report will be issued

• Topics for an opening meeting
• Introduction
• Thanks
• Scope
• Purpose
• Methods and techniques
• Detailed audit schedule
• Logistics
• Confirmation of the need for daily meetings and tentative exit
meeting

• Introduction:
• Time allocated for introduction ensures that the lead auditor is
identified and that all parties are introduced to each other
• Thanks:
• Whoever made the arrangements for or coordinated the audit
should be acknowledged
• Scope:
• This portion of the meeting reaffirms the area or groups to be
audited. Any confidentiality or accessibility limitations are
addressed.
• Purpose:
• The purpose listed on the audit plan is reviewed.

• Methods and techniques:
• The data collection plan should be described (e.g. records,
observation, and interviews to be completed)
• Detailed audit schedule:
• - Availability of personnel should be confirmed
• - Any scheduling conflicts with the auditee should be resolved
• Logistics:
• Audit team room, copy machine, rest room, lunchroom, the
distance between areas to be audited, tours.
• Confirmation of the exit meeting
• The date, time, and attendees of the exit meeting should be
verified.

• Roles and responsibilities at an opening meeting:
• Lead auditor:
Introduces team and presents credentials
Restates audit purpose and scope
Asks clarification question
Facilitates discussion
May present audit checklist
Presents detailed schedule
• Auditee:
Specifies who represents auditee
Confirms auditor access and areas to be examined
Confirms facilitates available for auditor use
Specifies support personnel
Review safety and regulatory requirements
Review confidentiality issue

• Importance of an opening meeting:
• Provides a forum for all parties to review expectations and
clarify areas that are unclear or are of concern
• Rapport building for all parties
• Verification and confirmation of information (e.g. audit scope,
purpose, schedule)

• Discussion of Auditee Concerns
• It is not unusual for an auditee to have concerns about an
audit.
• Auditees may be unfamiliar with the auditor’s mode of operation.
• The overly conscientious auditee may be obsessively concerned
about potential deficiencies.
• Alleviating potential auditee fears and/or providing clarification
contribute to two of the primary purposes of the opening
meeting
• -creating a positive climate for the audit and
• building rapport between the auditor and the client and/or auditee.
• Covering any other general questions either the auditor or the
auditee may have also furthers opening the dialogue and free
exchange of information between all parties.

Audit Management- DATA COLLECTION
DATA COLLECTION METHODS
Interview
Physical examination
Observe work activities
• All audit data must be
• Reliable (consistent, with relative absence of error)
• Valid (an accurate measure of what it is intended to
measure)

• Examining and verifying documents and records
• Three important reasons why an auditor needs to verify
documents and records:
1)Ensure accuracy- Verification checks that information is correct
and complete
2)Demonstrates understanding- verification illustrates that the
person completing a record understands what to record
3)Eliminates fraud- by verifying documents and records, an auditor
discern that documentation is truthful and that there has not
been any intent (or unintentional action) to deceive others or
misrepresents facts.

2. Application of Sampling techniques- only
a representative sample of the population is
observed.
1. For example, when an operating procedure lists a given task, the auditor
goes to where the task is being performed and asks questions of
responsible personnel such as “did you do this?” and “How do you
perform this task?”
3. Physical examination

• Interview
• Every auditor ultimately develops a personal style and technique of interview questioning.
Experience provides valuable lessons about what works well and pitfalls to avoid. Yet there
are some basic guidelines that experienced audit professionals recommend for an audit
interview protocol:
• Source: adopted from the Practice an Process of auditing by Frank X. Brown
1)introduced self, establish rapport, and put the person at ease
2)explain the purpose of the interview and approximately how long it will take
3)Find out that the person is doing. Ask open-ended questions and, if appropriate, “show-
me” questions
4)Analyze what the person is doing. Confirm your understanding of the information with the
requirements
5)make tentative conclusions. Provide feedback to the interviewee as appropriate.
6)Thank the person and explain the next step

• Methods of verifying documents and records
• The most common methods auditors use to verify
documents and records are:
1. Application of tracing techniques-
When employing tracing techniques, an auditor reads the procedure to be
performed and then observes others performing the procedure.
If the actions do not reflect the procedure, report the discrepancy.
Inability to perform a procedure is symptomatic of one of two problems:
 the procedure is inaccurate- i.e. Incorrectly written or the process has
‘changed’ without the document being updated
 the procedure is being performed improperly- this indicates inaccurate or
insufficient employee training

• Interviews
• Every auditor ultimately develops a personal style and technique
of interview questioning.
• Experience provides valuable lessons about what works well and
pitfalls to avoid.

•There are some basic guidelines that are
recommended for an audit interview technique:
1)Introduced your self and put the person at ease
2)Explain the purpose of the interview and approximately how
long it will take
3)Find out that the person is doing. Ask open-ended questions
and, if appropriate, “show-me” questions
4)Analyze what the person is doing. Confirm your
understanding of the information with the requirements
5)Make tentative conclusions. Provide feedback to the
interviewee as appropriate.
6)Thank the person – ‘appreciate their assistance’

Audit Management- DATA COLLECTION - Interviewing
Do’s & Don'ts:
• Do’s
• Do ask open-ended questions-the basic who, what, when, where and how questions:
• “what do you do?”
• “how are you involve in-------?”
• “how do you know how to do----?”
• Do ask for work processes to be explained,
• “Could you walk me through the inspection procedure?”
• *Do provide positive reinforcement as appropriate:
• “you really have this process down cold”
• “your area seems very well organized”
• *Be careful not to mislead them
• Do use body language to encourage the discussion:
• Tilted head to one side
• Raised eyebrows
• Half smile
• Hands on face
• ( don’t overdo body language – its not a sign f they are answering right)

Audit Management- DATA COLLECTION- Interviewing
Do’s & Don'ts:
• Interviewing Don’ts
• Don’t always ask yes-no question unless it is a specific strategy.
• Yes-no questions can lead to yes-no answers. The yes/ no approach is a lead in to
more probing questions.
• Don’t use obvious rhetorical questions:
• If a situation is incorrect or non-compliant you may use this technique to focus
awareness/acceptance of the problem.
• Don’t interject presumptions that may or may not be true:
• This type of action may lead to arguments. You are striving for dialogue
• Don’t ask leading questions: ( Generally an interrogation technique not an auditing one)
• leading questions sometimes help the interviewee focus on the topic.
• Once the communication link is established, return to the normal question
protocol)
• Don’t ask hypothetical questions:
• This is reserved for only those issues where preventative processes are not otherwise
reviewable eg Recall process knowledge

Audit Management- DATA COLLECTION- Interviewing
• Notes are taken throughout all interviews.
• Record questions, responses and who gave the response
• After the interview is over, the take a few minutes to ensure that
the notes are clear so they will be meaningful when they reviewed
at a later time.
• Remember thorough preparation and a sincerer desire to
understand the auditee’s viewpoint can contribute greatly to the
success of an interview.
•“you have two eyes and two ears so look and listen
twice as much as you use your one mouth”

• Physical examination:
• Refers to examination of tangible things e.g. examination of the
temperature and humidity records to see if they are indeed
within spec.
• Findings based on this type of examination is most reliable
because:
• they represent reality and
• the auditee should have little difficulty accepting the findings.

• Observation of work activities
• Much of the data collected during audits is the result of observations.
• Watching, listening, taking notes, and asking questions to assess the employee’s knowledge
are all appropriate actions for an auditor while observing a work activity.
• An auditor needs to realize that his or her presence will distract the auditee performing the
task.
• It is imperative that the auditor try to minimize any disruption, especially if a distraction
could be unsafe.
• It is best if the auditor observes people performing their actual tasks rather than a
simulation of real work activity.
• An auditor needs to understand :
 How to observe
 When to observe
 What to observe
 What they have seen after observing

• How to observe” pertains to watching closely but unobtrusively.
• Auditors observe work in process and monitor processes to determine if the
work being done meets requirements.
• An auditor needs to realize that his or her presence may distract the auditee
performing the task.
• It’s imperative that the auditor try to minimize disruption, especially if a
distraction could be unsafe.
• “When to observe” implies things such as observing work being
performed by different shifts without detaining people through
shift changes or breaks.

• “What to observe” involves items such as but not limited to the following:
 Products
 Equipment
 Facilities/environment
 Individuals
 Documents
• Overall, watching, listening, taking notes, and asking questions to
test the employee’s knowledge are all appropriate actions for an
auditor while observing a work activity.
• It is in this manner of observing that an auditor should be able to
discern if auditee actions meet requirements /expectations.

LESSON 06 – The paperwork
•Topics covered in this lesson
• Audit working paper
 Working papers and working documents
 The importance of working papers and working documents (checklists) in documenting
an audit trail
 Records of observations
• Audit analysis
 Corroboration and objectivity of data
 Classification of observations
 How audit conclusions are formulated.
• Exit meeting
 Basic elements of the exit meeting
 How audit results are presented at an exit meeting
 Exit meeting follow-up action steps

– The paperwork -
• Definition of working papers (or working documents)
• Working papers are all of the documents required for an effective
and orderly execution of the audit plan. The format and content
of working papers describe the scope and approach of the audit
assignment as well as its operational elements
• Examples of working documents:
Audit instructions and procedures
Audit planning documents
Auditor checklists
Auditor reporting forms such as nonconformity forms.

– The paperwork -
•Working documents should be designed so that:
• They do not restrict additional audit activities or investigations
which may become necessary as a result of information
gathered during the audit.
• Working documents involving confidential or proprietary
information must be suitably safeguarded.
• All working papers, regardless of who originates them, should
be approved by the head of the auditing organization or
designate.

– The paperwork -
•Working documents should be designed so that:
• Care should be taken that any instructions can be
clearly understood, (Diagrams should be utilized
where beneficial).
• Instructions interpreting each part of a quality
standard should be given to each team member.

– The paperwork -
• All questions asked/ statement on a working document should be
phrased in an unbiased way to solicit constructive and useful
answers.
• Avoid any questions or wording that will put the auditee on the
defensive.
• For example “ Do you have specifications for shipping and if not why?” This
can be viewed as an accusation of poor conduct

– The paperwork -
•Record of observations:
• All audit observations must be documented
• The audit team should review all of its observations
to determine which are to be reported as
nonconformities
• The audit team should ensure that these are
documented in a clear, concise manner and are
supported by evidence. Evidence is either in notes or
copies of data.
• If critical or major observations are present then documents
should be ‘verifed against the original’.

– The paperwork -
• Record of observations:
• Audit observations should be linked to the standard
or contract requirements specified in the audit plan.
• Observations should be reviewed by the lead auditor with
the responsible auditee manager of internal inspections.
They should be discussed at the closing meeting with a
Third party.
• All observations of nonconformities should be
acknowledged by the auditee management.”
• A written acknowledgement and response to findings is needed from the auditee within
an agreed to timeline

– Audit analysis -
• The audit process is effective when the audit team
continually analyzes data gathered.
• Throughout the fieldwork and daily briefings, team
members need to examine, sort, and verify all the
evidence collected.
• Any information that appears contradictory requires
additional data collection to deny or confirm a position.

- Audit analysis -
• Once the data gathering activities are complete, the audit
team meets one final time to finalize their analysis and
classify the evidence (RISK) before presenting results to the
auditee at the exit meeting.
• It is critical that each finding reported at the exit meeting
should be clear to a reasonable person.
• Discuss the issues only - remember the final wording may alter.
• Accept any CORRECTIVE action and
• Be willing to remove a comment ONLY when you accept your
original finding is in error

- Audit analysis -
•Corroboration and Objectivity of Evidence
• Objective evidence is information which can be proved
true, based on facts obtained through observation,
measurements, test, or other means.
• This definition compels an auditor to collect and present
data in a nonjudgmental, precise fashion without fear of
repercussions.
• Subjective opinions do however play a part in the audit as they relate to
perceived behaviour when performing tasks or in response to audit findings.

- Audit analysis -
• Evidence is objective when it is free of prejudice,
emotions and bias.
• Objective evidence should be collected on all matters
related to the audit objectives and scope.
• Such evidence may be:
Physical
Testimonial
Documentary
Analytical

- Audit analysis -
• Objective evidence may be quantitative or qualitative.
• It two auditors examine the same objective evidence; they should be
able to reach the same conclusion. No doubt. No room for bias or
misinterpretation.

- Audit analysis -
•How to corroborate data?
• If data supporting any conclusion is weak, lacks objectivity, or is
reported based on a single source, an auditor must consider
finding the means to corroborate the data as fact.
• Evidence gathered through interview must be corroborated.
• Corroboration does not necessarily mean that the
information is correct, but it does allow an auditor to
place faith in the information as factual.

- Audit analysis -
• Three methods an auditor can use to corroborate interview
information:
•
• Method 1: another person makes the same comment
• The second person does not have to use the same exact words, but the
“message” should be the same.
• Method 2: another member of the audit team hears the same thing
• If another team member hears the same message, the likelihood of any
missed communication is lessened.
• Method 3: an item, document, or record verifies the comment.
• After hearing an explanation, the auditor can corroborate it by reading a
procedure or a completed form with the information just as it was described

- Audit analysis -
•Data patterns and trends (repeat observations,
systemic problems)
• Throughout the course of an audit, auditors collect date
from a variety sources and constantly analyze the data
and looks for “common threads”.
• Analyzing audit data allows the audit team to distinguish
between systemic incidents (critical/major) and isolated
incidents (minor).
• When auditors detect a potential problem, they need to
gather additional data and look for a recurrence. If
additional data demonstrates a recurrence, the auditor
may classify the incident as “systemic”.

- Audit analysis -
•Overall, an auditor needs to keep a system
perspective when looking at problems:
• When a well-documented procedure is in place and
examination confirms that the procedure is working, with
only one or two minor problems that do not affect the
integrity of the product, the incident is generally
indicative of a minor isolated problem- not a systemic
problem
• Repeat occurrences affecting the integrity of the product
may be significant enough to make it a systemic problem.

- Audit analysis -
•Classification of observations:
• A quality audit observation is a statement of fact made
during a quality audit and substantiated by objective
evidence.
• There is various ways an auditor can classify and prioritize observations: The
one used by the HPFBI, FDA and EU is based on a critical, major, minor risk
evaluation
• Critical observation
• Requires that further operations stop until corrective action is completed.
• Critical findings have a high probability of causing adverse consequences to
the patients or consumer, any result in significant deviations in the safety,
identity, strength or purity of the product, are in direct violation of a
manufacturing licence or represent fraud, misrepresentation or falsification
of data.

- Audit analysis -
• Major observation:
• Requires as immediate corrective action plan and response
although operations can proceed.
• Major findings are those that could significantly impact the
quality of the product, may be considered major deviations by
regulatory authorities, or are a combination of minor deficiencies,
which indicates a major system failure.
• Minor observation: (“Other” in HPFBI Risk Classification)
• Requires corrective action.
• Minor findings are those that deviate from accepted standards
but have a relatively low probability of affecting the quality or
usability of the product.
• They represent opportunities for improvement in overall GMP
compliance.

- Audit analysis -
• Comments:
• Recommendations may be made by the auditor in areas nor directly
covered by GMP regulations. Comments should be used very carefully to
avoid misunderstandings
• Ultimately, an auditor needs to sort and classify facts based on the:
Severity of problem
Frequency of the problem occurrence
Risks associated with the problem
• In order to avoid any ambiguity, a lead auditor should ensure that the
audit team, the auditee, and the client understand the meaning of the
words being used. As appropriate, definitions should be provided to all
parties.

- Writing Non-Conformance Statements -
• An auditor must provide a thorough description of audit
observations when writing them.
• Describe the Evidence looked at
Describe the Nature of he nonconformity
Specify what the Requirement was
Make your statement (C4) :
 Clear,
Concise,
Complete, and
Correct

Formulating Conclusions
• Formulation of audit conclusion
• Prior to the audit exit meeting (during the final team meeting),
the audit team needs to agree to overall conclusions as to
whether system requirements are being met.
• Conclusions must be consistent with facts gathered during the
audit.

Formulating Conclusions
• The overall audit conclusions are presented at the exit meeting.
• The presentation of these conclusions should be concise but accurate.
• Supporting details should be reserved for the formal audit report.
• Similar to the classification of observations and non conformances, varying
conventions exist for classifying conclusions.
• Some of the more common terms auditors apply to conclusions are:
Concerns
Areas for improvement
Major/ minor noncompliance
Deficiency
• It is imperative that all parties understand what is implied by the classifications
being used.

The Exit Meeting
• Elements of the exit meeting (Also called closing meeting or post audit conference.)
1. Presenting the audit observations to the senior management in such
a manner so as to ensure that they clearly understand the results of
the audit.
2. The lead auditor should chair presenting the observations
3. Records of the meeting should be kept
4. The exit meeting should be held shortly after the data gathering
phase ends- following the final team meeting, if possible.
5. The formality and duration of a closing meeting is driven by the type
and scope of the audit and at this point the findings

The Exit Meeting
• Elements of the exit meeting (Also called closing meeting or post audit conference.)
6. Usually a tentative time is set up so attendees can note the time on their
schedules or advise the auditor of any scheduling conflicts
7. The entire audit team should attend the exit meeting
8. At least one management representative from the auditee organization
should attend the closing meeting. The escort may also be present to
help explain some of the details of the audit findings.
9. Attendance of senior-level management representing the auditee
organization is indicative of the auditee’s commitment to implementing
audit results.
(client representative may or may not attend the exit meeting)

• Audit results reported at an exit meeting may be communicated
through a narrative presentation or a handwritten or typed
document
1.Any findings and conclusions to be included in the final report
must be presented by the auditor at the exit meeting.
2.Audit findings not mentioned at the exit meeting may not be included in the
final report without prior consent of the auditee.
3.Failure to discuss all information at the exit meeting or to secure permission
of the auditee to include findings not discussed jeopardizes the credibility of
the entire audit effort.
4.At the end of the exit meeting, the auditor needs to let auditee and the client
know when the final report will be available. A general rule of thumb is that
the formal audit report should be sent to the auditee as soon as possible
after the closing meeting.

Roles – The Exit Meeting
• Auditor roles and responsibilities:
The lead auditor formally thanks the auditee for the hospitality and courtesies
extended to the audit team
Recaps the audit scope and purpose
Presents an audit summary
Leads a discussion of the audit details
Indicates how audit results are classified
Explain the nature of expected corrective action and required follow-up
Ensures that minutes and an attendance record are kept

Auditee roles and responsibilities
Listen to the summary without
interrupting the auditor
Ask for clarification as appropriate
Provide additional information as
appropriate to the auditors’ requests for
corrective action.

The Exit Meeting
• Presentation of audit results:
Individual auditors may present findings from area they
audited. Or the lead auditor may present all findings. If the
lead auditor presents all finding, individual auditors may clarify
statements and/or respond to specific auditee questions as
needed.
The lead auditor presents the final conclusions on the overall
effectiveness of the auditee’s quality system
• Three housekeeping items are recommended to help
ensure a well-run exit meeting:
1.Prepare a meeting agenda
2.Maintain a record of who attends the meeting
3.Take minutes of any discussions or agreements made at the
meeting.

Exit Agenda
• The agenda should list items that ate typically covered at every closing meting:
• Thanks- to allow the auditor to formally thank the auditee for the hospitality and
courtesies extended to the audit team
• Audit purpose and scope- to allow the auditor to provide a brief recap of these items
• Summary- to allow the auditor to present overall conclusions to the auditee
• Findings and/or positive practices- to allow the auditor to present highlights of
findings and any positive practices observed, without great detail.
• Corrections- to allow for a dialogue between the auditor and the auditee to explain
and/or clarify any areas
• Corrective actions- if problems were identified, to allow for a discussion of corrective
action requests and the follow-up process

Exit Agenda
• Meeting attendance record
• The lead auditor typically provides an attendance roster or makes available a
sheet for meeting attendees to sign.
• Meeting minutes
• Any important items discussed during the meeting need to be recorded. For
example: additional evidence provided by the auditee.
•
Discussion of follow-up actions
• After the findings and/or positive practices have been
presented and discussed, the exit meeting concludes with an
explanation of corrective action request and the follow-up
process required of the auditee.
• A period of 30 days is generally considered an acceptable time
frame for a response.

Lesson 7 – The Report-
•Audit reporting
Review and finalizing audit results
Written report format and content
Issuing the written report
Audit records retention

– The Report-
• Review and finalizing audit results
• The purpose of the audit report is to communicate the results of an
investigation.
• Reports should avoid “nitpicking”, but they should identify isolated errors
that in the judgment of the auditor may be a sign of an emerging weakness.
Such isolated errors should be reported as opportunities for improvement.
• An audit report shouldn’t make the auditee feel alienated or devalued. But
at the same time, appropriate organizational personnel must be made aware
of any negative findings so that they can take necessary corrective action.

– The Report-
• An audit report should have the following characteristics:
Accuracy
Conciseness
Clarity
Timelines
Tone
Objectivity
Relevance
Consistency
Comparability

– The Report-
• Audit report preparation:
• The audit report is prepared under the direction of the lead
auditor who is responsible for its accuracy and completeness.
• While the lead auditor has ultimate responsibility for the report,
other members of the audit team should review report contents,
as appropriate, to verify its accuracy and objectivity.

– The Report-
• Content of audit report:
• The audit report should reflect both the tone and content of
the audit.
• It should be dated and signed by the lead auditor.

– The Report-
• The Report should contain the following items, as
applicable:
 The scope and objective of the audit
 The details of the audit plan; the identification of audit team
members and auditee’s representative, audit dates, and
identification of the specific organization audited.
 Identification of the reference documents against which the
audit was conducted (quality system standards, auditee’s
quality manual, etc.)
 Observations of nonconformities
 Audit team’s judgment of the extent of the auditee’s
compliance with the applicable quality system standard and
related documentation
 The system’s ability to achieve defined quality objectives
 The audit report distribution list.

– The Report-
• Additional items may be considered for an audit report:
Statement of appreciation- a formal thank you to acknowledge the aduitee
participants for their cooperation (to reinforce the verbal thank you given at
the closing meeting)
Checklist/procedure/flow chart/ document identification- (this is optional)
Positive findings- a brief overview of any positive findings
Observations- a brief overview of any positive or negative observations
Response date- the time frame in which a corrective response will be due as
well as whom the response should be sent to

– The Report-
• Verifiable information
• Reports should be verifiable.
• While it is not realistic to provide specifics for every piece of
evidence, the report should include generally accepted
references (such as a machine name or reference to type of
files, locations, etc., but never name of personnel) to minimize
the danger of a message being misunderstood.

– The Report-
• Verifiable information
• Including references to items or locations at an audit site in an
audit report enhances the credibility of the information and
the reader’s confidence that the data are verifiable.
• Furthermore , verifiable information helps the auditee to fully
understanding the finding during the post –audit investigation
for root cause analysis and subsequent corrective action
activity.

– The Report-
• Summarized checklists
• A structured checklist ensures that the full scope of an audit was covered.
• A checklist is also a convenient document in which to record facts during the
audit.
• However, including checklists in a report can make the report cumbersome and
potentially unreadable.
• Readers may be bogged down reviewing detailed checklists and miss the overall
message of the report.
• Checklists, therefore, should be filed and kept as a backup should questions
arise; structured checklists should not be included with the report. A
Summarized checklist is recommended.

– The Report-
• Clear reporting
• Auditors should put themselves in the place of auditee. Reports should be user-
friendly, so the auditee can easily understand what is being discussed:
Reports should be user friendly; use terminology that the reader will understand
If you choose to use acronyms, they should always be defined
Be direct and to the point. Auditee executives and managers appreciate audit
reporting that gets to the bottom line without excessive verbiage
The KISS (Keep it Short and Simple) principal is a good rule of thumb in audit
report writing

– The Report-
• Auditors should put themselves in the place of auditee. Reports should be user-
friendly, so the auditee can easily understand what is being discussed:
Use a standard format for report organization. It is easier for the auditee to make
comparisons from audit to audit
Reference requirements to allow traceability
Define unfamiliar terms such as Critical observation, Major Observation,…
Do not include confidential information in the audit report unless it is specifically
related to a finding.
Information that was not mentioned or discussed during the exit meeting should
not be included
Deficiencies that were acted upon and corrected by the auditee during the audit
may be included in the final report as a closed finding.

– The Report-
• Written report format and content
• Report format, unlike report content, can vary widely from
industry to industry or even from one organization to another
within the same market environment.
• Order of information
• Most final reports contain the following (or equivalent) sections:
Introduction / Background
Executive summary
Audit results
Corrective action request
Exhibits
Summary

– The Report-
• Introduction / Background
The purpose and scope of the audit
Names of the auditee, escort, client, and auditing organization
Name of audit team members and their qualifications
The audit dates
The standards audited against
Report distribution

– The Report-
• The following information is sometimes included in the
Introduction:
People attending the opening meeting
Auditee personnel involved in or contacted during the audit
The auditee’s audit history
What documents or records were examined. This may be in a
Scope sub-section.

– The Report-
• Executive Summary
Audit team’s judgment of the extend of the auditee’s compliance
with the applicable quality system standard and related
documentation
The systems ability to achieve defined quality objectives
• Audit results
Positive findings
Observations of nonconformities
Recommendations
Provisions for the auditee’s response (response date)
Provisions for recording corrective action and follow up activities.

– The Report-
• Corrective action request
• Corrective action requests should ask the auditee to determine
the following information:
The root cause of the problem
Long-term corrective action planned for the cause when
applicable
Short-term and intermediate corrective action planned for each
of the finding listed
Schedules and responsibilities for these actions
Metrics for determining the effectiveness of corrective action

– The Report-
• Exhibits
People interviewed
Document reviewed
Summarized checklist

- The Report -
• Summary section
• The summary portion of the written report is the most
important section. It provides bottom line information. The
following questions are answered by a report summary:
Are the necessary controls present?
Are they implemented across all activities?
Do they work?
Is the group achieving higher-level controls necessary for world-
class quality?
What is the next activity ?
• As in other audit communications, report summary should be
clear, concise, completer, and correct.

- The Report-
•Report length
• There is no standard length for the audit report. An audit
report must be concise and still cover the prescribed
information, a two-page final report with findings and
positive observations shown as attachments is a typical
arrangement.
•Compliance
• The primary purpose of the audit is to verify compliance
and, with significant discrepancies or violations of the
quality program, request corrective action.

- The Report -
• Categories of compliance
• The results of the audit-the audit observations- may be classified
and reported as any one of the following:
Findings
Nonconformities
Deficiencies
Adverse conclusions
Significant observations and conclusions
• In any case, the auditor needs to be clear and consistent in terms used to report
audit results.

Pharmaceutical Auditing and Inspections Professor Peivand Pirouzi 2010

Pharmaceutical Auditing and Inspections Professor Peivand Pirouzi 2010

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (12)

Similar to Pharmaceutical Auditing and Inspections Professor Peivand Pirouzi 2010

Similar to Pharmaceutical Auditing and Inspections Professor Peivand Pirouzi 2010 (20)

More from Pharmaceutical Compliance Inspection unit, Crown College of Canada

More from Pharmaceutical Compliance Inspection unit, Crown College of Canada (20)

Recently uploaded

Recently uploaded (20)

Pharmaceutical Auditing and Inspections Professor Peivand Pirouzi 2010