The document discusses the importance of audit programs and reports in enhancing organizational performance, emphasizing the need for clear communication of audit findings. It introduces a new tool that converts ISO 9001:2015 audit results into radar charts, making it easier for auditees to visualize their performance and identify improvement areas. This approach addresses the common resistance encountered during audits by providing a comprehensive and visual representation of the audit results.

1. 34 QP October 2017 ❘qualityprogress.com
F E AT UR E
AUDITING
Increasing the e"ectiveness and e‰ciency of audit programs is
an essential key performance indicator for those in the qual- ity
community. That’s why audit programs and audit reports are
important tools to boost an organization’s performance.
The most valuable output of an audit program is the audit report.
The audit report should focus on future issues and o¦er insights on
areas to improve.
Because every auditor has an individual vision and perspective, it
can be di‰cultto calibrate all auditors to provide excellent outputs,
such as audit reports. To do this, you may enhance auditors’ qual-
ifications, perspectives and visions by training them, or you could
provide them with an audit tool designed to assess all aspects of
related processes without any subjective evaluation or gaps.
There are already some tools designed for such assessments, such
as the European Foundation for Quality Management’s excellence
model‘ or Michael Hammer’s process maturity assessment model.²
But there’s a new tool called the ISO 9001:2015 assessment tool
that converts the standard’s requirements to radar charts and can
provide better and more tangible results from quality management
system (QMS)audits.
Just the
Facts
Reports that
detail findings
from quality man-
agement system
audits can some-
times be unclear
and ambiguous
for those being
audited.
To allow auditees
to more easily
see gaps in their
system and areas
to improve, and
make iteasier
for auditors to
communicate the
need for change, a
new tool converts
the ISO9001:2015
requirements to
radar charts and
visually presents
the findings.
Visual
Aid
New tool converts QMS audit findings
to radar charts |by Ferhan Bugay

2. qualityprogress.com ❘October 2017 QP 35

3. 36 QP October 2017 ❘qualityprogress.com
F E AT UR E
AUDITING
Defensive and
resistant
As an auditor, how do you feel
during the closing meetings
of QMS audits? Think about it.
The audit you justperformed
progressed well, and at the end, you’ve unveiled
your findings and recommendations to the audi-
tee. Suddenly, the level of happiness on both
sides starts to drop. People begin to resist your
suggestions and get defensive, sometimes blam-
ing the organization, co-workers or the system.
How do you feel in such moments?
The problem is that the system we’ve created
for auditing QMSs is not realistic. Inreal life,
nobody wants to be observed or tested by
someone else. Even you don’t like it, do you?
Do you like to be audited, tested or graded by
somebody?
All living organisms have the instinct to resist
change. Even animals and plants resist change,
and that’s why evolution of all species around
the globe continues. That’s the reason immune
systems of organisms and creatures evolve over
time and develop defense mechanisms.
Let’s go back to the closing
meeting. When others resist,
auditors start to change their
behavior, trying to explain the
issue and convince others to
change, which can be awk-
ward. Certainly, this is not the case for all audits.
Sometimes, though, the auditee forces you to
further elevate the findings and dig deeper. We
must acknowledge their resistance, but we must
also explain to them the need for change.
Standards for some industries can con-
tain very specific and concrete requirements.
Wıth QMS audits, however, the results are not
tangible, and International Organization for
Standardization (ISO) standards encompass all
industries.
This is why, during a QMS audit’s closing
meetings, people can get upset. Findings and
observations can be ambiguous issues for
auditees. Nobody is intrested in boring meet-
ings. When an audit closure is not supported
with a visual performance report of findings and
recommendations, it becomes less comprehen-
sive. This tool encourages people involved to
TA B L E 1
Rankingandapplicable
scoreforeachitem
TA B L E 2
Assessmentresults
foreachclause
Ranking Auditorguidance for ranking
0 Not documented, not implemented or poorly
implemented.
1 Documented, poorly implemented.
2 Documented, implemented, poor improvement.
3 Documented, implemented, improved.
4
Documented, implemented, improved, shared with
interested parties for continuous sustainability.
Feedback gathered from employees, sub contractors,
customers for continuous improvement.
Clause
Performance
(average
points)
Desired
result
4. Context of theorganization 65.6 100
5. Leadership 83.3 100
6. Planning 50.0 100
7. Support 55.0 100
8. Operation 47.4 100
9. Performance evaluation 56.9 100
10. Improvement 41.7 100
If we can provide a detailed
picture of the audited
processes, it will be much
easier for auditees to
visualize the problems and
understand and accept
why improvements are
necessary.

4. qualityprogress.com ❘October 2017 QP 37
undertand the entire QMS system. When you
raise a generic finding regarding some clause
of a standard that the auditee is unaware of, for
example, the auditee often starts a long story to
explain the situation.
That’s why my concern has been finding ways
to achieve tangible results from QMS audits.
Statistical map
To cope withthe generic ISO 9001:2015 require-
ments, Ihave deployed a tool in which you can
convert audit results into a statistical map so
auditees can see their performance against best
practices.
My intention has always been to find a way to
better engage auditees and help them under-
stand QMS standards as a system designed to
improve processes in any industry.
Most of the time, auditees’ negative reactions
to a QMS audit are influenced by the auditor’s
approach and the audit outputs. If we can pro-
vide a detailed picture of the audited processes,
it will be much easier for auditees to visualize
the problems and understand and accept why
improvements are necessary.
That’s why the way audit results are com-
municated is so important, and we must be
careful during this part of the process. When
you convey information regarding the audit you
performed, most of the time, the information
you share is verbal and not visual. But our brains
love fun and vivid action, which is why pictures
can be helpful.
This new approach visualizes audit results
by starting with an audit assessment checklist
sheet (Online Table 1,which can be found on this
article’s webpage at www.qualityprogress.com).
The assessment is done by ranking (zero to four)
the organization’s system against each of the
standard’s subclauses.` Definitions of rankings
are listed inTable 1.
The sheet, which contains various colors,
documents what the auditee has in place and
what the standard requires. For example, the
FI G U R E 1
PerformanceofClause8—Operation
Performance
Desired result
= 100.0
50.0
62.5
54.2
50.0
50.0
50.0
37.5
25.0
0.0
20.0
40.0
60.0
80.0
8.1Operational
planning andcontrol
100.0
8.2Requirements
for products and
services
8.3 Design and
developmentof
products and
services
8.4 Control of
externallyprovided
process, products
and services
8.5 Productionand
service provision
8.6 Releaseof
products and
services
8.7 Control of
nonconforming
outputs
8.8Nonconforming
goods andservices

5. F E AT UR E
AUDITING
38 QP October 2017 ❘qualityprogress.com
yellow-shaded columns
require records, thered-
shaded columns require
documentation, and the
green-shaded columns require
documentation and records as
defined by ISO9001:2015.
The formulations for each
subclause assess the level of
conformity to ISO 9001:2015
subrequirements and provide
the end result for each clause
by averaging the points for
the subrequirements to find
an assessment total for each
clause (Table 2, p. 36). From
there, the radar maps(Figures
1-3,pp. 36-37) are drawn.They
are simply Excel sheets that
contain formulations for each
ISO 9001:2015 clause. While
you assess the requirements,
the Excel sheets calculate the
level of conformity and draw
the radar maps.
The tool can draw perfor-
mance for individual ISO 9001
clauses. The radar maps in Fig-
ures 1-2are samples ofspecific
clause performances. The tool
also can provide a snapshot of
overall performance by com-
bining all clauses on one radar
map (Figure 3).
Benefits of the
assessment tool
Using the ISO 9001:2015
assessment tool provides
many benefits for auditees
and auditors. The performance
scale may help organizations
determine the integrity of an
ISO standard and its interre-
lated clauses becausethere
is a deep correlation between
all clauses. Using this tool can
FI G U R E 3
OverallperformanceofISO9001:2015
FI G U R E 2
PerformanceofClause7.1—Resources
65.6
83.3
50.0
55.0
47.4
56.9
41.7
0.0
20.0
40.0
60.0
80.0
4. Context ofthe
organization
100.0
5. Leadership
6. Planning
7. Support8. Operation
9. Performance
evaluation
10.Improvement
Performance
Desired result
= 100.0
58.3
50.0
33.3
75.0
58.3
20.0
0.0
100.0
80.0
60.0
40.0
7.1Resources
7.2Competence
7.3 Awareness7.4 Communication
7.5Documented
information
Performance
Desired result
= 100.0

6. qualityprogress.com ❘October 2017 QP 39
help auditees save time address-
ing useless corrections and
refocus their energies onfinding
real root causes to problems and performance
gaps.
Auditors will see benefits from using the tool
because it:
+ Maintains integrity for the entire standard.
+ Ensures the audit covers allsubitems.
+ Provides a bottom-up approach.
+ Focuses on the entire system, ratherthan
subclauses.
+ Allows the auditor to use and study current
ISO references during audits instead of creat-
ing checklists.
Auditees will see benefits from the tool because:
+ The tool provides a holistic approach toISO
9001:2015requirements.
+ Comparisons can be made between desired
results and currentperformance.
+ Performance can be reviewed visually. Most of
the time, auditees are unable to determine the
impact of the findings to the system.
+ The assessment checklist sheet (Online
Table 1)can be referred to later to further
review subreferences for each finding or
recommendation.
+ Details on each issue using ISO 9001:2015
references can be focused on more closely.
Transparent outputs
As Figure 3 shows, the tool can provide an
overall picture to auditees. Instead of simply
conveying information in words, the auditee can
review current performance and the desired
level of performance via these maps.
This output may be helpful for senior mem-
bers of the organization being audited—like
busy CEOs and chief financial o‰cers who want
general information about the organization’s
performance and proposed areas to improve,
and not nitty-gritty audit findings.
The tool has been applied in many audits
and has proven to be beneficial for auditors
and auditees. The tool also may be used as an
audit report or draft report. In general, the audit
reports written by third-party bodies or internal
Ferhan Bugay is a quality
assurance lead auditor for Turkish
Airlines at Atatürk Airport in
Yesilkoy. He earned an MBA from
Beykent University in Istanbul. He
is an ASQ member.
In general, the audit reports
written by third-party bodies or
internal auditors contain only
findings and recommendations. auditors contain only find-
ings and recommendations.
Using this tool as a
reference may provide the opportunity to
visualize the gaps for improvement—even for
minor issues. Visual tables make the QMS easy
to understand and provide a holistic view. The
auditee also may compare good and bad-per-
forming processes and review them in the
organization.
Quality is in the details and the more we
dissect our processes, the easier we can realize
how to improve them and make them perform at
an excellentlevel.
Organizations will survive only if they adapt
quickly and stay in tune with their customers’
changing habits. This tool can help auditors
provide transparent outputs to stay in tune with
QMS requirements.
REFERENCES ANDNOTE
1. European Foundation for Quality Management
(EFQM), “The EFQM ExcellenceModel,” www.efqm.
org/the-efqm-excellence-model.
2. Michael Hammer, “Process Maturity Assessment”
www.hammerandco.com/pemm.htm.
3. The rankings and scores were inspired by Michael
Hammer’s “Process MaturityAssessment.”
BIBLIOGRAPHY
Hammer, Michael, “The Process Audit,” Harvard Business
Review, April 2007, https://hbr.org/2007/04/ the-
process-audit.
Internatıonal Organization of Standardization,
www.iso.org.
EDITOR’S NOTE
To access the tool, contact Bugay at https://tr.linkedin.
com/in/ferhanbugay or ferhanbugay@gmail.com.