This document provides information on surgical audit and clinical research. It defines clinical audit as a quality improvement process that systematically reviews care against criteria to implement change and improve outcomes. Surgical audit similarly analyzes surgical quality and care against standards to improve practice. Audits identify if standards are met and research is used in practice, help reduce risk, and improve patient care. They follow the clinical audit cycle of choosing topics, collecting data, analyzing results against criteria, improving care, and re-auditing. Research aims to generate new knowledge by testing treatments or regimens with study design and analysis. It asks different questions than audits and requires identifying topics, designing projects, analyzing data, and publishing findings.

1. SURGICAL AUDIT
AND CLINICAL
RESEARCH

2.  “Surgery without audit is like playing
cricket without keeping the
score.”(Hugh brendon devlin 1932-
1998)”

3. Definition
 Clinical audit is a quality improvement
process that seeks to improve patient care
and outcomes through systematic review of
care against explicit criteria and the
implementation of change.
 the word ‘auditing’ has been derived from
Latin word “audire” which means “to hear”.

4.  Surgical audit is a systematic, critical analysis
of the quality of care that is reviewed by peers
against explicit criteria or recognised
standards, and then used to further improve
the surgical practice with the ultimate goal of
improving the care of patients.

5. Surgical Audit –why do it?
 To identify whether standards are being met, and
evidence from research are being used in
practice.
 To identify baselines for development of
standards.
 To reduce clinical risk.
 To ensure cost effective use of resources, and
effectiveness of a service.
 To highlight problems, and help in the solution.
 To improve team working and communication.
 To improve patient care and outcomes.

6. HISTORY
 One of first ever clinical audits was
undertaken by Florence Nightingale during
the Crimean War of 1853-1855.

7. Aspects of patient care in
audit
1. Structure – what is in place.
2. Process – what you do
3. Outcome – the results you get

8. 1. Structure – what is in place
 The people,
 Their training,
 Their knowledge,
 The way they are led,
 The equipment,
 Their organization,
 The way they are paid, etc.

9. 2. Process – what you do
 How referrals are processed,
 What diagnostic tests are done,
 The antibiotics that are used,
 The thromboembolic prevention that is customary,
 The use of intensive care,
 The policy of feeding & mobilization after surgery,
 The discharge policy, etc.

10. 3. Outcome – The Results
You Get
 Wound dehiscence rate,
 Readmission rates,
 Mortality,
 Freedom from progression,
 Reduction in symptoms,
 Improvement in quality of life,
 Return to work, etc.

11. Explicit Criteria:
 An individual who needs training
 An instrument that needs replacing
 At team level e.g. nurses undertaking procedures instead of, or
in addition to, doctors
 At institutional level e.g. new antibiotic policy
 At regional level e.g. provision of a tertiary referral centre
 At national level e.g. screening programmes & health
education campaigns

12. Audit – can be done in different
forms
 A personal surgical audit (total/ practice/
selected);
 Group/ hospital / specialty audit (focused or
generic).
Total Practice or Workload Audit: This is an audit
that covers all the surgical operations performed.

13.  Selected Audit from Surgical Practice:
This is an audit that covers all patients who
undergo a selected procedure, or an audit that
covers all procedures conducted within a selected
time-frame.
 A Clinical Unit Audit:
This is an audit conducted by a clinical unit in
which individual surgeons may participate.

14.  Group or Specialty Audit:
This is an audit conducted by or under the auspices of a
group or Specialty Society
e.g. the National Breast Cancer Audit,
 A Focused Audit:
This is an audit which looks at one or more Council for
Health Care Standards indicators and the factors which
influence it:
e.g. what is the wound infection rate after large bowel surgery
– emergency/ elective procedure, type of surgery, antibiotic
prophylaxis blood loss

15. There are a number of types of audit that take place within
an institution, including:
 • Morbidity and mortality meetings
 • Local/regional audit
 • National or international comparative audit.

17. Purpose
 Selected cases are presented at mortality and
morbidity review (M&M) meetings for the
purpose of:
• Discussing management decisions
• Providing a learning opportunity
• Identifying opportunities to improve patient
safety and quality of care

18. Preparation
 Identify clinicians who will form the core group for
the department M&M meetings
 Appoint a senior consultant to be the chair and to
have responsibility for meeting management.
 Appoint a registrar or fellow with responsibility for
case coordination
 Book a regular meeting time. It is a requirement that
meetings are held monthly.

19. Case identification
 Cases presented at department M&M meetings
will be identified from a range of sources
including:
• all deaths which occur under the unit
• any case referred by a clinician
• Any case referred via the organisation.

20. standardized format for M&M
presentation

22. Conduct of meeting
 Encourage a focus on patient care.
 Establish a ‘safe’ environment
 M & m meetings are not convened for the purpose of
focussing on an individual’s performance.
 Promote participation from those attending the
meeting
 Ensure brief minutes and action items are taken
 Focus the discussion on identifying what went wrong,
why it went wrong, what could be done differently in
the future and what action is required

23. The Clinical Audit Cycle

24. Choose Topic
In this stage the audit team will identify a
problem, or an area of healthcare service
that is to be compared to standards.

25. When Choosing A Topic It Is
Recommended To Focus On Areas
Where
 Standards and guidelines exist, and there is conclusive
evidence about effective clinical practice.
 Problems have been encountered in practice.
 There have been recommendations or complains from the
patients or public.
 There is high volume, high risk, or high cost.
 There is clear potential of improvement of service

26. Determining scope
Common areas in the scope of an audit include:
 30 day mortality and significant morbidity;
 Length of hospital stay;
 Positive and negative outcomes
 Operation-specific complications
 Use of investigations
 Justification of management
 Patient satisfaction.

27. Choose Topic
Decisions regarding the overall purpose of the audit, either as what
should happen as a result of the audit, or what question you want the
audit to answer, should be written as a series of statements or tasks that
the audit will focus on. Collectively, these form the audit criteria.

28.  A criterion is a measurable outcome of care. For example,
‘Patients in the ICU should receive GIT bleeding
prophylaxis’.
 A standard is the threshold of the expected compliance for
each criterion (these are usually expressed as a percentage).
For the above example an appropriate standard would be:
‘GIT bleeding prophylaxis received in 90% of cases’.

29. Criteria can be classified as
 Structure criteria.
 Process criteria.
 Outcome criteria.

30. Remember, Criteria of an audit should
be:
S
M
A
R
T
Clear, not vague
Objectively
Realistic
To the topic
As well as Theoretically sound

31. Choose Topic
•Data collection sheets or
questioners developed by the
audit team.

32. COLLECT DATA
 The information necessary to answer the audit question.
 Collect prospectively or retrospectively.
 Follow up data collected.
 Data can be collected from a register, medical records
data, review of referrals, or from previous appointment
schedules.

33. Choose Topic
•Simple Statistical analysis
based on the type of data
collected.
•Graphical presentation.

34.  Analysis stage, whereby the results of the
data collection are compared with criteria
and standards.
 The end stage of analysis is concluding
how well the standards were met and, if
applicable, identifying reasons why the
standards weren't met in all cases.
 These reasons might be agreed to be
acceptable, i.e. could be added to the
exception criteria for the standard in future,
or will suggest a focus for improvement
measures.

35.  In theory, any case where the standard was not met in 100% of
cases suggests a potential for improvement in care.
 In practice, where standard results were close to 100%, it
might be agreed that any further improvement will be difficult
to obtain.
 And that other standards, with results further away from
100%, are the priority targets for action.
 This decision will depend on the topic area, in some ‘life or
death’ type cases, it will be important to achieve 100%, in
other areas a much lower result might still be considered
acceptable.

36. Choose Topic

37. Discuss your results…
 Conclude if standards were met, and if not, because of
what.
 What needs to be done about that?
 Is the solution practical?
 Was the standard itself applicable ?
Share your results…

38. Once the results of the audit have been published
and discussed, it is time to formulate and action
plan, that should include;
o What needs to be done or changed.
o Who is going to do it.
o When it is going to be done.
o How it will be done.
o Involve higher authorities.

39. Choose Topic

40. RE –AUDITING CYCLE…
 After an agreed time frame, the audit should be repeated.
 The same methods and data analysis are used to ensure
comparability.
 The re-audit should demonstrate that the changes have
been implemented and that improvements have been
made.
 This stage is critical to the successful outcome of the
audit process as it verifies whether the changes
implemented have had an effect.

42. Summary – The five stage
approach to
clinical audit
Step-By-Step
Guide for Doing
An Audit

43. Stage 1 – preparing for audit
 Think broadly.
 Funding..
 Skills.
 Time.
 Teamwork

44. Stage 2 – selecting criteria
 Think big.
 Check guidelines.
 Systematic reviews..
 Process or outcome.

45. Stage 3 – measuring the level
of performance
 Routine data..
 Electronic data.
 Medical records.
 Abstract data.
 Legalities.

46. Stage 4 – making improvements
 Barriers..
 Feedback..
 Discussion..
 Implementation methods.

47. Stage 5 – sustaining
improvement
 Re-audit.
 Structural change.
 Cultural change. .

48. What Makes Effective Audit?
1. Promotion of a culture of audit: audit is undertaken in an
atmosphere that highlights educational aspects, is regarded as
non-threatening or ‘safe’, and is carried out in a culture of
‘no-blame’. This atmosphere enables open discussion of
findings, and participants will be able to discuss their feelings
concerning audit reviews.
2. Allocate time and resources: Audit should not be allowed
to become a burden, as this will make participation difficult.
It should be considered as part of normal clinical practice.

49. 3. Oversee and verify data collection
The data should be accurate and complete, with clinical details
provided by clinicians.
Don’t forget to look for:
• The complication that didn’t occur;
• The death that was missed;
• The house surgeon’s diagnosis that was misconceived;
• The misinterpreted pathology report; and
• The reason for the misdiagnosis

50. 4. Productive peer review:
 Good follow up and implementation of change requires the surgeons to
work closely with management and putting in place systems for quality
improvement and risk management

51. Privacy
 Confidentiality in audit process is essential.
 It is also important to reassure participating surgeons
and other team members that peer review discussions
constitute confidential professional peer review rather
than a witch hunt.

52. Educational opportunities that can arise from audit
include:
• Encouraging collaboration, modifying attitudes and approaches to clinical
problems;
• Enhancing critical approaches and giving a rational basis to local changes
in clinical practice;
• Encouraging learning about new technologies and procedures and
auditing their introduction to provide justification;
• Indicating deficiencies in knowledge and skills, which leads to
development of educational activities to address these; and
• By developing required standards of care, giving guidance as to what is
expected

53. Is Clinical Audit A research
Research asks:
Are we singing the right song?
Audit asks:
Are we singing this song right?

55. Patient registry
 A patient registry gathers data on
patients in a systematic way. Such
registries can be very helpful in
understanding the levels of service
provision and can have a place in
research and audit. Service evaluations
 Evaluations usually assess the details of
the service provided. Unless the
evaluation assesses the service against
explicit criteria it cannot be regarded as
clinical audit.

56. Research study and study design
 Research is designed to generate new knowledge
and might involve testing a new treatment or
regimen.
 COMPONENTS OF RESEARCH:
1. IDENTIFYING A RESEARCH TOPIC.
2. PROJECT DESIGN.
3. STATISTICALANALYSIS.
4. ANALYSING A SCIENTIFIC ARTICLE.
5. PRESENTING AND PUBLISHING AN ARTICLE

57. WHAT IS THE PURPOSE OF THE
STUDY?
 A good clinical study starts with
 a good question based on good hypothesis that is based on good and
comprehensive review of the available evidence from pre-clinical and
clinical data
 Type of design depends on the question to be answered
 Most studies can be placed into one of two general categories, descriptive
(or exploratory) and analytic.

58. Good research study -FINER criteria
Feasible
Interesting
Novel
Ethical
Relevant

59. WHAT IS BEING COMPARED?
1. Complications,
2. Cost, efficacy,
3. Effectiveness,
4. Quality of life,
5. Functional status,
6. Patient satisfaction

60. WHAT IS THE OUTCOME OF
INTEREST?
1. Safety.
2. Effectiveness and Efficacy.
3. Patient-Reported Outcomes.
4. Resource Utilization.
5. Costs

61. Safety
Safety end points capture
 The inherent risks of an operation (e.., Surgical site
infection),
 Natural history of the underlying disease process in the
context of therapy (e.g, Malignancy-associated deep venous
thrombosis in a postoperative patient), and/or
 Operative mortality and postoperative complications
(morbidity) are the most commonly measured markers of
safety

62. Effectiveness and Efficacy
 Efficacy refers to the extent to which a treatment intervention
achieves its purported benefit and the durability of that result.
 Efficacy is usually, requires comparison of the selected
intervention to a control group, may include randomization, and
usually necessitates longer follow-up.
 Efficacy studies are more challenging to execute and more
expensive to fund than simple descriptive studies
 Effectiveness relates to outcomes in real-world practice.

63. Patient-Reported Outcomes
 Patient-reported outcomes measure subjective outcomes of care
reported by the patient directly, without further interpretation of
this response by a provider or researcher.
 PRO data are collected through the use of survey instruments.
Examples of common PRO concepts are
 Health-related quality of life (HRQOL),
 Satisfaction with care,
 Functional status,
 Well-being, and health status.

64. Resource Utilization
 Resource utilization refers to the use of health
services related to an intervention.
Includes
 Length of stay,
 Hospital readmission,
 Use of outpatient,
 Pharmacy, and durable medical equipment (e.G.,
Wheelchairs and oxygen) services, and
 Emergency room use

65. Costs
 Charges are the amount of money requested for health
services and supplies. By comparison, costs are the actual
amount of money required to deliver care.
 A cost-utility analysis quantifies health benefits in terms of
quality-adjusted life-years (QALYs). Utilities are a measure
of overall quality of life, usually scaled between 0 and 1,
with 1 being perfect health,
 Incremental cost-effectiveness ratio (ICER), which is the
difference in costs between two competing therapeutic
options divided by the difference in health outcome.

66. WHAT IS THE STUDY DESIGN?

67. Clinical Study Types
 Observational Studies
 Cohort (Incidence, Longitudinal)
 Case-Control
 Cross-Sectional (Prevalence)
 Case Series
 Case Report
 Experimental Studies
 Uncontrolled Trials
 Controlled Trials
67
Most descriptive studies should be considered
hypothesis-generating rather than causality-focused
whereas analytic studies test a prespecified
hypothesis.

68. Important issues in Study
Design
 Validity: Truth
 External Validity:
 Can the study be generalized to the population
 Internal Validity:
 Results will not be due to chance, bias or
confounding factors
 Symmetry Principle: Groups are similar
 Misclassification

69.  Confounding: distortion of the effect of one risk factor by
the presence of another
 Bias: Any effect from design, execution, & interpretation
that shifts or influences results
 Confounding bias: failure to account for the effect of
one or more variables that are not distributed equally
 Measurement bias: measurement methods differ
between groups
 Sampling (selection) bias: design and execution errors
in sampling

70. Misclassification
 Misclassification is the incorrect categorization of a subject into a
study group.
There are two types of misclassification,
1. Non-differential
2. differential.
 Non-differential misclassification indicates an equal and random
chance that any one subject will be misclassified (or included as
part of the wrong study group).
 Differential misclassification, the chance a subject is misclassified
is nonrandom.

71.  Stage migration, also known as the Will Rogers
phenomenon, is a classic example of
misclassification.
Patients only assessed clinically might be
understaged—categorized as early-stage cancer will
have less survival rate or if overstaged patients were
considered with truly late-stage patients, survival
would be better than in actuality.

72. Characteristics of observational
studies
 Can study risk factors that
have serious consequences
 Study individuals in their
natural environment (>>
extrapolation)
 Possibility of confounding.
 Evaluate the effect of a
suspected risk factor
(exposure) on an outcome(e.g.
disease)  define
‘exposure’ and ‘disease.
 Describe the impact of the risk
factor on the frequency of
disease in a population

73. Cross - Sectional Study
 Exposure and disease measured once,
i.e. at the same point in time
present futurepast
n
exposed ?
diseased ?

74. Cross - Sectional Study
 Random sample from population
 i.e. results reflect reference population
 Estimates the frequencies of both exposure and
outcome in the population
 Measuring both exposure and outcome at one point
in time
 Typically a survey

75. Cross - Sectional Study
 Can study several exposure factors and outcomes
simultaneously
 Determines disease prevalence
 Helpful in public health administration & planning
 Quick
 Low cost (e.g. mail survey)
 Limitation:
 Does not determine causal relationship
 Not appropriate if either exposure or outcome is rare

76. Cohort studies
 Follow-up studies; subjects selected on presence or
absence of exposure & absence of disease at one
point in time. Disease is then assessed for all
subjects at another point in time.
 Typically prospective but can be retrospective,
depending on temporal relationship between study
initiation & occurrence of disease.

77. Cohort Study
 Individuals selected by exposure status and
future occurrence of disease measured
present futurepast
n
Exposed
yes
no
disease ?
disease ?
n
Exposed
yes
no
disease ?
disease ?

78. Cohort studies
 More clearly established
temporal sequence between
exposure & disease
 Allows direct measurement of
incidence
 Examines multiple effects of a
single exposure (OC and breast,
ovarioan cancers)
•Limitations:
•time consuming and
expensive
•loss to follow-up &
unavailability of data
•inefficient for rare diseases

79. Case-Control Study
 Good for rare disease (e.g. cancer)
 Can study many risk factors at the
same time
 Usually low cost
 Confounding likely

80. Case-Control Study Design
Cases
Controls
Exposed
Unexposed
Exposed
Unexposed
TimeData
collection
Direction of inquiry
Q: What happened?

81. Case-Control study
• Study subjects selected on basis of whether
they have (case) or do not have (control) a
disease
• Useful for disease with long latency period
• Efficient in terms of time & costs
• Particularly suited for rare diseases
• Examines multiple exposures to a single
disease 81

82. Case-control study
Limitations:
(1) Susceptible to bias (particularly selection &
recall)
(2) Difficulties in selection of controls
(3) Ascertainment of disease & exposure status
(4) Inefficient for rare exposures unless
attributable risk is high

83. Randomized Controlled Trials
An experimental comparison study where participants are allocated to
treatment/intervention or control/placebo groups using a random
mechanism. Best for studying the effect of an intervention.
Advantages:
 unbiased distribution of confounders
 blinding more likely
 randomisation facilitates statistical analysis
Disadvantages:
 expensive: time and money
 volunteer bias
 ethically problematic at times

84. Meta-Analysis
 Meta-analysis is a technique that pools available published
data in an effort to increase the statistical power of an
analysis.
 QUOROM (Quality of Reporting of Meta-Analyses)and
MOOSE (Meta-Analysis of Observational Studies in
Epidemiology) guidelines have been developed to ensure the
quality and validity of results obtained through meta-analysis

85. HOW WERE THE DATA ANALYZED?
 A continuous variable is one that can take on an infinite number
of values. Age and length of stay are examples of a continuous
variable.
 Categorical variables have discrete values.
 The simplest categorical variable is a binary variable that can
only take on one of two values, such as sex [male, female]).
 Ordinal variables are ordered categorical variables. Cancer
stage is a classic example of an ordinal categorical variable.
 Nominal variables are unordered categorical variables, such
as race.

87. Hypothesis Testing
 Hypothesis testing uses comparative or analytic statistics to
determine whether observed differences between two or more
groups are real or are attributable to chance.
 The P value is a statistical summary measure for hypothesis
testing.
 A significance level of 5% (P = .05) is widely used to indicate a
statistically significant finding, although this value is arbitrary.
 P value is interpreted as the probability that the observed
difference in outcomes between groups is the result of chance.

88.  For smaller studies: Hypothesis testing is also
possible by examining 95% confidence interval
 A wide CI indicates a lack of precision, whereas
a tight (small) interval would be indicative of
minimal uncertainty

89. Two types of errors can occur with hypothesis testing.
 An alpha (or type I) error occurs when one
observes a difference in outcomes when one does
not actually exist.
 A beta (or type II) error occurs when no difference
in outcomes is observed when a difference truly
exists (a false-negative finding).

90.  The unpaired t-test is used to compare two independent
groups that have continuous outcome variables.
 Paired t-test is used to compare two dependent groups that
have continuous outcome variables.
 An analysis of variance (ANOVA) is used when comparing
more than two groups with a continuous outcome variable.
 The chi-square statistic is often used to compare the
distributions of two or more groups with categorical
outcome variables.
 Fisher’s exact test is more appropriate for such
comparisons when the sample size is small

91. Multivariable Analysis
 Multivariate regression models are among the most
commonly used methods to evaluate the relationship between
variables and outcomes in the absence of the influence of
other measured variables.
 Linear regression is used to evaluate the relationship
between factors potentially associated with a continuous
outcome variable, such as length of stay.
 Logistic regression is used when the outcome variable is
binary (e.g., operative mortality)

92. Propensity Score Analysis
 Propensity score analysis is an alternative method of
risk adjustment. When two groups are being
compared, logistic regression is used to calculate a
given subject’s risk or probability (or propensity) of
having an exposure of interest (e.g., minimally
invasive as compared with open surgery).
 propensity scores are not superior to multivariate
techniques, but are simply an adequate alternative

93. Instrumental Variable
Analysis
 Instrumental variable analysis is another method of
accounting for unmeasured confounding and controlling
bias.
 The principle underlying this type of analysis is that there
are unmeasured, or immeasurable, confounders that might
bias the study’s results.
 The best example of an instrumental variable is
randomization.

94. ARE THERE ETHICAL
CONSIDERATIONS?
 In the first instance, common sense is
the best guide to whether or not a study
is ethical.
 Still important to seek advice from an
ethics committee whenever research is
contemplated.

95. ANALYSING A SCIENTIFIC
ARTICLE

96. Take Home Message
 Keep audits simple
 Get everyone involved
 Do not confuse clinical audit with research.
 Take care with statistics – errors can lead to inaccurate conclusions
 Share learning - tell everyone about your audit
 Tell ‘The Organisation’ about your audit
 Re-audit to ensure improvement in clinical care

97.  Only by becoming more critical evaluators
of the surgical literature will the next
generation of surgeons be able to embrace
fully the promise of evidence based surgery.

98. References
 Bailey and love's short practice of surgery 26th ed.
 Sabiston textbook of surgery, 19th edition.
 Principles for best practice in clinical audit-national institute
for clinical excellence.
 Clinical audit: handbook.
 A practical guide to clinical audit-the national clinical audit
advisory group