Clinical Audit SSU March 2008 Contact Session 1

Introduction What is clinical audit? Audit versus research The audit cycle Five stages of clinical audit Summary

What is clinical audit?

Audit versus research

The audit cycle

Five stages of clinical audit

Summary

What is clinical audit? “ The systematic critical analysis of the quality of medical care, including the procedures used for diagnosis and treatment, the use of resources and the resulting outcome and quality of life for the patient” Working for Patients 1989

“ The systematic critical analysis of the quality of medical care, including the procedures used for diagnosis and treatment, the use of resources and the resulting outcome and quality of life for the patient”

Working for Patients 1989

Reactions to audit? Enthusiasm Cynicism Obligation Doubt Exhaustion

Enthusiasm

Cynicism

Obligation

Doubt

Exhaustion

What is clinical audit? Clinical audit is at the heart of clinical governance and clinical effectiveness

Clinical audit

is at the heart of

clinical governance and clinical effectiveness

Audit:- are we doing the best thing in the best way? Measures current practice against specific standards Never experimental Uses data in existence by virtue of practice May require ethical approval Aims to improve delivery of patient care

Measures current practice against specific standards

Never experimental

Uses data in existence by virtue of practice

May require ethical approval

Aims to improve delivery of patient care

Research:- What is the best thing to do/the best way to do it? Provides sound basis for clinical audit Involves experimental trials Uses detailed and often sophisticated data collection Needs ethical approval and registration Aims to add to body of scientific knowledge

Provides sound basis for clinical audit

Involves experimental trials

Uses detailed and often sophisticated data collection

Needs ethical approval and registration

Aims to add to body of scientific knowledge

The audit cycle Problem or objective identified

The audit cycle Problem or objective identified Criteria agreed and standards set

The audit cycle Problem or objective identified Criteria agreed and standards set Audit (Data collected)

The audit cycle Problem or objective identified Criteria agreed and standards set Audit (Data collected) Identify areas for improvement

The audit cycle Problem or objective identified Criteria agreed and standards set Audit (Data collected) Identify areas for improvement Make necessary changes

The audit cycle Problem or objective identified Criteria agreed and standards set Audit (Data collected) Identify areas for improvement Make necessary changes Re-audit

The audit cycle Problem or objective identified Criteria agreed and standards set Audit (Data collected) Identify areas for improvement Make necessary changes Re-audit

The audit cycle

Five stages of clinical audit

Stage 1: Preparing for audit Selecting a topic Defining the purpose Planning

Selecting a topic

Defining the purpose

Planning

Suitable topic Determine which aspects of current work are to be considered Describe and measure present performance Develop explicit standards Decide what needs to be changed Negotiate change Mobilise resources for change Review and renew the process

Determine which aspects of current work are to be considered

Describe and measure present performance

Develop explicit standards

Decide what needs to be changed

Negotiate change

Mobilise resources for change

Review and renew the process

Selecting a topic There seems little point in trying to audit a rare condition, with a cheap intervention with a fairly superficial outcome. Which topic?

There seems little point in trying to audit a rare condition, with a cheap intervention with a fairly superficial outcome.

Which topic?

Selecting a topic High cost/volume/risk Area of local clinical concern Mis-diagnosing MI Pre elective screen right tests right patient Potential for improvement National priority Organisational priority Evidence based

High cost/volume/risk

Area of local clinical concern

Mis-diagnosing MI

Pre elective screen right tests right patient

Potential for improvement

National priority

Organisational priority

Evidence based

Defining the purpose The following series of verbs may be useful in defining the aims of an audit (Buttery, 1998): to improve to enhance to increase to change to ensure

The following series of verbs may be useful in defining the aims of an audit (Buttery, 1998):

to improve

to enhance

to increase

to change

to ensure

Planning Involve ALL the right people Time and resources Access the evidence Methodology Pilot Report and Action Re-audit

Involve ALL the right people

Time and resources

Access the evidence

Methodology

Pilot

Report and Action

Re-audit

Stage 2: Selecting criteria Defining criteria Sources of evidence Appraising the evidence

Defining criteria

Sources of evidence

Appraising the evidence

Defining criteria Criteria: are explicit statements that define what is being measured represent elements of care that can be measured objectively.

Criteria:

are explicit statements that define what is being measured

represent elements of care that can be measured objectively.

Sources of evidence NeLH /Cochrane/WISH NICE Official Websites NSFs Local or regional guidelines/policy Royal College or Professional Body Recognised journals

NeLH /Cochrane/WISH

NICE

Official Websites

NSFs

Local or regional guidelines/policy

Royal College or Professional Body

Recognised journals

Appraising the evidence Aim /objectives Methodology Results /conclusions Applicable to your patient group Bias/ causes for concern

Aim /objectives

Methodology

Results /conclusions

Applicable to your patient group

Bias/ causes for concern

Stage 3: Measuring level of performance Planning data collection Methods of data collection Handling data

Planning data collection

Methods of data collection

Handling data

Planning data collection Need to establish the user group to be included, with any exceptions noted the healthcare professionals involved in the users ’ care the time period over which the criteria apply.

Need to establish

the user group to be included, with any exceptions noted

the healthcare professionals involved in the users ’ care

the time period over which the criteria apply.

Methods of data collection Computer stored data Case notes/Medical Records Surveys Questionnaires Interviews Focus Groups Prospective recording of specific data

Computer stored data

Case notes/Medical Records

Surveys

Questionnaires

Interviews

Focus Groups

Prospective recording of specific data

Handling data Health service professionals must be aware of the ethical implications of and their responsibilities under the Data Protection Act (1998) when collecting data and presenting results.

Health service professionals must be aware of the ethical implications of and their responsibilities under the Data Protection Act (1998) when collecting data and presenting results.

Stage 4: Making improvements Identifying barriers to change Implementing change External relationships

Identifying barriers to change

Implementing change

External relationships

Identifying barriers to change Fear of change Lack of understanding Low morale Poor communication Culture Pushing too hard Consensus not gained

Fear of change

Lack of understanding

Low morale

Poor communication

Culture

Pushing too hard

Consensus not gained

Implementing Change Discuss the results with those likely to be affected Agree an Action Plan Clearly define – who is doing what Check progress Produce report and disseminate Share findings and changes in practice

Discuss the results with those likely to be affected

Agree an Action Plan

Clearly define – who is doing what

Check progress

Produce report and disseminate

Share findings and changes in practice

Stage 5: Sustaining improvement Monitoring and evaluation Re-audit Maintaining and reinforcing improvement

Monitoring and evaluation

Re-audit

Maintaining and reinforcing improvement

Monitoring and evaluation Although improving performance is the primary goal of audit, sustaining that improvement is also essential. Indeed, any systematic approach to changing professional practice should include plans to: monitor and evaluate the change maintain and reinforce the change ( NHS Centre for Review and Dissemination, 1999 ).

Although improving performance is the primary goal of audit, sustaining that improvement is also essential. Indeed, any systematic approach to changing professional practice should include plans to:

monitor and evaluate the change

maintain and reinforce the change ( NHS Centre for Review and Dissemination, 1999 ).

Re-audit Close the loop Review evidence Measure effectiveness Decide how often to re-audit

Close the loop

Review evidence

Measure effectiveness

Decide how often to re-audit

The great coffee audit A case study

A case study

The great coffee audit Problem The doctors feel that their coffee isn’t hot enough after slogging through morning surgery

Problem

The doctors feel that their coffee isn’t hot enough after slogging through morning surgery

The great coffee audit Problem Criteria The doctors feel that their coffee isn’t hot enough after slogging through morning surgery The coffee shall be hot and satisfying to the hard pressed docs

Problem

Criteria

The doctors feel that their coffee isn’t hot enough after slogging through morning surgery

The coffee shall be hot and satisfying to the hard pressed docs

The great coffee audit Problem Criteria Standards The doctors feel that their coffee isn’t hot enough after slogging through morning surgery The coffee shall be hot and satisfying to the hard pressed docs The coffee shall be served at a temperature of 85-90  C on 80% of occasions and there will be 90% satisfaction level expressed by the docs

Problem

Criteria

Standards

The doctors feel that their coffee isn’t hot enough after slogging through morning surgery

The coffee shall be hot and satisfying to the hard pressed docs

The coffee shall be served at a temperature of 85-90  C on 80% of occasions and there will be 90% satisfaction level expressed by the docs

The great coffee audit Methods The junior receptionist shall check the temperature of the coffee daily for two weeks and circulate a questionnaire to the partners asking them to score a coffee satisfaction level between 1 and 10. The practice manager shall visit Tesco’s and interview the manager about the availability, costs, quality and sell-by dates of the coffee brands available

Methods

The junior receptionist shall check the temperature of the coffee daily for two weeks and circulate a questionnaire to the partners asking them to score a coffee satisfaction level between 1 and 10. The practice manager shall visit Tesco’s and interview the manager about the availability, costs, quality and sell-by dates of the coffee brands available

The great coffee audit Review After a rather tense audit team meeting it was found that the coffee temperature fell below 37  C on at least 33% of occasions and reached the standard on only 10% of occasions. The doctors scored the coffee at an average 3/10 and two expressed it undrinkable. The practice manager reported the results of her Tesco’s visit.

Review

After a rather tense audit team meeting it was found that the coffee temperature fell below 37  C on at least 33% of occasions and reached the standard on only 10% of occasions. The doctors scored the coffee at an average 3/10 and two expressed it undrinkable. The practice manager reported the results of her Tesco’s visit.

The great coffee audit Change It was agreed to replace the aged coffee maker ( after agreeing suitable redundancy terms for the senior receptionist) with a shiny new machine from Argos. As an additional ‘quality initiative’, cream cakes would be served after surgery. The coffee contract would be switched from the corner shop to Tesco PLC Trust

Change

It was agreed to replace the aged coffee maker ( after agreeing suitable redundancy terms for the senior receptionist) with a shiny new machine from Argos. As an additional ‘quality initiative’, cream cakes would be served after surgery. The coffee contract would be switched from the corner shop to Tesco PLC Trust

The great coffee audit Re-audit For a further two weeks it was agreed to measure the coffee temperature and re-circulate the questionnaire. It was gratifying to find 100% correlation with agreed standards with the exception of one partner who didn’t like coffee anyway.

Re-audit

For a further two weeks it was agreed to measure the coffee temperature and re-circulate the questionnaire. It was gratifying to find 100% correlation with agreed standards with the exception of one partner who didn’t like coffee anyway.

The great coffee audit Re-audit Future audit For a further two weeks it was agreed to measure the coffee temperature and re-circulate the questionnaire. It was gratifying to find 100% correlation with agreed standards with the exception of one partner who didn’t like coffee anyway. Cost implications of standard maintenance Cholesterol assays for partners

Re-audit

Future

audit

For a further two weeks it was agreed to measure the coffee temperature and re-circulate the questionnaire. It was gratifying to find 100% correlation with agreed standards with the exception of one partner who didn’t like coffee anyway.

Cost implications of standard maintenance

Cholesterol assays for partners

A word about clinical governance “ A framework through which NHS organisations are accountable for continuously improving the quality of their services and safeguarding high standards by creating an environment in which excellence in clinical care will flourish” A First Class Service 1998

“ A framework through which NHS organisations are accountable for continuously improving the quality of their services and safeguarding high standards by creating an environment in which excellence in clinical care will flourish”

A First Class Service 1998

‘Clinical Governance’

‘Clinical Governance’ Research

‘Clinical Governance’ Research Audit

‘Clinical Governance’ Research Audit Evidence based medicine

‘Clinical Governance’ Research Audit Evidence based medicine Dissemination of guidelines

‘Clinical Governance’ Research Practice development plans Audit Evidence based medicine Dissemination of guidelines

‘Clinical Governance’ Research Practice development plans Audit Evidence based medicine Dissemination of guidelines Personal development plans

‘Clinical Governance’ Research Practice development plans Postgraduate medical education Audit Evidence based medicine Dissemination of guidelines Personal development plans

‘Clinical Governance’ Research Practice development plans Postgraduate medical education Audit Evidence based medicine Dissemination of guidelines Personal development plans Practice accreditation

‘Clinical Governance’ Research Practice development plans Postgraduate medical education Audit Evidence based medicine Dissemination of guidelines Personal development plans Practice accreditation Special interest groups

‘Clinical Governance’ Accountability

Summary Audit Defined clinical audit Compared audit and research The audit cycle. Five stages of clinical audit

Defined clinical audit

Compared audit and research

The audit cycle.

Five stages of clinical audit

Celebrate Next contact session progress report from you Final contact session Share learning Publicise results Give credit where credit is due! Work on the assignment and poster

Next contact session progress report from you

Final contact session

Share learning

Publicise results

Give credit where credit is due!

Work on the assignment and poster