Pharmacy: Criterion audit

1,729 views

Published on

Valerie Reilly for Pharmacy Directorate
NHS Education for Scotlan

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,729
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
23
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Hello my name is Val Reilly I’d like to describe the process of conducting and completing a Criterion Audit
  • I’m going to talk about what is meant by the term criterion audit, the audit proforma and just what is involved in completing it. We’ll look at choosing an audit subject, defining criteria and setting standards in a little more detail Finally I’ll describe the peer review process, how you can you integrate a criterion audit into your practice and the benefits this can bring you and the team you work with.
  • Before we go any further lets get our terms clarified. The National Institute for Clinical excellence (NICE) defines clinical audit as a quality improvement process that seeks to improve patient care and outcomes through systematic review of care against explicit criteria and the implementation of change. Aspects of the structure, processes, and outcomes of care are selected and systematically evaluated against explicit criteria. Where indicated, changes are implemented at an individual, team, or service level and further monitoring is used to confirm improvement in healthcare delivery So we can see that the NICE definition is describing the use of criteria as an integral part of the clinical audit process and in this presentation the terms clinical audit and criterion audit are interchangeable
  • While I’m talking about definitions just a quick word about research, service development and audit Simply put Research asks ‘What should we do?’ and Service Development asks ‘How do we put research into practice?’ Audit comes along after research and service development has taken place it asks the question ‘ Are we doing what we should be doing (or what we think we are doing)?’ It is important to be clear about the distinction between these activities as a research project will probably need ethics clearance. However, if you ever have any doubt your local research and ethics committee will usually be happy to give advice.
  • Here is a classic diagram representing the audit cycle It illustrates a circular process where criteria and standards are set, practice is measured and then compared with those criteria and standards and change is implemented if necessary.
  • But audit is better described as a spiral process where each audit/re-audit cycle is a loop of the spiral Stages 1 to 4 describe the classic audit re-audit cycle Criteria and standards are set, practice is assessed and compared with the criteria and standards. Changes are made to practice if necessary and re audit is applied In clinical settings an audit is often not a one off project it may be necessary to repeat the process at regular intervals. This maybe because the activity audited is associated with high risk to the organisation for example the hand washing audits conducted by infection control teams. Or criteria may be modified and standards changed in response to new research, guidelines or critical incidents for example guidance on the use of concomitant use of aspirin and clopidogrel after angioplasty. Step 5 describes the start of the audit cycle again.
  • I’ve already said that for our purposes the terms clinical audit and criterion audit are interchangeable. But exactly what do we mean by a criterion audit? For our purposes it’s a structured audit process followed by using a proforma available on the NES website It may comprise the completion of 5 or 8 sections of the proforma but only the completion of 8 sections represents the full audit /re audit cycle Criterion audit isn’t new it’s been around for some time now and it is a process followed by other clinical disciplines Completion of a criterion audit forms part of our GP colleagues reaccreditation process I’ve already said that for our purposes the terms clinical audit and criterion audit are interchangeable. But exactly what do we mean by a criterion audit? For our purposes it’s a structured audit process followed by using a proforma available on the NES website It may comprise the completion of 5 or 8 sections of the proforma but only the completion of 8 sections represents the full audit /re audit cycle Criterion audit isn’t new it’s been around for some time now and it is a process followed by other clinical disciplines Completion of a criterion audit forms part of our GP colleagues reaccreditation process
  • Here is a list of the 8 sections of the proforma The form is easy to download from the NES website to be completed offline It is filled in, in much the same fashion as an SEA report You can type straight onto the form or cut and paste ready prepared text. It also includes some useful guidance Lets have a look at the actual sections in more detail and you’ll see what I mean
  • This first section should be used to describe the background to your audit. You should explain clearly and simply why you are conducting the audit and don’t forget to mention any relevant guidelines or local policies that you may be drawing your criteria and standards from. It is important that the potential for change is described as there is little reason for conducting an audit when no improvements to practice are possible or required You should also describe how you may effect or influence any improvements to practice to highlight the audit’s relevance to your practice.
  • There are a number of reasons why you may wish to conduct an audit on a particular subject It may be a National Priority such as infection control, a commonly occurring event such as running out of stock of a particular drug It may be about the dispensing or administration of a high risk drug such as methotrexate or lithium or there may be an area of clinical concern which may have been highlighted by a significant event such as administration of IV paracetamol Changes in practice may prompt an audit to check compliance with new developments High cost of a drug or procedure may make it of interest to an organisation to audit Whatever subject is chosen any evidence base such as local policy or SIGN or NICE guidelines should be referred to and there should be a clear potential for change in practice for example There would be no point in auditing the prescribing and administration of an expensive drug if you or your team had no influence over the process or if it’s use was so specialist you wouldn’t be in a position to judge whether it was appropriate or not. However, there may be a lot of benefit to be gained from auditing the prescribing and administration of drugs for which there is a robust evidence base and for which you have direct clinical responsibility such as a clinical pharmacist auditing the use of inhalers on a respiratory ward
  • Criteria are simple, logical statements used to describe a definable and measurable item of health care The proforma provides an example Patients with type II diabetes should have a fundoscopy every 12-months. You may only need one or two of these as they can be very broad based statements
  • An audit standard describes the level of care to be achieved for any particular criterion And again the proforma provides an example of how a standard may be defined 90% of Patients with type II diabetes should have a fundoscopy every 12-months. Its very similar to the criterion but as you can see this statement includes a target figure of the number of patients who should have a fundoscopy It’s very important to set a realistic target figure to achieve in your audit. In practice you will never get 100% of patients to attend for fundoscopy even if you sent a car around to their house to take them to the appointment. On the other hand you should be wary of setting too easy a target or your audit might not do anything to contribute to improved patient care or safety Local policies or guidelines or the results of a previous audit may help you decide where you set your target If you really can’t decide or a suitable figure isn’t clear discuss what might be a reasonable target with your colleagues or individuals who may be interested and involved in the subject, set a figure see what your audit achieves and review your standard for the second round of the audit if necessary
  • The proforma gives you examples of a criterion and standard refer to these when you are defining yours They may look very similar and it can be tricky getting these right. It’s worth taking a little time on defining these because they provide the focus for your work The slide shows some examples which might help Patients should be prescribed aspirin 75mg and clopidogrel 75mg, if clinically appropriate after an angioplasty, for no longer than 12 months post procedure. Patients should have their prescribed medicines dispensed within 48 hours of presenting a GP10 at a community pharmacy 100% of patients being prescribed concomitant aspirin 75mg and clopidogrel 75mg should have their notes assessed every six months to determine the reason for the therapy and the length of time they have been prescribed it. No more than 25% patients prescribed concomitant aspirin 75mg and clopidogrel 75mg therapy should be prescribed this therapy for longer than 4 months post acute coronary syndrome alone, or 15 months post stenting procedure with or without an incident of ACS. 100% of patients prescribed concomitant aspirin 75mg and clopidogrel 75mg after a stroke should have this changed to aspirin 75mg alone 2 months post discharge from hospital secondary care.
  • 90% of patients being prescribed concomitant aspirin 75mg and clopidogrel 75mg should have their notes assessed every six months to determine the reason for the therapy and the length of time they have been prescribed it. 95% of all prescriptions presented to a pharmacy should be available for collection by the patient, if required, within 48 hours.
  • This is just your method really It is possible that you might prepare and plan an audit all by yourself but the power and success of any audit activity is team working. A learning process derived from an audit by a team process will help ensure that any changes to practice identified will be made and maintained. So ideally you should demonstrate evidence of team involvement and mention any discussions you may have with interested parties such as nurses, medics your colleagues or even your boss.
  • Results of your audit should be clearly stated. Whether this is in the form of a graph, table , chart or simply stated text will depend on your project but keep these simple if you can It is important that however you display your results you relate them to your standards and it is easy to see whether the standard was achieved or not For example if your standard states ‘90% of patients prescribed a salbutamol inhaler as an inpatient on a general medical ward should be given a demonstration of inhaler technique by the ward pharmacist’ Your results should clearly state how many patients on the ward in total were prescribed a salbutamol inhaler, how many of these received a demonstration of inhaler technique and what the percentage was compared to the standard of 90%. Don’t make the person reading or reviewing your audit work it out Might be an idea here to show some simple results on the screen shot
  • This is a very important section and where evidence of team involvement is a key factor Simply describe what you and your team have done to improve performance. Describe any meetings with colleagues and partners any education or training delivered and any guidelines or policies developed or amended Try and describe processes undertaken as simply as possible It is possible that you may conduct an audit and demonstrate that you are performing above the standard you set. This may suggest that the audit wasn’t needed in the first place or most likely that you have set your standard too low. If this is the case describe the processes and discussions that you undertake to reset and agree a new higher standard
  • To demonstrate the full audit cycle all 8 sections of the criterion audit should be completed Make your changes repeat the audit method and describe the second round of results Display these in the same way as the first set of results and clearly relate these back to the first round of results
  • This is the last section of the criterion audit Obviously you will comment on your practice and whether it was worthwhile undertaking the audit You should also compare your results from the first round of audit to those of the second State whether any improvements were made. If a standard wasn’t achieved discuss why this might have happened Include any recommendations for repeated audit in future e.g. this should be repeated annually, repeated after further staff training has been conducted Mention any improvements which may be made to your processes in future and be undertaken before re audit if you know about them for example you may know that a new policy is being developed which may impact on this particular area of practice in future Mention any related audit work which might be undertaken as a result of this project Can we bullet point on this screen shot?
  • While audit activity is always worthwhile, the benefits of conducting a criterion audit are derived by the educational feedback provided by the peer review process A small network of trained pharmacists which i ncludes pharmacists from community, primary care, hospital, academia, education and management will review your criterion audit after you have submitted it to NES. They come from a variety of backgrounds and may not necessarily come from your particular pharmacy discipline They will use a pro forma to assess your audit and return comments to NES with educational feedback. NES will collate the outcome of peer review & educational feedback and send it back to the submitting individual for their consideration The focus of feedback is on the individual/author, but the process of reflection and learning naturally encourages team learning so the feedback may be used to share with the wider team. I think it will be helpful to discuss the peer review process and pro forma used in a little more detail so you can see just what the reviewers will be looking for
  • This section relates directly to the first section of your audit The reviewer will evaluate whether there was a clear potential for change described and whether the project was relevant to the authors practice That’s why it’s so important to describe clearly why you are undertaking the audit and why and how you may effect or influence improvements to current practice. Don’t get too hung up on what kind of mark the reviewer might select. You’ll see as we go through it that each section on the review form has space for the reviewer to make comments They may not necessarily have a comment to make in every section but what is important is that when they do these will be constructive The reviewer will not want to shoot you down in flames this is an educational process NES will share these with you when they report back
  • In this section the reviewer will offer an assessment of the criteria and standards selected Selecting criteria and standards can be difficult they won’t mark an audit down just because you didn’t get the wording of these quite right, but they will try and demonstrate how a criterion or standard might be reworded to make them clearer if possible
  • The reviewer will assess the preparation and planning for the audit in this section The most important thing they will be looking for is evidence that other people were involved in the audit for example a description of discussions with colleagues, medical staff, nurses or team meetings.
  • In this section the reviewer will offer their assessment of how the audit’s results have been presented. Lets be clear here results of an audit don’t have to be offered in a particularly sophisticated format with coloured graphs and pie charts The reviewer will be looking for a clear comparison of the audit results to the standards set so as I said earlier if you have a standard expressing a percentage of patients state the audit result as a percentage. Don’t make them have to reach for a calculator to work it out for themselves.
  • This is the section were the reviewer will give an assessment of any changes to practice described in the audit. As with the preparation and planning of the audit they will place a very high value on evidence of teamwork. They will also be looking for a clear example of an actual change to practice as a result of the audit. That’s why it is helpful to keep your description of changes made after the first round of your audit as simple as possible
  • The key thing here that the reviewer will assess is the comparison and discussion of results from the first round of audit and the second
  • There are many benefits to be gained from conducting an audit Obviously patients and staff will benefit from improvements to clinical and working practice For the person conducting the audit there are benefits to their continuing professional development. A NES criterion audit may usefully be used as documentary evidence of CPD The process of change and improvements to practice described in the audit underpins its educational value If conducted thoroughly the audit will have built teamwork and team learning And most importantly audit is a key clinical governance tool The importance of demonstrating commitment to clinical governance should never be underestimated
  • You may be preparing to conduct a criterion based audit as part of an evaluation process Okay so you have to do one and you will probably be able to set aside some of your working time to conduct it on that basis However once you’ve cut your teeth on your first audit considering the benefits I’ve just mentioned you might want to continue to conduct audit on a regular basis. Most of us are working in positions where time is precious resource but you can plan to integrate audit into your everyday practice Criterion audit can be used to support the development of junior colleagues. So if you are in a position where you are required to set objectives and discuss the personal development plans of junior staff you might consider asking them to conduct audit with the benefit of your experience and review from NES it would be a particularly good learning experience for them As I have said criterion audit can be part of your CPD plans. If you work in the NHS you could identify audit activity as part of your objectives and personal development plan when reviewing these with your manager as part of the KSF process. That way it becomes an identifiable part of your workload and as with evaluation means that you can justify spending some if not all of your working time on it. Audit is an important clinical governance tool It may be conducted after something goes wrong or if it is felt that working practices just aren’t up to standard but it is most effective when it is part of regular quality assurance processes You could discuss with the team you work with which regular audits might be conducted to improve clinical and working practice. Spreading the work for these across several team members would make it easier to achieve For example you may want to regularly audit SOPs and protocols used by dispensary staff to ensure they are up to date or conduct an audit of items owed on a periodic basis to ensure that stock management is optimised
  • Add cheesy picture? So in conclusion The conduct of audit clearly delivers benefits To an organisation or service by improving patient safety or working practice And to the individual as it is a first class way to demonstrate CPD Conducting a NES criterion based audit has added benefits in that further educational feedback is provided by experienced reviewers which provides strong documentary evidence of learning And although we are all very busy people there are ways in which audit can be integrated into our working practice without necessarily adding an onerous challenge to our time management. The key to a successful audit is teamwork So although you may be the one who writes the audit up and reports the results you shouldn’t necessarily conduct the project by yourself, there’s no harm in letting your colleagues do some of the work for you. I hope this has been helpful to you and I wish you success with your criterion audit, I am sure it will be worthwhile project and a useful learning experience for you. Thankyou
  • Pharmacy: Criterion audit

    1. 1. Audit Criterion audit Valerie Reilly for Pharmacy Directorate NHS Education for Scotland
    2. 2. Objectives <ul><li>After listening to this presentation you should be able to; </li></ul><ul><li>Discuss the meaning of the term ‘Criterion Audit’ </li></ul><ul><li>Describe the Criterion Audit proforma for submitting an audit for peer review </li></ul><ul><li>Describe 3 aspects of what a Criterion Audit involves </li></ul><ul><li>Decide how to choose a subject to audit </li></ul><ul><li>Define the terms ‘Criteria’ and ‘Standards’ </li></ul><ul><li>Describe the Peer Review Process </li></ul><ul><li>Discuss with others how you can you integrate this into your practice and the benefits of completing and submitting an audit for peer review. </li></ul>
    3. 3. Definition of audit <ul><li>‘ Clinical audit is a quality improvement process that seeks to improve patient care and outcome, through the systematic review of patient care against explicit criteria and the implementation of change. </li></ul><ul><li>Aspects of the structure, processes, and outcomes of patient care are selected and systematically evaluated against explicit criteria. </li></ul><ul><li>Where indicated, changes are implemented at an individual, team, or service level and further monitoring is used to confirm improvement in healthcare delivery’. The National Institute for Clinical Excellence </li></ul>
    4. 4. Audit versus Research <ul><li>Research asks </li></ul><ul><li>What should we do? </li></ul><ul><li>Service Development asks </li></ul><ul><li>How do we put our research into practice? </li></ul><ul><li>Audit asks </li></ul><ul><li>Are we doing what we should be doing (or what we think we are doing) </li></ul>
    5. 5. Audit cycle Define criteria/standards Implement change Measure practice Compare with criteria/standards
    6. 6. It’s really more a continuous spiral process….. Define criteria/standard Assess practice Repeat audit Re audit Compare practice with criteria/standard and change if necessary
    7. 7. What is a Criterion Audit ? <ul><li>Structured audit process </li></ul><ul><li>Audit pro forma available on NES Pharmacy website </li></ul><ul><li>5 or 8 parts </li></ul><ul><li>8 parts completed comprise the full audit /re-audit cycle </li></ul><ul><li>Part of our GP colleagues accreditation process </li></ul>
    8. 8. The sections of a Criterion Audit <ul><li>Reason for the Audit </li></ul><ul><li>Criteria to be measured </li></ul><ul><li>Standards set </li></ul><ul><li>Preparation and Planning </li></ul><ul><li>Data Collection 1 </li></ul><ul><li>Description of Change </li></ul><ul><li>Data Collection 2 </li></ul><ul><li>Conclusions </li></ul>
    9. 9. The Reason for the Audit <ul><li>Explain why the audit topic was chosen and that as a result of this choice </li></ul><ul><li>there is the potential for change to be introduced which is relevant to the </li></ul><ul><li>practice or you as an individual practitioner. </li></ul>
    10. 10. Potential reasons for an audit <ul><li>National priority </li></ul><ul><li>Common </li></ul><ul><li>High risk – </li></ul><ul><li>Clinical concern/ problems – risk of ADRs, interactions </li></ul><ul><li>Changes in practice - new development/evidence </li></ul><ul><li>High cost </li></ul><ul><li>Evidence base </li></ul><ul><li>Potential for change </li></ul>
    11. 11. Defining Criteria <ul><li>Criteria are simple, logical statements used to describe a definable and </li></ul><ul><li>measurable an item of health care. </li></ul><ul><li>e.g. Patients with type II diabetes should have a fundoscopy every 12-months. </li></ul><ul><li>See Audit Guidance for examples of criteria if greater understanding is </li></ul><ul><li>required. </li></ul><ul><li>Focusing on one or two criteria makes data collection more manageable </li></ul><ul><li>and the introduction of small changes to practice less challenging. </li></ul><ul><li>Where available, evidence should be cited in support of criteria eg. nGMS </li></ul><ul><li>Contract or a clinical guideline . </li></ul><ul><li>A single criterion is acceptable for Appraisal purposes. </li></ul>
    12. 12. Defining Standards <ul><li>An audit standard describes the level of care to be achieved for any </li></ul><ul><li>particular criterion </li></ul><ul><li>90% of Patients with type II diabetes should have a fundoscopy every 12-months. </li></ul><ul><li>Standard levels may be influenced by the target levels contained in the nGMS </li></ul><ul><li>contract or by discussing and agreeing the desired or ideal level of care with </li></ul><ul><li>colleagues. </li></ul><ul><li>State how long you estimate it will take you to reach your chosen </li></ul><ul><li>standard(s) eg. 3 months. </li></ul>
    13. 13. Selecting and defining criteria <ul><li>Patients should be prescribed aspirin 75mg and clopidogrel 75mg, if clinically appropriate after an angioplasty, for no longer than 12 months post procedure. </li></ul><ul><li>Patients should have their prescribed medicines dispensed within 48 hours of presenting a GP10 at a community pharmacy. </li></ul>
    14. 14. Selecting and defining standards <ul><li>90% of patients being prescribed concomitant aspirin 75mg and clopidogrel 75mg should have their notes assessed every six months to determine the reason for the therapy and the length of time they have been prescribed it. </li></ul><ul><li>95% of all prescriptions presented to a pharmacy should be available for collection by the patient, if required, within 48 hours. </li></ul>
    15. 15. Preparation and planning <ul><li>Explain briefly who was involved in discussing and planning the audit. </li></ul><ul><li>How the data were identified, collected, analysed, and disseminated ? </li></ul><ul><li>Who gave you assistance at any stage of the project ? </li></ul><ul><li>-with a literature </li></ul><ul><li>-review or with collecting or analysing data (if this was required). </li></ul><ul><li>Teamwork is essential to audit and evidence of this should be provided in </li></ul><ul><li>the report. </li></ul>
    16. 16. Results 1 <ul><li>Initial data collected should be presented using simple descriptive statistics </li></ul><ul><li>as part of the text, in table format or using graphs (e.g. bar charts, pie </li></ul><ul><li>charts) </li></ul><ul><li>Quote actual numbers (n) as well as the percentages (%). </li></ul><ul><li>There is no need to quote irrelevant data (e.g. age, gender, or past </li></ul><ul><li>medical history) if it bears no relation to your chosen audit criteria. </li></ul><ul><li>Compare and contrast your initial data with the standard(s) you set. </li></ul>
    17. 17. Description of change <ul><li>Presentation of data should be as Data One. </li></ul><ul><li>In this section, compare and contrast the results of the second data </li></ul><ul><li>collection with data collection one and the standard(s) you originally set. </li></ul><ul><li>Has your standard been met or surpassed? If not, comment on why you </li></ul><ul><li>think that is the case. </li></ul>
    18. 18. Results 2 <ul><li>Second set of data collected should be presented using simple descriptive </li></ul><ul><li>statistics as part of the text, in table format or using graphs (e.g. bar </li></ul><ul><li>charts, pie charts). </li></ul><ul><li>Compare to Results 1. </li></ul>
    19. 19. Conclusions <ul><li>The final section should briefly and simply summarise what the audit </li></ul><ul><li>Achieved. </li></ul><ul><li>What were the main learning points gained from this </li></ul><ul><li>exercise? </li></ul><ul><li>In doing this, the benefits achieved through the audit should be </li></ul><ul><li>discussed along with any problems encountered with the process or </li></ul><ul><li>findings. </li></ul><ul><li>Some thought should also be given as to whether the audit will be </li></ul><ul><li>repeated in future and if so when. </li></ul>
    20. 20. The Peer Review Process <ul><li>A small network of trained pharmacists </li></ul><ul><li>Returned with educational feedback </li></ul><ul><li>Outcome of peer review & educational feedback collated and sent to submitting individual </li></ul><ul><li>Focus of feedback is on individual/author </li></ul>
    21. 21. Reason for the choice of audit
    22. 22. Criterion chosen and standards set
    23. 23. Preparation and planning
    24. 24. Data Collection
    25. 25. Changes to be evaluated
    26. 26. Project conclusions
    27. 27. Benefits of conducting an audit <ul><li>Patients benefit from improvements to clinical practice </li></ul><ul><li>Staff benefit from improvements to working practice </li></ul><ul><li>Supports individual learning and CPD </li></ul><ul><li>Builds teamwork </li></ul><ul><li>Supports team learning </li></ul><ul><li>Demonstrates commitment to clinical governance </li></ul>
    28. 28. How do I integrate audit into my practice? <ul><li>Undertake as part of evaluation process </li></ul><ul><li>As part of junior staff development </li></ul><ul><li>Form part of CPD </li></ul><ul><li>As a KSF objective </li></ul><ul><li>As part of regular quality assurance processes </li></ul>
    29. 29. Finally……………. <ul><li>Conducting audit delivers benefits </li></ul><ul><li>NES criterion audit has added value </li></ul><ul><li>Can be integrated into working practice </li></ul><ul><li>Key to success is teamwork </li></ul>

    ×