Does ProVationtm MDcapture colonoscopy KPIs?                David Theobald                  Clinical Director National End...
National Endoscopy Quality         Improvement Programme•   Catalyst - Bowel Screening Pilot•   Appointment of NCLGIE and ...
National Endoscopy Quality  Improvement Programme             Goal The primary aim is to have every   endoscopy unit in th...
The strategy to reach this        goal will betotal service improvement encompassing individual    performance, unitperfor...
The methodology to reach      this goal will bechange based on evidence  (principles of the ‘Audit           Cycle’)
The primary tool to reach    this goal will be     the endoscopyGlobal Rating Scale (GRS)
What is the Global Rating Scale (GRS)?  The GRS is a web-based self assessment toolthat provides a standard for accreditat...
GRS Standards1. Clinical Quality       2. Quality of the Patient Experience1. Information/consent    1. Equality of access...
GRS Standards1. Clinical Quality                   2. Quality of the Patient Experience1. Information/consent             ...
GRS Standards1. Clinical Quality                      2. Quality of the Patient Experience1. Information/consent          ...
GRS achievement levels• Units score themselves on several items for  each standard• This gives a level attained and, thus,...
Audit Cycle
National Endoscopy Quality          Improvement Programme•   Total service improvement•   Approach is that of the audit cy...
Auditable Outcomes and Quality               StandardsAn auditable outcome refers to an outcome, which isimportant to moni...
Auditable Outcomes and Quality                 StandardsA quality standard is an auditable outcome for whichthere is an ev...
Auditable Outcomes and Quality               Standards• Both require acquisition of large amounts of  data• This must be a...
Audit Cycle• Standards – initially, for the Development Trial,  most are UK standards• Reliable Data – various sources• Sc...
Clinical data as a KPI• Choice of data• Do not collect data because it can be collected• Using data suggested by British S...
KPI’s as per BSG
Requirements of a clinical data           collection solution• Accurate• Require minimal extra effort over and above  repo...
Endoscopy Reporting in NZ• Mixture• Dictation• Hand written• Legacy standalone electronic reporting  systems• In some loca...
Acquisition of ProVationtm MD• Joint purchase after extensive tender process  by the three Auckland DHBs• Auckland Install...
ProvationtmMDproducesterrificendoscopy reports
How easy is it to get data into          ProVationtmMD?• Menu structure is regarded as cumbersome by  clinicians• Leads to...
How easy is it to get data from         ProVationtm MD?• All keystrokes are recorded• Theoretically everything entered can...
Which BSG KPIs could beextracted from ProvationtmMD
Initial appraisal of Provationtm MD          and clinical KPIs
Initial appraisal of Provationtm MD and               clinical KPIs• Proof of concept investigation to assess  whether sta...
Initial appraisal of Provationtm MD and               clinical KPIs• Prospective audit across three Auckland DHBs• Seven K...
EM Johns1, PD Frankish1, RS Walmsley1, DS Rowbotham2, RK Ogra3, DR Theobald1                  Departments of Gastroenterol...
Initial appraisal of Provationtm MD                and clinical KPIsKPI                         Problem                   ...
Initial appraisal of Provationtm MD                and clinical KPIsKPI                      Problem                  Sugg...
Initial appraisal of Provationtm MD                and clinical KPIsKPI                   Problem                  Suggest...
Does ProVationtm MDcapture colonoscopy KPIs?    Well, sort of -ish
Can ProVationtm MDcapture colonoscopy KPIs?       Well, yes
Where are we now?Suggested solution            Suggested solution            Suggested solutionNew mandatory field for    ...
Where are we now?Suggested solution          Suggested solution    Suggested solutionNew mandatory field forpost surgical ...
Where are we now?Suggested solution            Suggested solution    Suggested solution                              Shoul...
Where are we now?Suggested solution    Suggested solution            Suggested solution                                   ...
The way forward• Progress must be clinician led• Clinicians must collaborate across the entire  country to formulate an ag...
The way forward• The NZ ProVation Users Group then  approaches company for software updates  (NZ cf US market share)• The ...
The way forward• Should all DHBs purchase ProvationtmMD?• IT Board of the NHB supports a single software  solution without...
Does ProVationtm MDcapture colonoscopy KPIs?    Well, sort of - ish
Can ProVationtm MDcapture colonoscopy KPIs?       Definitley
Upcoming SlideShare
Loading in...5
×

Does ProVation MD capture colonoscopy KPIs?

608
-1

Published on

David Theobald
Clinical Director
National Endoscopy Quality Improvement Programme
(Thursday, 11.15am, Forum Room)

Published in: Health & Medicine
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
608
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Does ProVation MD capture colonoscopy KPIs?

  1. 1. Does ProVationtm MDcapture colonoscopy KPIs? David Theobald Clinical Director National Endoscopy Quality Improvement Programme
  2. 2. National Endoscopy Quality Improvement Programme• Catalyst - Bowel Screening Pilot• Appointment of NCLGIE and wider team• Colonoscopy• Widened brief
  3. 3. National Endoscopy Quality Improvement Programme Goal The primary aim is to have every endoscopy unit in the countryproviding a patient focused service.
  4. 4. The strategy to reach this goal will betotal service improvement encompassing individual performance, unitperformance and training
  5. 5. The methodology to reach this goal will bechange based on evidence (principles of the ‘Audit Cycle’)
  6. 6. The primary tool to reach this goal will be the endoscopyGlobal Rating Scale (GRS)
  7. 7. What is the Global Rating Scale (GRS)? The GRS is a web-based self assessment toolthat provides a standard for accreditation and a quality framework for service improvement
  8. 8. GRS Standards1. Clinical Quality 2. Quality of the Patient Experience1. Information/consent 1. Equality of access2. Safety 2. Timeliness3. Comfort 3. Choose and book4. Quality 4. Privacy and dignity5. Appropriateness 5. Aftercare6. Results to referrer 6. Patient feedback
  9. 9. GRS Standards1. Clinical Quality 2. Quality of the Patient Experience1. Information/consent 1. Equality of access2. Safety 2. Timeliness3. Comfort 3. Choose and book4. Quality 4. Privacy and dignity5. Appropriateness 5. Aftercare6. Results to referrer 6. Patient feedback3. Workforce1. Skill mix review and recruitment2. Orientation and training3. Assessment and appraisal4. Staff are cared for5. Staff are listened to
  10. 10. GRS Standards1. Clinical Quality 2. Quality of the Patient Experience1. Information/consent 1. Equality of access2. Safety 2. Timeliness3. Comfort 3. Choose and book4. Quality 4. Privacy and dignity5. Appropriateness 5. Aftercare6. Results to referrer 6. Patient feedback3. Workforce 4. Training1. Skill mix review and recruitment 1. Environment and opportunity2. Orientation and training 2. Endoscopy trainers3. Assessment and appraisal 3. Assessment and appraisal4. Staff are cared for 4. Equipment and materials5. Staff are listened to
  11. 11. GRS achievement levels• Units score themselves on several items for each standard• This gives a level attained and, thus, monitors progress• Levels D through A• These levels can later be used for accreditation purposes
  12. 12. Audit Cycle
  13. 13. National Endoscopy Quality Improvement Programme• Total service improvement• Approach is that of the audit cycle• Principal tool is the Global Rating Scale• Standards – Quality standards – Auditable outcomes
  14. 14. Auditable Outcomes and Quality StandardsAn auditable outcome refers to an outcome, which isimportant to monitor and review, for which it is notpossible to assign a standard. Examples of this might beuse of reversal agents for over sedation, minimum numberof procedures required to maintain competence,or outcome of endoscopic therapy for variceal bleeding
  15. 15. Auditable Outcomes and Quality StandardsA quality standard is an auditable outcome for whichthere is an evidence base that can recommend a minimumstandard, for example completion rates for colonoscopy orbleeding rates for sphincterotomy. As the evidence baseimproves it is expected that it will be possible to convertauditable outcomes into quality standards.
  16. 16. Auditable Outcomes and Quality Standards• Both require acquisition of large amounts of data• This must be accurate if important decisions are to be made on it• Clinical KPIs are part of this data• Data entry should be as effortless as it is accurate
  17. 17. Audit Cycle• Standards – initially, for the Development Trial, most are UK standards• Reliable Data – various sources• Schedule of audits - EUG• Action plans - EUG
  18. 18. Clinical data as a KPI• Choice of data• Do not collect data because it can be collected• Using data suggested by British Society of Gastroenterology (BSG)
  19. 19. KPI’s as per BSG
  20. 20. Requirements of a clinical data collection solution• Accurate• Require minimal extra effort over and above reporting input• Collected in a format that is usable• Is connected to a system that can use it• Exportable
  21. 21. Endoscopy Reporting in NZ• Mixture• Dictation• Hand written• Legacy standalone electronic reporting systems• In some locations more than one method on the same site
  22. 22. Acquisition of ProVationtm MD• Joint purchase after extensive tender process by the three Auckland DHBs• Auckland Installation 2010• Christchurch installation 2011• Other DHBs at various stages of engagement
  23. 23. ProvationtmMDproducesterrificendoscopy reports
  24. 24. How easy is it to get data into ProVationtmMD?• Menu structure is regarded as cumbersome by clinicians• Leads to more than desirable us of free text fields• Structure undeniably US biased with many redundant fields for a NZ setting• Changes possible but difficult
  25. 25. How easy is it to get data from ProVationtm MD?• All keystrokes are recorded• Theoretically everything entered can be extracted• Database queries regarded as cumbersome by clinicians• Macros
  26. 26. Which BSG KPIs could beextracted from ProvationtmMD
  27. 27. Initial appraisal of Provationtm MD and clinical KPIs
  28. 28. Initial appraisal of Provationtm MD and clinical KPIs• Proof of concept investigation to assess whether standard KPIs could be extracted from ProVation reports. KPIs were chosen in line with NHS, ASGE & NZ standards
  29. 29. Initial appraisal of Provationtm MD and clinical KPIs• Prospective audit across three Auckland DHBs• Seven KPIs assessed• Two six-week cycles with a planned preliminary analysis after week six• Analysis performed using the data export function built into ProVation (“automated”) and compared to results collated from individual reports (“manual”).• Feedback provided after the first round via department meetings or by email Minor changes made after first round – Index colonoscopy field added, bowel preparation field made mandatory – Endoscopists asked to use post-surgical note for appropriate patients
  30. 30. EM Johns1, PD Frankish1, RS Walmsley1, DS Rowbotham2, RK Ogra3, DR Theobald1 Departments of Gastroenterology, Waitemata1, Auckland2 & Counties Manukau3 District Health BoardsIntroduction MethodsQuality colonoscopy is integral to the success of bowel cancer screening programs. Key Prospective audit across three Auckland DHBsperformance indicators (KPIs) are well established but require meticulous collection of a Initial appraisal of Provationtm MD Seven KPIs assessedlarge volume of information. The endoscopy database program ProVation has recently been Two six-week cycles with a planned preliminary analysis after week sixintroduced across the Auckland region. Its potential role as a quality assurance tool was a Analysis performed using the data export function built into ProVation (“automated”)key reason for its implementation, and its future use as a nationwide audit tool is under and compared to results collated from individual reports (“manual”).consideration. Feedback provided after the first round via department meetings or by emailAim and clinical KPIsProof of concept investigation to assess whether standard KPIs could be extracted from Minor changes made after first round Index colonoscopy field added, bowel preparation field made mandatory Endoscopists asked to use post-surgical note for appropriate patientsProVation reports. KPIs were chosen in line with NHS, ASGE & NZ standards1-3. Results Hospital X Hospital Y Hospital Z Round 1 Round 2 Round 1 Round 2 Round 1 Round 2 Automated Manual Automated Manual Automated Manual Automated Manual Automated Manual Automated Manual Caecal intubation rate 208/226 199/209 225/243 216/229 245/264 231/243 268/282 253/263 200/239 202/230 226/257 228/245 92% 95% 93% 94% 93% 95% 95% 96% 84% 88% 88% 93% Bowel preparation 81% 83% 94% 96% 64% 65% 99% 100% 80% 82% 99% 100% quality documented Withdrawal times no 53% 43% 60% 46% 35% 40% 37% 37% 40% 31% 28% 29% manoeuvre#: proportion >6minutes Withdrawal times no 7:41 6:20 7:15 6:02 5:57 6:04 5:46 5:11 5:49 5:11 5:24 4:52 manoeuvre#: mean Polyp detection rate age Insufficient data Insufficient data 39/86 42/95 Insufficient Insufficient data 37/103 32/101 Insufficient data Insufficient data 45/101 33/98 50-80 (index exams) 45% 44% data 37% 32% 45% 34% Adenoma detection rate Not supported Insufficient data 26/86* 28/95 Not supported Insufficient data 24/103* 24/101 Not supported Insufficient data 26/101* 24/98 age 50-80 (index exams) 30% 29% 23% 24% 26% 24% Polyp recovery rate Not supported 165/175 Not supported 190/215 Not supported 150/159 Not supported 159/183 Not supported 243/261 Not supported 191/221 94% 88% 94% 87% 93% 86% Complications Not supported None Not supported 2 readmissions Not supported None Not supported 2 perforations Not supported 3 perforations Not supported 2 readmissions 1 readmission 1 readmission *histology manually retrieved #no polypectomy, biopsy etc
  31. 31. Initial appraisal of Provationtm MD and clinical KPIsKPI Problem Suggested solutionCaecal Intubation rate Difficulty identifying intact New mandatory field for colons post surgical patientsBowel preparation quality Variably reported when not Made mandatory mandatory No standardization Need agreement on definitions of prep quality
  32. 32. Initial appraisal of Provationtm MD and clinical KPIsKPI Problem Suggested solutionWithdrawal times if no Times overestimated if Should improve withmanouevre manouevres not familiarity documentedPolyp and adenoma Easily retrieved if Link to a pathology programdetection rates polypectomy documented (currently not possible) Histology not linked Manual linkage
  33. 33. Initial appraisal of Provationtm MD and clinical KPIsKPI Problem Suggested solutionPolyp recovery rate Provation records Requires software qualitatively and not modification quantitavelyComplications Manual entry by person Needs dedicated audit with admin privileges personnel
  34. 34. Does ProVationtm MDcapture colonoscopy KPIs? Well, sort of -ish
  35. 35. Can ProVationtm MDcapture colonoscopy KPIs? Well, yes
  36. 36. Where are we now?Suggested solution Suggested solution Suggested solutionNew mandatory field for Should improve with Requires softwarepost surgical patients familiarity modificationMade mandatory Link to a pathology program Needs dedicated audit (currently not possible) personnelNeed agreement ondefinitions of prep quality
  37. 37. Where are we now?Suggested solution Suggested solution Suggested solutionNew mandatory field forpost surgical patientsMade mandatory These are done and were easy requiring just a software tweak within the existing program. This was done by clinicians
  38. 38. Where are we now?Suggested solution Suggested solution Suggested solution Should improve with familiarity Needs dedicated audit personnelNeed agreement ondefinitions of prep quality These are partly done or partly not done and require behavioural change from clinicians
  39. 39. Where are we now?Suggested solution Suggested solution Suggested solution Requires software modification Link to a pathology program (currently not possible) These are not done and require major software work and behavoural change from clinicians
  40. 40. The way forward• Progress must be clinician led• Clinicians must collaborate across the entire country to formulate an agreed set of clinical requirements• This requires strong clinical leadership• A NZ ProVation Users Group is required for this• So far abortive attempts to establish this
  41. 41. The way forward• The NZ ProVation Users Group then approaches company for software updates (NZ cf US market share)• The NZ ProVation Users Group needs to have ongoing close ties with the National Endoscopy Quality Improvement Programme
  42. 42. The way forward• Should all DHBs purchase ProvationtmMD?• IT Board of the NHB supports a single software solution without naming a specific product• The clinical information required for KPIs is not very complex……• …..but it does need to be collected with something more reliable than a pencil and a piece of paper
  43. 43. Does ProVationtm MDcapture colonoscopy KPIs? Well, sort of - ish
  44. 44. Can ProVationtm MDcapture colonoscopy KPIs? Definitley

×