TRIP = clinical search engine designed to allow clinicians to quickly find answers to their clinical questions using the best available evidence.
Development of Cochrane Response - part II
Development of Cochrane Rapid Response (Part 2)• Chantelle Garritty, Centre for Practice-Changing Research, Ottawa Hospital Research Institute, Ottawa, Canada October 2012
Rapid Reviews vs. Traditional Systematic ReviewGeneral Differences General Similarities- Condensed timeline - Retains core values of being- Limits set on inclusion criteria of transparent to facilitate the search including how grey replication literature is dealt with - Adopted the of standard of- Post-hoc rethinking of eligibility assessing the quality of included criteria depending on volume and studies (SRs) applicability of evidence - Share preference for inclusion- Heavy reliance on systematic of the highest quality studies review evidence vs. Inclusion of primary studies (e.g., RCTs or observational studies)
Systematic vs. Rapid Reviews SYSTEMATIC REVIEW RAPID REVIEWTIMEFRAME 6 months – 2 years <5 weeks; 6-8 wksQUESTION Focused clinical question, narrow Focused to broad, clinical or health parameters services question; possibly broader parametersSOURCES AND Comprehensive sources searched Sources may be limited butSEARCHES and explicit strategies sources/strategies made explicit;SELECTION Exclusion/inclusion defined a priori Exclusion/inclusion defined a priori and post hocAPPRAISAL Rigorous; Rigorous; Critical appraisal Critical appraisal (SRs only)SYNTHESIS Narrative synthesis +/- Narrative synthesis/ categorization of Quantitative synthesis the dataINFERENCES Evidence-based – generates a Limited/cautious interpretation of the conclusion to answer the research findings to answer the research question question
Evolution of review methods vs. time 3 Systematic review 2.5Review „truth‟/validity 2 Scenario 1 1.5 Scenario 2 Rapid review? 1 0.5 Traditional review 0 0 0.5 1 1.5 Time 2 2.5 3•If trying to get at truth, narrative reviews fall very short of the mark; traditional SRs get the closest; while we argue RRs fall somewhere along this continuum• A moderately robust summary of the evidence in the form of a RR may be better for informing health care decisions than no evidence at all
Overview of our reports:• Conducted a total of 19 rapid evidence summaries• 13 – focused on clinical initiatives • 9 across the field of obstetrics/gynaecology • Better Outcomes Registry & Network Ontario (BORN Ontario) – an initiative to increase capacity to plan and deliver improved maternal-child services• 6 – focused on health systems/ health services initiatives • Effectiveness of Antimicrobial Stewardship Programs in Hospitals• Requests came from various stakeholders (n=9)
Evidence summaries to date COMPLETED REVIEW REQUESTED BY USED FOR1. Pre-diabetes Champlain Diabetes Strategy Backgrounder for clinical initiatives Advisory Committee2. Health system LHIN CEO Backgrounder for system changesreform/integration3. Electronic health records LHIN CEO Backgrounder for system changes4. Post-partum care for Champlain Diabetes Strategy Backgrounder for clinical initiativesGestational Diabetes Advisory Committee5. Timing of Elective, repeat BORN Ontario Evidentiary support for qualityC-section <39wks indicator „dashboard‟6. Intrapartum management of Champlain Diabetes Strategy Backgrounder for clinical initiativesGDM Advisory Committee7. Pedometers & Chronic LHIN CEO; Chronic disease Backgrounder for clinical initiativesDiseases collaborative8. Formula supplementation BORN Ontario Evidentiary support for qualityin-hospital indicator „dashboard9. 3rd/4th degree lacerations BORN Ontario Evidentiary support for quality indicator „dashboard10. Elective induction of term BORN Ontario Evidentiary support for qualitypregnancies indicator „dashboard
Evidence summaries to date REVIEW REQUESTED BY USED FOR 11. ED short stay units The Ottawa Hospital Backgrounder for system changes 12. Models of patient flow The Ottawa Hospital Backgrounder for system changes 13. Unsatisfactory blood spot BORN Ontario Evidentiary support for quality samples for newborn screening of indicator „dashboard congenital diseases* 14. Episiotomy BORN Ontario Evidentiary support for quality indicator „dashboard 15. Screening to prevent newborn BORN Ontario Evidentiary support for quality group B streptococcal infection indicator „dashboard 16. Models of elderly care† Regional Geriatric Program of Backgrounder for Eastern Ontario policy/program planning 17. Physical activity and chronic LHIN chronic disease Backgrounder for disease† collaborative policy/program planning 18. Pre-op rehabilitation Alberta Bone and Joint Group Backgrounder for interventions for total knee policy/program planning arthroplasty 19. Antimicrobial stewardship The Ottawa Hospital – Patient Backgrounder for programs Safetey policy/program planning*Evidence brief; † Evidence map
Dissemination• Reviews are disseminated on our website (www.ohri.ca/kta), to end-users directly, and when possible, through publication
KTA Interviews – Knowledge Users’ Comments of Rapid Reviews ApproachStakeholders(CHAMPLAIN LHIN, BORN ONTARIO, OTTAWA HOSPITAL RESEARCH INSTITUTE (OHRI), THE OTTAWA HOSPITAL (TOH))• Scientific Director of Performance Measurement – TOH• Chief Information Officer, Champlain LHIN• Former CEO, Champlain LHIN• Clinician, transport nurse, educator, and lactation consultant, BORN ONTARIO• Scientific Manager, BORN ONTARIO• Division Chief, Endocrinology and Metabolism, TOH• Clinical Investigator (MD), Clinical Epidemiology, OHRI, TOH• Senior Health Planning & Engagement Specialist at Champlain LHIN
Knowledge Users’ CommentsUtility• Structure/ format of summaries is important – eyes being drawn to key messages (capture reader‟s attention)• Thought to be incredibly useful to have evidence at your fingertips and summarized as it was• Format was user-friendly for non-researcher decisions makers and clinicians• Format increased level of comfort with data for non-clinicians, and validated that decisions are made based in part on the evidence provided
Knowledge Users’ Comments-2• Notion that rapid reviews assists with having wide group of stakeholders from various backgrounds to get on the same page in terms of understanding what the latest evidence is and what it means (levels out playing field in decision-making so process less prone to group think scenarios)• Directly influenced practice change in local delivery of care (Example: summaries are being used to move forward on the regional plan on delivering diabetes care)
Knowledge Users’ Comments-3• Approach has helped build relationships between those creating the summaries and the implementers - key ingredient to ensure ongoing dialogue• Notion that the evidence is coming from an “objective, third party” lens – and not from one particular angle/clinical slant, made the findings of the rapid reviews seem more credible• Rapid reviews were used to help set the context for LHIN discussions, and were key to planning• Also, used to lay the groundwork as part of a broader environmental scans; ultimately helping with prioritizing of LHIN projects
Knowledge Users’ Comments-4Barriers• Policy and practice implementation difficult – taking general evidence and applying it locally found to be challenging• System issues provide obstacles (for example, hospital sites where they only have OR times for 1-day/ week this then controls when surgeries can happen – even though that does not match with when it should be done according to the evidence). How does one reconcile this on the implementation side?• Unfavourable attitudes towards evidence requires behaviour change efforts (interventions difficult to implement)• Taking the scientific evidence and translating it into a user-friendly format for a variety of end-users (dissemination science applied)
Local Program Development The Ottawa Hospital Technology Assessment Program (TOHTAP) • Hospital-based knowledge support service with focus on rapid reviews • Knowledge synthesis + local data + economical evaluation • Inaugural year with possible long- term funding commitment (OHRI, TOH Performance Management & Patient Safety groups)
International Program Development KTA approach has informed development of… Cochrane Rapid Response (specifics to follow – Part 3)
Developed into a Program with Three Key Streams
Logistical Considerations for the Conduct of Rapid Reviews
Things to consider…1) Existence of evidence to summarize • Evidence exists and is reported • Evidence exists, but is not reported (or is reported poorly) Synthesis and Topic (no question) interpretation of • Evidence does not exist relevant evidence2) Balance between breadth of evidence and depth of rapid review synthesis for particular question (tradeoff between going deeper if Q is more narrow vs. only touching the surface if Q is broader)3) Size of team conducting rapid review – what resources are available for short, intense period of time?
4) Important to anticipate the level of engagement/availability of end user – especially during protocol development and screening of records • Vital component to this process • Need to identify a go to person within your stakeholder group; someone willing to be on call to answer your questions; relay information back and forth from knowledge users5) Access (internally or externally) to skilled resources • Information specialists • Data managers • Content experts • Other?6) Access to library subscriptions for resources • If not, alternative sources/approaches? Limits on interpretation? Should not be under estimated!
Literature accessibility• Clinical/health related topics: • PubMed http://www.ncbi.nlm.nih.gov/pubmed/advanced • PubMedHealth http://www.ncbi.nlm.nih.gov/pubmedhealth/ • Cochrane Library www.thecochranelibrary.com • Subscription-based resource but anyone can create a user name and login to access everything but the full text of the reviews and the critical appraisals. • CRD Databases http://www.crd.york.ac.uk/CMS2Web/ • TRIP http://www.tripdatabase.com/ • Health-evidence.ca http://health-evidence.ca/ • Public health focus • Evidence Updates http://plus.mcmaster.ca/EvidenceUpdates/ • NHS Evidence http://library.nhs.uk/mylibrary/default.aspx • CADTH “Grey Matters” - http://www.cadth.ca/index.php/en/cadth/products/grey- matters • Most of the resources listed are free; all the various clinical guideline registries are free with the exception of G-I-N.• Social science topics: • Campbell Library: http://www.campbellcollaboration.org/library.php
DistillerSR®• Distiller SR = indispensible • Canadian • Online, web-based interface (Google platform) • Allows for multiple users • Online reference and document storage (PDF‟s) • Used for screening, extraction, data presentation • Automated collation • Reviewer conflicts • References (Inclusions/Exclusions) • Intuitive to use • Real-time reporting and monitoring facilitates project management • Extremely responsive online and/or phone support • Monthly/yearly subscriptions