COHG Editors’ &
Clinical Advisors’
Meeting
23 October 2013

The National Institute for Health Research (NIHR) is the large...
Agenda
COHG Editors & Clinical Advisors Meeting
23 October 2013
1. Welcome
2. Role of Editor
3. Prioritisation of topics
4...
The role of a
Cochrane OHG
Editor

The National Institute for Health Research (NIHR) is the largest single funder of the C...
COHG’s Editorial team










Co-ordinating Editors
Managing Editor/Deputy Managing Editor
Trials Search Co-ord...
COHG’s membership

To support review teams:
805 registered-COHG authors
Aim of session
 To explore the role of Editors

 Minimum requirements
 Input into editorial process
 Role of Contact E...
Editors
 Minimum requirements:
 Completed a review prior becoming an editor
 Attend at least two Cochrane/COHG events a...
Editorial process input
 Selecting and shaping the Cochrane Reviews
prepared by COHG
 new review proposals
 priorities ...
Act as Contact Editor
 Allocated reviews within specialty;
 To provide ad-hoc methodological and/or content advice to
re...
Discussion
 What aspects of the Editor’s role could be made
easier?
 List of assigned reviews?
 Improved checklist form...
Prioritisation of
Cochrane OHG Reviews

The National Institute for Health Research (NIHR) is the largest single funder of ...
Funding providers
 Base-line salaries (NIHR)
 Global Alliance (donations from dental bodies)
 British Orthodontic Socie...
Prioritisation of reviews
 Importance of our reviews asking relevant
questions
 Important reviews are kept up-to-date
 ...
Paediatric prioritisation
exercise process
1. Invitation to all COHG paediatric authors;
2. Invitation to international ex...
Expert panel members:
paediatric dentistry












Paul Ashley, UK
Joel Berg, USA
Michael Casas, Canada
Urs...
Results of paediatric
prioritisation exercise
 Prevention
 Fluoride
varnishes, gels, mouthrinses, toothpastes
(8)
 Seal...
Results of paediatric
prioritisation exercise
 Treatment
 Different methods for treating and
managing decay in primary a...
Results of paediatric
prioritisation exercise
 Additional topics (with reviews)
 Primary school-based behavioural
interv...
Results of paediatric
prioritisation exercise
 Additional topics (no reviews)
 Complex interventions for preventing cari...
Expert panel discussion:
paediatric guidelines
 Paediatric experts wanted to collaborate over
guidelines
 Not what we ar...
Guideline repository
 The purpose of the repository is to:
 help identify priority review topics that could
inform guide...
Guideline implementation
 We already liaise with NICE and SIGN to ensure we
are prioritising the reviews they need
 Fluo...
Prioritisation of reviews
 Prioritisation exercises need to:
 include all stakeholders
 cover whole of oral health
Introduction
to MECIR

The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral...
MECIR
Methodological Expectations of Cochrane
Intervention Reviews
 Methodological standards for the conduct of new Cochr...
Standards for conduct
Standards for conduct
Standards for reporting
Standards for reporting
Reporting of PLS (PLEACS)
Reporting of PLS (PLEACS)
Aim
 To highlight some of the key features of a
Cochrane Review and apply MECIR standards
where appropriate:






A...
Worksheet
 Sambunjak D, Nickerson JW, Poklepović T, Johnson
TM, Imai P, Tugwell P, Worthington HV.
Flossing for the manag...
Abstract
 You’ve been provided with:
 A set of MECIR standards for the reporting of
abstracts
 An abstract for the flos...
Abstract
 Do the conclusions follow on from the
results?

 What changes, if any, would you make?
Risk of Bias
 Chapter 8 of the Cochrane Handbook
 It’s about validity of findings

 It’s not about ‘quality’
 A main c...
Example RoB table
Bias
Random sequence generation
(selection bias)
Allocation concealment
(selection bias)
Blinding of par...
Further examples
 Children were sequentially allocated. The first
child who came for examination to the first
group, seco...
Further examples
 The study was conducted in a single blind
fashion
One of the blinding domains, but we don’t know
which ...
Further examples
 An envelope was opened, in which it indicated
whether the patient should receive the bioadhesive gel or...
Further examples
 22 participants had amalgam restorations
whilst 23 had composite restorations. 8/47
teeth had restorati...
Further examples
 In a trial of drug X versus placebo to treat oral
mucositis, 13/35 were withdrawn from the
drug X group...
Summarising RoB
Which outcomes?
 Include ‘all outcomes that are likely to be
meaningful to clinicians, patients
(consumers), the general
...
Prioritisation
 What are the main outcomes of interest for decision
making?
 Importance of outcomes from clinicians, con...
Outcomes should…
 be well-defined measuring a single concept
 or where an outcome is complex, this should be clearly spe...
Problem illustration


"Placement of a preformed metal crown in one tooth compared with caries
removal followed by placem...
Problem illustration


"Placement of a preformed metal crown in one tooth compared with caries
removal followed by placem...
Possible solutions
Primary outcome measures :
 Pain from dental decay (after the procedure) at time points x, y and z
 T...
Comparable review
Primary outcomes
The main outcome of interest was success or failure of the replacement or
repair restor...
Comparable review
Secondary outcomes
In addition to the main outcome measure, any outcomes reported
perioperatively (e.g. ...
Study eligibility
Remember!
Reporting of outcomes should rarely determine the
eligibility of studies for a review.
 Studi...
Summary of Findings
Based on work presented by
Nancy Santesso, Quebec 2013

The National Institute for Health Research (NI...
Summary of Findings
Developed PICO

Located studies
Abstracted data from studies

Assessed the risk of bias in the studies...
Summary of Findings
How do we:
 interpret results and draw conclusions?
 GRADE

 present results to reader/users?
 Sum...
Summary of Findings
 Two main concepts
when interpreting
results:
 magnitude of
effect
 confidence in
that effect
Summary of Findings
Summary of Findings
Summary of Findings
Determinants of quality of evidence/
confidence in effect
Consider 5 factors to evaluate the quality of the
evidence:
 Ri...
Pulling it together
Quality of Evidence varies from:
HIGH



MODERATE





LOW
VERY LOW

Confidence
Pulling it together
Quality

Interpretation

We are very confident that the true effect lies close to that of the
estimate...
Systematic review and clinical question information:
Participants, interventions and comparisons
Outcomes
Outcomes
Present important outcomes:
 Select from primary outcomes identified a priori
in protocol or those important to ...
Results: Absolute effects
 What happens to people with, and without, intervention?
Results: Relative effects
Results: Number of
participants/studies
 Only some studies contributed information about an outcome
Results:
Quality of the evidence
Results: Comments
Results: Footnotes

 Clarification
 Judgements
 Transparency
Conclusions
Considerations when making conclusions
 Do I believe the results from these studies?
Risk of bias
 Are the r...
Flossing review: SoF
 You are provided with a SoF table for the
flossing review.
 Two forest plots showing the results f...
Internal consistency
 You are provided with:
 MECIR standards that relate to the conclusions
from the main text of the r...
Internal consistency
• What changes, if any, do you think are necessary
in the conclusion?
• How does the information in t...
Discussion
 What aspects of the Editor’s role could be made
easier?
 List of assigned reviews?
 Improved checklist form...
Slide set for editors training day edited for blog
Upcoming SlideShare
Loading in …5
×

Slide set for editors training day edited for blog

941 views

Published on

Published in: Health & Medicine, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
941
On SlideShare
0
From Embeds
0
Number of Embeds
709
Actions
Shares
0
Downloads
1
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Helen
  • 105 authors22 members of expert panel
  • Mindful of international national debate. As funders are NIHR we need to be mindful of this when deciding which reviews to prioritise
  • Mindful of international national debate. As funders are NIHR we need to be mindeful of this when deciding which reviews to prioritise
  • Mindful of international national debate. As funders are NIHR we need to be mindful of this when deciding which reviews to prioritise
  • Mindful of international national debate. As funders are NIHR we need to be mindeful of this when deciding which reviews to prioritise
  • The repository will be prospective (no attempt will be made to identify previously published guidelines), although as soon as GDGs are aware that a guideline will be produced it can be added to the repository. GDGs are encouraged to submit brief details of their guidelines to the COHG for uploading to the repository. The repository will not be restricted to guidelines that aim to include Cochrane Reviews; however, where it is likely that a guideline will cover questions that could be/are covered by COHG reviews, we would hope to establish dialogue with the GDG to determine if it is feasible to complete/update reviews in line with the guideline schedule. If you are aware of any additional unlisted guidelines currently in development or currently being updated, please notify the repository administrator via email to cohg@manchester.ac.uk 
  • Helen
  • Presentation covers material presented in sections 5.4.1 to 5.4.4 of the Cochrane Handbook.Careful thought should be given to include all important outcomes in a review (not from a purely clinical perspective). Clinical versus patient reported outcomes: e.g. size of ulcer versus pain, morbidity (ability to eat or speak)Outcomes of economic evaluations: cost, resource useAbsence of outcomes in primary studies can highlight gaps in the research and be part of the implications for future research section.
  • Specify a priori.Avoid being data driven.
  • Caries trials little utility in measuring caries increment at 6 or 12 months.Multiple measurement: single summary maximum pain score in 24 hours; measurement closest to the end of treatment, specific point in time. Indirect or surrogate outcome measures, such as laboratory results or radiologic results (e.g. loss of bone mineral content as a surrogate for fractures in hormone replacement therapy), are potentially misleading and should be avoided or interpreted with caution because they may not predict clinically important outcomes accurately. Surrogate outcomes may provide information on how a treatment might work but not whether it actually does work. Many interventions reduce the risk for a surrogate outcome but have no effect or have harmful effects on clinically relevant outcomes, and some interventions have no effect on surrogate measures but improve clinical outcomes.
  • Clarify definitionsState timepoints of interestUse of clinical and indirect / surrogate outcome measures
  • Sharif MO, Merry A, Catleugh M, Tickle M, Brunton P, Dunne SM, Aggarwal VR. Replacement versus repair of defective restorations in adults: amalgam. Cochrane Database of Systematic Reviews 2010, Issue 2. Art. No.: CD005970. DOI: 10.1002/14651858.CD005970.pub2.
  • Sharif MO, Merry A, Catleugh M, Tickle M, Brunton P, Dunne SM, Aggarwal VR. Replacement versus repair of defective restorations in adults: amalgam. Cochrane Database of Systematic Reviews 2010, Issue 2. Art. No.: CD005970. DOI: 10.1002/14651858.CD005970.pub2.
  • The following slides to be used in conjunction with the worksheet
  • Best to distinguish between studies, and results
  • Will abstract, PLS be covered else where?
  • 29
  • 48
  • Summary of all the factors that need to consider.
  • Slide set for editors training day edited for blog

    1. 1. COHG Editors’ & Clinical Advisors’ Meeting 23 October 2013 The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.
    2. 2. Agenda COHG Editors & Clinical Advisors Meeting 23 October 2013 1. Welcome 2. Role of Editor 3. Prioritisation of topics 4. Title registration – discussion of proposed titles 5. Introduction to MECIR 6. Screening of reviews/ Tips for good practice 7. Worked example – flossing review: a. Abstract b. Risk of Bias c. Outcomes d. Summary of findings e. Consistency across review
    3. 3. The role of a Cochrane OHG Editor The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.
    4. 4. COHG’s Editorial team          Co-ordinating Editors Managing Editor/Deputy Managing Editor Trials Search Co-ordinator (Feedback Editor) Consumer Co-ordinator Editors Clinical Advisors Referees Research/Editorial Support Co-ordinator Administrative Assistant
    5. 5. COHG’s membership To support review teams: 805 registered-COHG authors
    6. 6. Aim of session  To explore the role of Editors  Minimum requirements  Input into editorial process  Role of Contact Editor  Open discussion
    7. 7. Editors  Minimum requirements:  Completed a review prior becoming an editor  Attend at least two Cochrane/COHG events a year (e.g. editorial meetings, review proposal meetings, regional meetings, colloquia, training days…)  Input into editorial process  Act as Contact Editor
    8. 8. Editorial process input  Selecting and shaping the Cochrane Reviews prepared by COHG  new review proposals  priorities / emerging areas of research  Providing comments on review drafts
    9. 9. Act as Contact Editor  Allocated reviews within specialty;  To provide ad-hoc methodological and/or content advice to review teams throughout entire editorial process;  Judge whether the review is ready for peer review and help identify appropriate referees;  Look over submitted comments; outline expected revisions; advise authors on response to referee comments;  Examine revised reviews and confirm whether review should go forward for signing off/publication.
    10. 10. Discussion  What aspects of the Editor’s role could be made easier?  List of assigned reviews?  Improved checklist form?  More/less time for comments?  Sufficient support from Editorial base?  Unmet training needs?  Ways to keep up-to-date with developments in Cochrane or COHG?
    11. 11. Prioritisation of Cochrane OHG Reviews The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.
    12. 12. Funding providers  Base-line salaries (NIHR)  Global Alliance (donations from dental bodies)  British Orthodontic Society, UK  Royal College of Surgeons of Edinburgh, UK  New York University College of Dentistry, USA  British Society of Paediatric Dentistry, UK  National Center for Dental Hygiene Research & Practice, USA  Canadian Dental Hygienists Association, Canada  British Society of Periodontology, UK  British Society of Oral Surgeons, UK  Mayo Clinic, USA
    13. 13. Prioritisation of reviews  Importance of our reviews asking relevant questions  Important reviews are kept up-to-date  Two international prioritisation exercises so far:  Orthodontics and paediatric dentistry
    14. 14. Paediatric prioritisation exercise process 1. Invitation to all COHG paediatric authors; 2. Invitation to international expert panel; 3. Two teleconferences; 4. Final list of reviews
    15. 15. Expert panel members: paediatric dentistry            Paul Ashley, UK Joel Berg, USA Michael Casas, Canada Urshla Chaudhry (BSPD), UK Ben Cole, UK Michael Cranfield, UK Ivana Čukovid-Bagid, Croatia Peter Day, UK Chris Deery, UK Monty Duggal, UK Mike Harrison, UK            Marie-Therese Hosey, UK Milton Houpt, USA Norbert Krämer, Germany Jessica Lee, USA Nick Lygidakis, Greece David Manton, Australia Jeanette Mooney, UK Linda Rosenberg, USA Svante Twetman, Denmark Jaap Veerkamp, Netherlands Richard Welbury, Scotland
    16. 16. Results of paediatric prioritisation exercise  Prevention  Fluoride varnishes, gels, mouthrinses, toothpastes (8)  Sealants (2)
    17. 17. Results of paediatric prioritisation exercise  Treatment  Different methods for treating and managing decay in primary and permanent dentition (6)  Pulp treatment (1)  Preoperative analgesics (1)
    18. 18. Results of paediatric prioritisation exercise  Additional topics (with reviews)  Primary school-based behavioural interventions for preventing caries  Hypnosis for children undergoing dental treatment  Sedation of children undergoing dental treatment
    19. 19. Results of paediatric prioritisation exercise  Additional topics (no reviews)  Complex interventions for preventing caries  Diagnosis of caries  Management of early caries lesions in children
    20. 20. Expert panel discussion: paediatric guidelines  Paediatric experts wanted to collaborate over guidelines  Not what we are here for, but we thought of an idea to help:  Guideline repository
    21. 21. Guideline repository  The purpose of the repository is to:  help identify priority review topics that could inform guideline development;  identify areas of duplication/overlap, where evidence tables could be shared between Guideline Development Groups (GDGs);  increase stakeholder involvement in guideline development by widening dissemination.
    22. 22. Guideline implementation  We already liaise with NICE and SIGN to ensure we are prioritising the reviews they need  Fluoride varnish review  Initial review published in 2002 (9 trials)  Update published July 2013 (22 trials)
    23. 23. Prioritisation of reviews  Prioritisation exercises need to:  include all stakeholders  cover whole of oral health
    24. 24. Introduction to MECIR The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.
    25. 25. MECIR Methodological Expectations of Cochrane Intervention Reviews  Methodological standards for the conduct of new Cochrane Intervention Reviews  December 2012  Standards for the reporting of new Cochrane Intervention Reviews  December 2012  Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS)  February 2013
    26. 26. Standards for conduct
    27. 27. Standards for conduct
    28. 28. Standards for reporting
    29. 29. Standards for reporting
    30. 30. Reporting of PLS (PLEACS)
    31. 31. Reporting of PLS (PLEACS)
    32. 32. Aim  To highlight some of the key features of a Cochrane Review and apply MECIR standards where appropriate:      Abstract Risk of Bias Outcomes Summary of Findings Consistency across the review
    33. 33. Worksheet  Sambunjak D, Nickerson JW, Poklepović T, Johnson TM, Imai P, Tugwell P, Worthington HV. Flossing for the management of periodontal diseases and dental caries in adults. Cochrane Database of Systematic Reviews 2011, Issue 12. Art. No.: CD008829. DOI: 10.1002/14651858.CD008829.pub2.
    34. 34. Abstract  You’ve been provided with:  A set of MECIR standards for the reporting of abstracts  An abstract for the flossing review  Read through the abstract and consider whether the reporting of the results and conclusion in the abstract complies with the MECIR standards.
    35. 35. Abstract  Do the conclusions follow on from the results?  What changes, if any, would you make?
    36. 36. Risk of Bias  Chapter 8 of the Cochrane Handbook  It’s about validity of findings  It’s not about ‘quality’  A main component of the Summary of Findings
    37. 37. Example RoB table Bias Random sequence generation (selection bias) Allocation concealment (selection bias) Blinding of participants and personnel (performance bias) Blinding of outcome assessment (detection bias) Incomplete outcome data (attrition bias) Selective reporting (reporting bias) Other bias Authors’ Support for judgement judgement
    38. 38. Further examples  Children were sequentially allocated. The first child who came for examination to the first group, second child to the second group Random sequence generation: HIGH - systematic method, not true randomisation
    39. 39. Further examples  The study was conducted in a single blind fashion One of the blinding domains, but we don’t know which one: UNCLEAR – who is blinded?
    40. 40. Further examples  An envelope was opened, in which it indicated whether the patient should receive the bioadhesive gel or not Allocation concealment: UNCLEAR – is it opaque, sealed and sequentially numbered, and were they centrally/remotely assigned???
    41. 41. Further examples  22 participants had amalgam restorations whilst 23 had composite restorations. 8/47 teeth had restoration failures in the amalgam group compared to 6/52 in the composite group. Selective reporting: HIGH - reported at tooth level rather than at patient level, ignoring the paired nature of the data and clustering of teeth in the mouth
    42. 42. Further examples  In a trial of drug X versus placebo to treat oral mucositis, 13/35 were withdrawn from the drug X group, 7 of these due to lack of efficacy. 25/34 were withdrawn from the placebo group, 17 due to lack of efficacy. An intention-to-treat (ITT) analysis was carried out including those remaining. Incomplete outcome data: HIGH – only 9 of the 34 in placebo group analysed
    43. 43. Summarising RoB
    44. 44. Which outcomes?  Include ‘all outcomes that are likely to be meaningful to clinicians, patients (consumers), the general public, administrators and policy makers….’  may not have considered/evaluated all in trial  unlikely to be reported in same publication  useful to acknowledge even if not recorded/ reported
    45. 45. Prioritisation  What are the main outcomes of interest for decision making?  Importance of outcomes from clinicians, consumer advisory groups, guidelines, scoping search  COMET initiative  ≤3 primary outcomes (beneficial and adverse)  Info for SoF table – key information  ‘There should be no more than seven main outcomes, which should generally not include surrogate or interim outcomes.’  Directly relevant to review question, not ‘what the review found’
    46. 46. Outcomes should…  be well-defined measuring a single concept  or where an outcome is complex, this should be clearly specified  specify the timing of measurement  clinically important period of measurement  multiple observations introduce unit of analysis issues  specify the nature of measurement  subjective or objective outcomes, validated scale of measurement, consistent with other reviews in same clinical area?  be directly measured wherever possible, avoiding surrogate outcome measures  have relevance for future updates of the review  The outcomes of a review should evaluate both the beneficial and negative effects of the interventions.
    47. 47. Problem illustration  "Placement of a preformed metal crown in one tooth compared with caries removal followed by placement of a filling material or no treatment“  The main outcome measures for children and carers were long-term freedom from the main symptom of dental decay - pain.   Primary outcome measures freedom from clinical or radiographic signs or symptoms of pulp pathology including pain/pulp infection/discharging sinus/swelling Single or multiple outcomes time until filling or crown needs to be replaced or requires further intervention proportion of filled or crowned teeth retained until appropriate age of shedding  
    48. 48. Problem illustration  "Placement of a preformed metal crown in one tooth compared with caries removal followed by placement of a filling material or no treatment“  Other measures of success recorded and analysed  absence of clinical or radiographic evidence of secondary caries     Different outcomes (surrogate?) other clinical signs of pathology (fracture of tooth or filling, wear of crown, inflammation of gingival (gum) tissue) or multiple outcomes Single patient satisfaction costs to patient and provider adverse events Any adverse events or those likely to be related to the intervention?
    49. 49. Possible solutions Primary outcome measures :  Pain from dental decay (after the procedure) at time points x, y and z  Treatment success at x year (defined as freedom from clinical or radiographic signs or symptoms of pulp pathology including pain/pulp infection/discharging sinus/swelling)  Time until filling or crown needs to be replaced or requires further intervention  Proportion of filled or crowned teeth retained until appropriate age of shedding Other measures of success recorded and analysed:  Absence of clinical evidence of secondary caries (add definition how clinical evidence of secondary caries are to be measured). Where studies do not report clinical outcomes for secondary caries, radiographic evidence will be reported if available.*  Inflammation of gingival (gum) tissue  Peri-operative pain - Pain (as rated by children) during the procedure (adverse event)  Bleeding ( adverse events)  Patient satisfaction  Costs to patient and provider *This means that only one of these will be reported if both have data are available in the studies. If only radiographic is available, then this will be included as a surrogate for the real outcome of interest.
    50. 50. Comparable review Primary outcomes The main outcome of interest was success or failure of the replacement or repair restoration and associated tooth as assessed by clinical examination. The primary outcome measures were therefore the clinical acceptability or unacceptability of each restoration, defined by the USPHS criteria, Ryge criteria or modifications of these scales, and assessed by clinical examination. It was anticipated that this would be recorded as success or failure of the restoration, and/or that further repair or replacement of the restoration was necessary.  Other outcome measures indicating the failure of a replacement or repaired amalgam restoration included the following (occurring in relation to the repaired/replaced amalgam):  placement of an additional restoration e.g. crown or inlay;  root filling;  clinical symptoms e.g. pain, swelling, diagnosis of pulpitis, abscess formation, and extraction of the tooth. Studies should have determined success or failure according to the same criteria used in the decision to replace or repair the restoration.
    51. 51. Comparable review Secondary outcomes In addition to the main outcome measure, any outcomes reported perioperatively (e.g. pain/discomfort) and postoperatively (i.e. within 48 hours; e.g. pain/discomfort) were to be recorded. Where any other outcomes were presented e.g. related to patient experience or aesthetics these were also to be recorded. Timing of outcome assessment The decision on which outcome period to use for the review was to be based on the most commonly reported period(s) of assessment amongst studies meeting the inclusion criteria. Outcome data from all periods of follow-up were to be included, but where the period of follow-up differed between studies, this was to be categorised as medium-term (less than 5 years) or long-term (5 years and above). Time-to-event (survival data) was to be collected and analysed where available.
    52. 52. Study eligibility Remember! Reporting of outcomes should rarely determine the eligibility of studies for a review.  Studies shouldn’t be excluded because they provide no 'usable' data. Even in absence of any information for a review outcome there will still be an entry into the SoF table.  Look for ‘expected’ key outcomes in the clinical area are routinely measured
    53. 53. Summary of Findings Based on work presented by Nancy Santesso, Quebec 2013 The National Institute for Health Research (NIHR) is the largest single funder of the Cochrane Oral Health Group.
    54. 54. Summary of Findings Developed PICO Located studies Abstracted data from studies Assessed the risk of bias in the studies Synthesised the data for each outcome Assessed heterogeneity and publication bias Assess the evidence and draw conclusions
    55. 55. Summary of Findings How do we:  interpret results and draw conclusions?  GRADE  present results to reader/users?  Summary of Findings tables
    56. 56. Summary of Findings  Two main concepts when interpreting results:  magnitude of effect  confidence in that effect
    57. 57. Summary of Findings
    58. 58. Summary of Findings
    59. 59. Summary of Findings
    60. 60. Determinants of quality of evidence/ confidence in effect Consider 5 factors to evaluate the quality of the evidence:  Risk of bias  Inconsistency (or heterogeneity)  Indirectness (PICO and applicability)  Imprecision (number of events and confidence intervals)  Publication bias See also Cochrane Handbook, Chapter 12
    61. 61. Pulling it together Quality of Evidence varies from: HIGH  MODERATE    LOW VERY LOW Confidence
    62. 62. Pulling it together Quality Interpretation We are very confident that the true effect lies close to that of the estimate of the effect. We are moderately confident in the effect estimate: Moderate The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different. Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate Low of the effect. We have very little confidence in the effect estimate: Very low The true effect is likely to be substantially different from the estimate of effect. High
    63. 63. Systematic review and clinical question information: Participants, interventions and comparisons
    64. 64. Outcomes
    65. 65. Outcomes Present important outcomes:  Select from primary outcomes identified a priori in protocol or those important to making recommendations  Choose patient-important outcomes  Include outcomes, whether you have results or not  Describe the outcomes (e.g. Provide scale range and indicate a high score)
    66. 66. Results: Absolute effects  What happens to people with, and without, intervention?
    67. 67. Results: Relative effects
    68. 68. Results: Number of participants/studies  Only some studies contributed information about an outcome
    69. 69. Results: Quality of the evidence
    70. 70. Results: Comments
    71. 71. Results: Footnotes  Clarification  Judgements  Transparency
    72. 72. Conclusions Considerations when making conclusions  Do I believe the results from these studies? Risk of bias  Are the results consistent across studies? Inconsistency  Are these all of the studies? Publication bias  Is this effect size precise? Imprecision  How do these results apply? Applicability, directness
    73. 73. Flossing review: SoF  You are provided with a SoF table for the flossing review.  Two forest plots showing the results for gingivitis and plaque at 1 month.  Examine the SoF and the Forest Plots for discussion
    74. 74. Internal consistency  You are provided with:  MECIR standards that relate to the conclusions from the main text of the review  the conclusions from the flossing review  First consider the information presented in the conclusion alone, and then compare it to that in the abstract and summary of findings
    75. 75. Internal consistency • What changes, if any, do you think are necessary in the conclusion? • How does the information in the conclusion compare to that in the abstract and summary of findings? • Are there any additional changes you would make and why?
    76. 76. Discussion  What aspects of the Editor’s role could be made easier?  List of assigned reviews?  Improved checklist form?  More/less time for comments?  Sufficient support from Editorial base?  Unmet training needs?  Ways to keep up-to-date with developments in Cochrane or COHG?

    ×