Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Standard wording for formulating evidence conclusions and implications for recommendations

.

  • Login to see the comments

  • Be the first to like this

Standard wording for formulating evidence conclusions and implications for recommendations

  1. 1. Standard wording for formulating evidence conclusions and implications for recommendations Emmy De Buck Manager Centre for Evidence-Based Practice Belgian Red Cross-Flanders Rode Kruis
  2. 2. Disclosure of interests I certify that, to the best of my knowledge, no aspect of my current personal or professional situation might reasonably be expected to affect significantly my views on the subject on which I am presenting, other than the following: Financial relationships: Employment at the Centre for Evidence-Based Practice of Belgian Red Cross-Flanders
  3. 3.  Centre for Evidence-Based Practice (CEBaP) of Belgian Red Cross-Flanders: provides evidence-based support to all programmes and activities of Belgian Red Cross-Flanders, by developing: − Practice guidelines − Systematic reviews  Staff: 4 methodologists, PhD’s Introduction
  4. 4. Rationale  In our centre several methodologists (reviewers) work in parallel to develop evidence reviews in a timely way  The evidence conclusions go from the methodologists to other collaborators (colleagues of operational services, experts)  Þ Important to standardize the evidence synthesis and to provide standard wording of the evidence conclusions/statements
  5. 5. Wording of evidence conclusions (1)  Information from guideline developers: − NICE1: “A short evidence statement should be presented alongside the evidence profile, summarising the key features of the evidence on clinical and cost effectiveness” − SIGN2: no narrative evidence conclusions − Finnish Medical Society Duodecim3: wording varies depending on the level of evidence 1 http://publications.nice.org.uk/the-guidelines- manual-pmg6/reviewing-the-evidence 2 http://www.sign.ac.uk/methodology/index.html 3http://2011.colloquium.cochrane.org/sites/2011.colloquium.cochrane.org/files/uploads/users/ u721/Wording%20of%20evidence%20and%20recommendations%20in%20EBM%20Guidelines.doc
  6. 6. Wording of evidence conclusions (2)  Information from systematic review developers: − Wide variation among authors in the Cochrane Library in reporting results and conclusions1 − The Cochrane Handbook, Chapter 9, Section 9.12: • “A narrative assessment of the evidence can be challenging...” • “If a descriptive paragraph is provided for the results from each study, this should be done consistently, including the same elements of information for each study.” • “Organizing the studies into groupings or clusters is encouraged (e.g. by intervention type, population groups, setting etc).”  Conclusion: almost no guidance about wording of evidence conclusions 1 Van Tulder et al. 2003. SPINE 28(12): 290–1299 2 http://handbook.cochrane.org/chapter_9/9_1_introduction.htm
  7. 7. Criteria for evidence conclusions (1)  Target population for reading the evidence conclusions: − colleagues who write draft recommendations − experts who validate the draft recommendations  Evidence conclusions should contain: − number and type of studies − intervention, comparison − outcome − level of evidence − direction of effect
  8. 8. Criteria for evidence conclusions (2)  Wording of evidence conclusions should reflect: − if results were statistically significant or not − the quality of the evidence − the difference between “evidence of no effect” and “no evidence of effect” 1  Evidence conclusions should be organized according to the intervention, outcome,…  Categories of evidence conclusions: − Evidence and limited evidence − No evidence − Conflicting evidence 1 Altman and Bland 1995. BMJ 311: 485
  9. 9. START Is the intervention considered as common sense? Is there any evidence available? Y Is the evidence conflicting? N Is the body of evidence of low or very low quality? N Is there any imprecision? “No evidence” N Y Y “Conflicting evidence” “Limited evidence” Y “Limited evidence” N “Evidence” Y N Is it an important practical point Flowchart of which the expert panel reaches a consensus and nobody is likely to question it? Y N Is the balance made by the expert panel positive? No Good Practice Point Y Recommendation N No Recommendation Is the balance made by the expert panel positive? Y N Is the balance made by the expert panel positive? Y N Good Practice Point Literature review Consensusmeeting with experts Recommendation No Recommendation Recommendation No Recommendation
  10. 10. Standard wording Category of evidence Wording of evidence conclusion No evidence No relevant studies were identified using the above search strategy and criteria. Conflicting evidence There is conflicting evidence from # experimental studies and/or # observational studies… (Limited) evidence Wording depends on: -statistical significance -level of evidence (LOE) -imprecision See extended table
  11. 11. P-value LOE Imprecisio n Wording Standard wording < 0.05 A/B No There is evidence from # experimental studies and/or # observational studies in favour of [intervention] (<Author> <year>, <Author> <year>, etc.). It was shown that <intervention> resulted in a statistically significant increase/decrease of <outcome>, compared to <comparison> (<Author> <year>). Evidence is of high/moderate quality. Yes There is limited evidence ... It was shown that ... Evidence is of moderate quality and results cannot be considered precise due to limited sample size, lack of data and/or large variability of results. C/D No There is limited evidence ... It was shown that ... Evidence is of low/very low quality. Yes There is limited evidence ... It was shown that ... Evidence is of low/very low quality and results cannot be considered precise due to ...
  12. 12. P-value LOE Imprecisio n Wording > 0.05 A/B No There is evidence from # experimental studies and/or # observational studies, neither in favour of the intervention nor the control (<Author> <year>, <Author> <year>, etc.). It was shown that <intervention> did not result in a statistically significant difference of <outcome>, compared to <comparison> (<Author> <year>). Evidence is of high/moderate quality. Yes There is limited evidence ... A statistically significant increase/decrease of <outcome>, using <intervention> compared to <comparison>, could not be demonstrated (<Author> <year>). Evidence is of moderate quality and results of this study/these studies are imprecise due to limited sample size, lack of data and/or large variability of results. C/D No There is limited evidence ... It was shown that <intervention> did not result ... Evidence is of low/very low quality. Yes There is limited evidence ... A statistically significant increase/decrease ... could not be demonstrated ... Evidence is of low/very low quality and results of this study/these studies are imprecise due to ...
  13. 13. Implications for recommendations  Only in case of p<0.05, LOE A/B and no imprecision, a draft recommendation for or against the intervention should be included in the guideline  In the other cases: a draft recommendation may be included in the guideline  The final recommendation will be based on the judgement of the multidisciplinary expert panel, taking into account: − the scientific evidence − preferences of the target group − expertise and practical experience
  14. 14. Example  PICO: Is rice water (I) effective compared to standard Oral Rehydration Solution (ORS) (C) to improve diarrhoea (O) in adults or children with diarrhoea (P)?  Included studies: 2 RCTs: Metha 1986, Fakhir 1990 High [A] Downgrading due to Limitations of study design -1 Lack of blinding/blinding unclear; lack of allocation concealment in 1 study Imprecision -1 Limited sample size Inconsistency 0 Indirectness 0 Publication bias 0 QUALITY (GRADE) Low [C]
  15. 15. Example Outcome Comparison Effect Size #studies # parti-cipants Ref Stool frequency between 1 and 3 per day Rice water versus glucose ORS Day 1: Not statistically significant: 4/100 vs 0/100; RR: 9.00, 95% CI [0.49, 165.00] (p=0.14) Day 2: Statistically significant: 16/100 vs 6/100; RR: 2.67, 95% CI [1.09, 6.54] (p=0.03) Day 3: Statistically significant: 46/100 vs 27/100; RR: 1.70, 95% CI [1.16, 2.51] (p=0.007) Day 4: Statistically significant: 68/100 vs 42/100; RR: 1.62, 95% CI [1.24, 2.11] (p=0.0004) 1, 100 vs 100 Metha, 1986 Small stool volume Day 1: Not statistically significant: 2/100 vs 0/100; RR: 5.00, 95% CI [0.24, 102.85] (p=0.30) Day 2: Not statistically significant: 40/100 vs 30/100; RR: 1.33, 95% CI [0.91, 1.96] (p=0.14) Day 3: Not statistically significant: 56/100 vs 42/100; RR: 1.33, 95% CI [1.00, 1.78] (p=0.05) Day 4: Statistically significant: 76/100 vs 44/100; RR: 1.73, 95% CI [1.35, 2.21] (p<0.0001) Duration of purging in the hospital (hours) Statistically significant: 60.2±2.6 vs 78.6±4.6 MD: -18.40, 95%CI ; [-20.60;-16.20] (p=0.00001) 1, 20 vs 23 Fakhir, 1990
  16. 16. P-value LOE Imprecisio n Wording Example < 0.05 A/B No There is evidence from # experimental studies and/or # observational studies in favour of [intervention] (<Author> <year>, <Author> <year>, etc.). It was shown that <intervention> resulted in a statistically significant increase/decrease of <outcome>, compared to <comparison> (<Author> <year>). Evidence is of high/moderate quality. Yes There is limited evidence ... It was shown that ... Evidence is of moderate quality and results cannot be considered precise due to limited sample size, lack of data and/or large variability of results. C/D No There is limited evidence ... It was shown that ... Evidence is of low/very low quality. Yes There is limited evidence ... It was shown that ... Evidence is of low/very low quality and results cannot be considered precise due to ...
  17. 17. Example  PICO: Is rice water (I) effective compared to standard Oral Rehydration Solution (ORS) (C) to improve diarrhoea (O) in adults or children with diarrhoea (P)?  Evidence conclusion: There is limited evidence from 2 experimental studies in favour of rice water (Metha 1986, Fakhir 1990). It was shown that rice water resulted in a statistically significant decrease of duration of purging, stool frequency on day 2 to 4 and stool volume on day 4 compared to using standard ORS. Evidence is of low quality and results cannot be considered precise due to a limited sample size.
  18. 18. Take home message  The use of standard wording in our evidence conclusions resulted in: − standardized training of new staff members − guidelines with uniform evidence statements − a better interpretation of the available evidence − more transparency in our methodology  Recommendation for guideline developers: − use standard wording of evidence conclusions, in order to improve consistency within one guideline and between guidelines
  19. 19. Acknowledgements  Thanks to my colleagues Tessa, Nele and Hans for the fruitful discussions concerning this subject.
  20. 20. Centre for Evidence-Based Practice  Thank you for your attention !  Questions ?  Contact:  Cebap@rodekruis.be  Emmy.Debuck@rodekruis.be

×