Plenary 1 Summing up: challenges and possible solutions when summarising the  findings of Cochrane reviews Claire Glenton ...
Summing up:  challenges and possible solutions when summarising the findings of Cochrane reviews Claire Glenton,  Nordic C...
Past achievements, future challenges <ul><li>Cochrane Library: 4410 Cochrane reviews and protocols  </li></ul><ul><li>Repu...
Improving access by summarising results <ul><li>Cochrane reviews already summarised in: </li></ul><ul><li>journals and com...
The Cochrane Collaboration can improve understanding and avoid the possibility of misrepresentation.. <ul><li>..by develop...
Experiences when summarising Cochrane reviews <ul><li>Norwegian branch, Nordic Cochrane Centre </li></ul><ul><li>The GRADE...
We have tried to extract and consistently present what the reviews say about: <ul><li>what is known about benefits  and  h...
Challenges we faced  when trying to answer these questions <ul><li>Missing information about important outcomes </li></ul>...
Challenges 1.  Missing information about important outcomes <ul><li>Often, data for important outcomes is missing or it is...
Challenges  2.  Continuous outcomes often difficult to interpret <ul><li>” 10 more points on the XY-Z scale ” </li></ul><u...
Challenges   3. Effect presented in different ways Risk difference Odds ratio Numbers Needed to Treat Qualitative presenta...
Challenges  3 . Quality of evidence evaluated in different ways <ul><ul><li>Quality of each individual study </li></ul></u...
<ul><li>what is known </li></ul><ul><li>what is not known </li></ul><ul><li>how sure we can be </li></ul>How can we give a...
Solution: Summary of Findings Tables (SoF)
SoF Tables Focus on important outcomes only <ul><li>Authors encouraged to include: </li></ul><ul><ul><li>Outcomes with no ...
SoF Tables Present effect consistently <ul><li>OR </li></ul><ul><li>RD </li></ul><ul><li>RRR </li></ul><ul><li>RRI Risk Ra...
SoF Tables Continuous outcomes still a challenge <ul><li>WMD </li></ul><ul><li>Risk Ratio and Absolute Risk? </li></ul><ul...
SoF Tables Present quality consistently <ul><li>Quality for each outcome evaluated using the GRADE system  </li></ul>
<ul><li>SoF Tables are precise and consistent overviews </li></ul><ul><li>May still be difficult to interpret for many tar...
While statistics may be precise <ul><li>...they are  often hard to understand </li></ul><ul><li>Different statistical pres...
Qualitative presentations not the solution   <ul><li>Current lay summaries often rely on qualitative statements 1 </li></u...
Event rates (“10 people out of 100”) <ul><li>More precise than qualitative statements  </li></ul><ul><li>Easier to underst...
Lay version of SoF Tables   Event rates Standardised comments when data not statistically significant
Continuous outcomes: problems still remain <ul><li>For continuous outcomes that are hard to understand  - standardised qua...
Tables as a basis for Plain Language Summaries <ul><li>Description of the treatment and who the results apply to </li></ul...
Future plans <ul><li>SoF Tables software available with RevMan 5 </li></ul><ul><li>Continued testing and improvement of fo...
Conclusion: All summaries involve judgement and can lead to loss of precision, but... <ul><li>... better to do this oursel...
Upcoming SlideShare
Loading in...5
×

The Cochrane Collaboration Colloquium: Summing up: challenges and possible solutions when summarising the findings of Cochrane reviews

983

Published on

Claire Glenton speaking at the XIV Cochrane Colloquium in Dublin, Ireland

Published in: Health & Medicine, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
983
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Transcript of "The Cochrane Collaboration Colloquium: Summing up: challenges and possible solutions when summarising the findings of Cochrane reviews"

    1. 1. Plenary 1 Summing up: challenges and possible solutions when summarising the findings of Cochrane reviews Claire Glenton Pembroke
    2. 2. Summing up: challenges and possible solutions when summarising the findings of Cochrane reviews Claire Glenton, Nordic Cochrane Centre, Norwegian branch/ Norwegian Knowledge Centre for the Health Services
    3. 3. Past achievements, future challenges <ul><li>Cochrane Library: 4410 Cochrane reviews and protocols </li></ul><ul><li>Reputation for reliability </li></ul><ul><li>Our next challenge: to improve the accessibility of this information </li></ul>
    4. 4. Improving access by summarising results <ul><li>Cochrane reviews already summarised in: </li></ul><ul><li>journals and commercial websites </li></ul><ul><li>HTAs and guidelines </li></ul><ul><li>Cochrane products </li></ul><ul><li>the media </li></ul>
    5. 5. The Cochrane Collaboration can improve understanding and avoid the possibility of misrepresentation.. <ul><li>..by developing its own summaries that are: </li></ul><ul><li>Precise </li></ul><ul><li>Consistent </li></ul><ul><li>Understandable </li></ul>
    6. 6. Experiences when summarising Cochrane reviews <ul><li>Norwegian branch, Nordic Cochrane Centre </li></ul><ul><li>The GRADE Working Group </li></ul><ul><li>The Cochrane Back Group </li></ul><ul><li>The Cochrane Musculoskeletal Group </li></ul><ul><li>The Cochrane Consumer Network </li></ul>
    7. 7. We have tried to extract and consistently present what the reviews say about: <ul><li>what is known about benefits and harms </li></ul><ul><li>what is not known about benefits and harms </li></ul><ul><li>how sure we can be about this information </li></ul>
    8. 8. Challenges we faced when trying to answer these questions <ul><li>Missing information about important outcomes </li></ul><ul><li>Continuous outcomes that are difficult to interpret </li></ul><ul><li>Variations in how effect and quality are presented </li></ul>
    9. 9. Challenges 1. Missing information about important outcomes <ul><li>Often, data for important outcomes is missing or it is not significant 1 </li></ul><ul><li>Data on adverse effects sometimes missing: </li></ul><ul><ul><li>not always looked for in reviews 1 </li></ul></ul><ul><ul><li>not always found in individual trials 1 </li></ul></ul><ul><li>When data is missing: important to convey this lack of information rather than present secondary outcomes </li></ul>1 Vist 2003
    10. 10. Challenges 2. Continuous outcomes often difficult to interpret <ul><li>” 10 more points on the XY-Z scale ” </li></ul><ul><li>Hard to understand unfamiliar scales </li></ul><ul><li>Information is sometimes missing about scale range 1 </li></ul>1 Kho, Glenton, Pennick, Koy, Underland, Vist, Oxman, 2003
    11. 11. Challenges 3. Effect presented in different ways Risk difference Odds ratio Numbers Needed to Treat Qualitative presentations Relative risk reduction Relative risk improvement Weighted Mean Difference Percentages Standard mean difference Risk ratio
    12. 12. Challenges 3 . Quality of evidence evaluated in different ways <ul><ul><li>Quality of each individual study </li></ul></ul><ul><ul><li>Quality of all studies together </li></ul></ul><ul><ul><li>Quality of each individual outcome 1,2 </li></ul></ul>1 Vist 2003 2 Kho, Glenton, Pennick, Koy, Underland, Vist, Oxman, 2003
    13. 13. <ul><li>what is known </li></ul><ul><li>what is not known </li></ul><ul><li>how sure we can be </li></ul>How can we give a precise, consistent and understandable presentation of: ?
    14. 14. Solution: Summary of Findings Tables (SoF)
    15. 15. SoF Tables Focus on important outcomes only <ul><li>Authors encouraged to include: </li></ul><ul><ul><li>Outcomes with no data or non-significant data </li></ul></ul><ul><ul><li>Outcomes tied to adverse effects </li></ul></ul>
    16. 16. SoF Tables Present effect consistently <ul><li>OR </li></ul><ul><li>RD </li></ul><ul><li>RRR </li></ul><ul><li>RRI Risk Ratio and Absolute Risk (# out of 100) </li></ul><ul><li>RD </li></ul><ul><li>% </li></ul><ul><li>NNT </li></ul>
    17. 17. SoF Tables Continuous outcomes still a challenge <ul><li>WMD </li></ul><ul><li>Risk Ratio and Absolute Risk? </li></ul><ul><li>SMD </li></ul><ul><li> </li></ul><ul><li>Information about scales for continuous outcomes </li></ul>
    18. 18. SoF Tables Present quality consistently <ul><li>Quality for each outcome evaluated using the GRADE system </li></ul>
    19. 19. <ul><li>SoF Tables are precise and consistent overviews </li></ul><ul><li>May still be difficult to interpret for many target groups, including: </li></ul><ul><ul><li>patients and consumers </li></ul></ul><ul><ul><li>journalists </li></ul></ul><ul><ul><li>other “non-statisticians” </li></ul></ul>Improving accessibility to lay people
    20. 20. While statistics may be precise <ul><li>...they are often hard to understand </li></ul><ul><li>Different statistical presentations also affect the decisions that people make 1 (risk ratios and absolute risk) </li></ul>1 Moxey, O’Connell, McGettigan, Henry, 2003
    21. 21. Qualitative presentations not the solution <ul><li>Current lay summaries often rely on qualitative statements 1 </li></ul><ul><li>” Medication is likely to lead to less pain” </li></ul><ul><li>Qualitative presentations are interpreted differently by different people 2 </li></ul>1 Allen, Smith, Thornton 2001 2 Wills, Holmes-Rovner, 2003
    22. 22. Event rates (“10 people out of 100”) <ul><li>More precise than qualitative statements </li></ul><ul><li>Easier to understand than probabilities 3 </li></ul><ul><li>Easy to extract from SoF Tables </li></ul><ul><li>Have their own shortcomings </li></ul><ul><ul><li>imply high level of precision </li></ul></ul><ul><ul><li>mask results that are not statistically significant </li></ul></ul>1 Schünemann, Herrin, Vist, Oxman, Akl, 2003
    23. 23. Lay version of SoF Tables Event rates Standardised comments when data not statistically significant
    24. 24. Continuous outcomes: problems still remain <ul><li>For continuous outcomes that are hard to understand - standardised qualitative statements </li></ul>
    25. 25. Tables as a basis for Plain Language Summaries <ul><li>Description of the treatment and who the results apply to </li></ul><ul><li>Qualitative description of effect </li></ul><ul><li>Lay version of Summary of Findings Table </li></ul>
    26. 26. Future plans <ul><li>SoF Tables software available with RevMan 5 </li></ul><ul><li>Continued testing and improvement of format and content of Tables </li></ul>
    27. 27. Conclusion: All summaries involve judgement and can lead to loss of precision, but... <ul><li>... better to do this ourselves in a systematic and explicit fashion so that we can support decision makers in the best possible way </li></ul>

    ×