Carlos Cuello, profile picture
Uploaded byCarlos Cuello
2,977 views

GRADE approach for making recommendations in clinical practice guidelines #wac2015

The document outlines the development and importance of clinical practice guidelines, which are designed to optimize patient care through evidence-based recommendations. It discusses the challenges in research methods and the need for trustworthy guidelines that consider patient values and preferences. Furthermore, it emphasizes the necessity of systematic reviews and transparent processes to ensure the reliability and effectiveness of medical recommendations.

Embed presentation

Downloaded 114 times
1 / 105
2 / 105
Most read
3 / 105
4 / 105
5 / 105
6 / 105
7 / 105
8 / 105
9 / 105
10 / 105
11 / 105
12 / 105
13 / 105
14 / 105
Most read
15 / 105
16 / 105
17 / 105
18 / 105
19 / 105
20 / 105
21 / 105
22 / 105
23 / 105
24 / 105
25 / 105
26 / 105
27 / 105
28 / 105
29 / 105
30 / 105
31 / 105
32 / 105
33 / 105
34 / 105
35 / 105
36 / 105
37 / 105
38 / 105
39 / 105
40 / 105
41 / 105
42 / 105
43 / 105
44 / 105
45 / 105
46 / 105
47 / 105
48 / 105
49 / 105
50 / 105
Most read
51 / 105
52 / 105
53 / 105
54 / 105
55 / 105
56 / 105
57 / 105
58 / 105
59 / 105
60 / 105
61 / 105
62 / 105
63 / 105
64 / 105
65 / 105
66 / 105
67 / 105
68 / 105
69 / 105
70 / 105
71 / 105
72 / 105
73 / 105
74 / 105
75 / 105
76 / 105
77 / 105
78 / 105
79 / 105
80 / 105
81 / 105
82 / 105
83 / 105
84 / 105
85 / 105
86 / 105
87 / 105
88 / 105
89 / 105
90 / 105
91 / 105
92 / 105
93 / 105
94 / 105
95 / 105
96 / 105
97 / 105
98 / 105
99 / 105
100 / 105
101 / 105
102 / 105
103 / 105
104 / 105
105 / 105
Carlos A. Cuello Garcia MD, PhD(c) Health Research Methodology Program Department of Clinical Epidemiology and Biostatistics Guideline development Levels of evidence and the GRADE approach
DISCLOSURE • currently working at MacGRADE centre & Cochrane GRADEing group • Cochrane author • I have received support from the WAO for travel / meetings • no other COIs related to this talk GRADEing Methods
OUTLINE • what are clinical practice guidelines? • why should we use them? • what are recommendations and levels of evidence? • how to interpret a CPG recommendation?
ATOPIC DERMATITIS
EASY! PIMECROLIMUS
EASY! PIMECROLIMUS WHAT ABOUT CANCER REPORTED?
WE BETTER STAY WITH TOPICAL CORTICOSTEROIDS
WE BETTER STAY WITH TOPICAL CORTICOSTEROIDS PIMECROLIMUS IS BETTER THAN TCS
? ??
PIMECROLIMUS OR TOPICAL CORTICOSTEROIDS? Meanwhile, at the Ministry of Health…
CLINICAL PRACTICE GUIDELINES ๏ statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options. IOM · AHRQ 2011
CLINICAL RECOMMENDATION EVIDENCE PATIENT VALUES ACCEPTA- BILITY BENEFITS VS HARMS COSTS IN PATIENTS (CLINICAL) GUIDELINES
DECISION EVIDENCE PATIENT VALUES ACCEPTA- BILITY BENEFITS VS HARMS FEASIBILITY EQUITY RESOURCE USE IN PUBLIC HEALTH GUIDELINES
guidelines we can trust ๏ Based on a systematic review of the existing evidence 1
guidelines we can trust ๏ Developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups 2
guidelines we can trust ๏ Considers important patient subgroups and patient preferences 3
guidelines we can trust ๏ Based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest 4
guidelines we can trust ๏ Provides a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of recommendations 5
guidelines we can trust ๏ It is revised as appropriate when important new evidence warrants modifications of recommendations. 6
CLINICAL RECOMMENDATION or DECISION EVIDENCE PATIENT VALUES ACCEPTA- BILITY BENEFITS VS HARMS FEASIBILITY EQUITY RESOURCE USE
CLINICAL RECOMMENDATION or DECISION EVIDENCE
RESEARCH GENERATION PUBLISHING
RESEARCH GENERATION PUBLISHING
RESEARCH GENERATION PUBLISHING
RESEARCH GENERATION PUBLISHING systematic reviews clinical practice guidelines
RESEARCH GENERATION PUBLISHING systematic reviews clinical practice guidelines
RESEARCH GENERATION PUBLISHING systematic reviews clinical practice guidelines
CHALLENGES AHEAD why we need trustworthy clinical practice guidelines?
๏ Every year $100 billion dollars are invested in biomedical research ๏ Only 10% is used to test treatments Chalmers 2009 $
๏ only half of researchers use a systematic review that has been previously done on their topic Chalmers I, Glasziou P. Lancet 2009;374:86-9 Cooper NJ, et al Clin Trials 2005;2:260–4. 50%
๏ only half of researchers use adequate tools to ensure the quality of their work Chalmers I, Glasziou P. Lancet 2009;374:86-9 Cooper NJ, et al Clin Trials 2005;2:260–4. CONSORT
๏ not all research is registered, reported, or published… Cressey D. Secrets of trial data revealed. Nature. 2013 Oct 10;502(7470):154-5 RESEARCH GENERATION
and many times we overlook research priorities and patient important outcomes Chalmers I, Glasziou P. Lancet 2009;374:86-9 Cooper NJ, et al Clin Trials 2005;2:260–4.
100 participants parachute placebo
100 participants 36.5 (2.1) 36.7 (1.9) 4.3 (1.2) 6.1(1.3) 25 (3) 22(2.4) Temp ºC –mean (SD) Fear scale –mean (IQR) Serum adrenaline (mcg/dL) – mean (SD) parachute placebo
100 participants 36.5 (2.1) 36.7 (1.9) 4.3 (1.2) 6.1(1.3) 25 (3) 22(2.4) Temp ºC –mean (SD) Fear scale –mean (IQR) Serum adrenaline (mcg/dL) – mean (SD) p=0.00001 parachute placebo
antiarrhythmics Patients with MI ecainide flecainide placebo ↓ ↓ ↑↑ arrhythmias death ?? Epstein AE, et al. JAMA 1993
outcomes Patients with asthma nitrous oxide placebo O2 sat FEV1 Length of hospital stay Death
so, do you recommend..?
so, do you recommend..? ๏ pimecrolimus or topical corticosteroids for atopic dermatitis?
so, do you recommend..? ๏ pimecrolimus or topical corticosteroids for atopic dermatitis? ๏ nitrous oxide for severe asthma?
so, do you recommend..? ๏ pimecrolimus or topical corticosteroids for atopic dermatitis? ๏ nitrous oxide for severe asthma? ๏ probiotics for allergy prevention?
so, do you recommend..? ๏ pimecrolimus or topical corticosteroids for atopic dermatitis? ๏ nitrous oxide for severe asthma? ๏ probiotics for allergy prevention? ๏ epinephrine for anaphylactic shock?
expert recommendations cross-sectional case series, case reports case-control cohort Randomized trial Systematic review CTFPHE 1979
expert recommendations cross-sectional case series, case reports case-control cohort Randomized trial Systematic review CTFPHE 1979 I good II fair III poor EVIDENCE
expert recommendations cross-sectional case series, case reports case-control cohort Randomized trial Systematic review CTFPHE 1979 I good II fair III poor EVIDENCE A B C RECOMMENDATION D
I want to jump from this airplane
do you recommend a parachute?
Grading of Recommendations Assessment, Development and Evaluation
expert recommendations cross-sectional case series case-control cohort RCT
cross-sectional case series case-control cohort RCT RCT Observational experience is not evidence, it’s a tool
By default... cross-sectional case series case-control cohort RCT RCT Observational HIGH QUALITY LOW QUALITY
QUALITY OF EVIDENCE ๏ You can also call it: ๏ certainty of evidence ๏ confidence in the estimates Balshem H, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401-6
QUALITY OF EVIDENCE Balshem H, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401-6 SYSTEMATIC REVIEW CLINICAL PRACTICE GUIDELINE the extent of our confidence that the estimates of the effect are correct the extent of our confidence that the estimates of the effect are adequate to support a particular decision or recommendation
QUALITY OF EVIDENCE HIGH We are very confident that the true effect lies close to that of the estimate of the effect We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect We have very little confidence in the effect estimate: The true is likely to be substantially different from the estimate of effect MODERATE LOW VERY LOW
how is the quality (confidence) of the evidence determined?
what would make you loose confidence? ๏ Overall, probiotics reduced the risk of eczema in infants when given to their pregnant mothers ๏ RR=0.72 (95% CI 0.61 to 0.85) ๏ 3’509 participants, 15 trials
QUALITY OF EVIDENCE risk of bias STUDY 1 STUDY 2 STUDY 3
QUALITY OF EVIDENCE inconsistencyrisk of bias STUDY 1 STUDY 2 STUDY 3
QUALITY OF EVIDENCE inconsistency indirectness P I C O ? risk of bias STUDY 1 STUDY 2 STUDY 3
QUALITY OF EVIDENCE inconsistency indirectness P I C O ? imprecision risk of bias STUDY 1 STUDY 2 STUDY 3
QUALITY OF EVIDENCE inconsistency indirectness P I C O ? imprecision publication bias risk of bias STUDY 1 STUDY 2 STUDY 3
By default... cross-sectional case series, etc. case-control cohort RCT RCT Observational HIGH QUALITY LOW QUALITY
By default... RCT Observational HIGH QUALITY LOW QUALITY
RCT HIGH MODERATE LOW VERY LOW QUALITY OF EVIDENCE
RCT HIGH MODERATE LOW VERY LOW QUALITY OF EVIDENCE risk of bias
HIGH MODERATE LOW VERY LOW QUALITY OF EVIDENCE RCT risk of bias
HIGH MODERATE LOW VERY LOW QUALITY OF EVIDENCE RCT risk of bias inconsistency
HIGH MODERATE LOW VERY LOW QUALITY OF EVIDENCE RCT risk of bias inconsistency imprecision
HIGH MODERATE LOW VERY LOW QUALITY OF EVIDENCE RCT risk of bias inconsistency imprecision indirectness
HIGH MODERATE LOW VERY LOW QUALITY OF EVIDENCE RCT risk of bias inconsistency imprecision indirectness publication bias
VERY LOW QUALITY OF EVIDENCE RCT risk of bias inconsistency imprecision indirectness publication bias
OBSERVATIONAL HIGH MODERATE LOW VERY LOW QUALITY OF EVIDENCE
OBSERVATIONAL HIGH MODERATE LOW VERY LOW QUALITY OF EVIDENCE strong association
HIGH MODERATE LOW VERY LOW QUALITY OF EVIDENCE OBSERVATIONAL strong association dose response
HIGH MODERATE LOW VERY LOW QUALITY OF EVIDENCE OBSERVATIONAL strong association dose response opposingresidual confounding
HIGH QUALITY OF EVIDENCE OBSERVATIONAL strong association dose response opposingresidual confounding
from evidence to recommendations
Quality of evidence Strength of a recommendation the extent of our confidence that the estimates of an effect are adequate to support a particular decision or recommendation the extent to which one can be confident that the desirable consequences of an intervention outweigh its undesirable consequences
NO YES YES NO strengthofarecommendation direction of a recommendation for against
NO YES YES NO strengthofarecommendation direction of a recommendation for against STRONG
NO YES YES NO strengthofarecommendation direction of a recommendation for against CONDITIONAL STRONG
patients clinicians policy makers Most individuals in this situation would want the recommended course of action, and only a small proportion would not Most individuals should receive the intervention. Adherence to this recommendation could be used as a quality criterion or performance indicator The recommendation can be adopted as policy in most situations. STRONG CONDITIONAL The majority of individuals in this situation would want the suggested course of action, but many would not Different choices will be appropriate for individual patients. Must help each patient arrive at a management decision consistent with his or her values and preferences. Policy making will require substantial debate and involvement of various stakeholders.
FRAMEWORKS evidence to recommendation
just 4 columns & conclusions
CRITERIA
CRITERIA Problem Quality of evidence Benefits & harms Values Resource use Equity Acceptability Feasibility
CRITERIA Problem Quality of evidence Benefits & harms Values Resource use Equity Acceptability Feasibility JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION
CRITERIA Problem Quality of evidence Benefits & harms Values Resource use Equity Acceptability Feasibility JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION what is to be considered?
CRITERIA Problem Quality of evidence Benefits & harms Values Resource use Equity Acceptability Feasibility JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION what do we consider about it?
CRITERIA Problem Quality of evidence Benefits & harms Values Resource use Equity Acceptability Feasibility JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION what do we consider about it?
CRITERIA Problem Quality of evidence Benefits & harms Values Resource use Equity Acceptability Feasibility JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION based on what evidence?
CONCLUSIONS Balance of consequences Decision /recommendation Justification Implementation considerations Monitoring Evaluation Research priorities
example
Explanations Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 1 Evidence to decision framework Question 1: Should probiotics vs. no probiotics be used in pregnant women? Population:!pregnant!women! Option:!probiotics! Comparison:!no!probiotics! Setting:3outpatient! Perspective:3individual!patient Background: The$intestinal$microbiome$could$play$an$important$role$in$the$immune$system$maturation,$ and$it$has$been$suggested$that$early6life$probiotic$administration,$whether$directly$to$the$infant$or$in$ their$mothers$breast$milk,$may$reduce$the$risk$of$allergies$in$childhood.$The$objective$of$this$question$is$ to$evaluate$the$impact$of$probiotics$administered$to$the$expecting$mothers$on$their$infant. Subgroup considerations: subpopulation of women at high risk for allergy in a child CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION PROBLEM Is the problem a priority? No Probably no Uncertain Probably yes Yes X Allergic diseases represent a spectrum of health conditions and a worldwide burden in different populations. (1) Are a large number of people affected? No Probably no Uncertain Probably yes Yes X As many as 40% of the worldwide population is affected by any type of allergy. In infants prevalence depends highly on the allergic status of their parents, being approximately of 10% in those without an allergic parent or sibling, versus 20% to 30% in those with the atopic background in their relatives. (2) Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
Explanations Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 2 CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL CONSIDERATIONS VALUES Is there important uncertainty or variability about how much people value the main outcomes? Important uncertainty or variability Possibly important uncertainty or variability Probably no important uncertainty or variability No important uncertainty or variability No known undesirable outcomes X Detailed judgements The relative importance or values of the main outcomes of interest: Outcome Relative importance Certainty of the evidence Eczema critical low Asthma/wheezing critical low Food allergy critical low Adverse effects critical low We judged that the outcomes eczema, asthma and food allergy are critical for people. The adverse outcomes are probably of high importance and the burden of taking daily pills is limited. Some immunocompromised women might not accept the risk. BENEFITS&HARMSOFTHEOPTIONS What is the overall certainty of the evidence of effectiveness? No included studies Very low Low Moderate High X Summary of findings: Outcome With [intervention] Without [intervention] Difference (per 100) (95%CI) Relative effect (RR) (95%CI) Certainty of the evidence (GRADE) Eczema (follow-up 1 to 5 years) 365/1520 (24%) 484/1515 (31.9%) 9 fewer per 100 (from 4 fewer to 13 fewer) RR 0.72 (0.6 to 0.86) ⊕⊝⊝⊝ VERY LOW Asthma/wheezing (follow-up 2 to 7 years) 143/992 (14.4%) 139/982 (14.2%) 0 fewer per 100 (from 3 fewer to 3 more) RR 0.97 (0.77 to 1.22) ⊕⊕⊝⊝ LOW Food allergy (follow-up 1 to 2 years) 36/279 (12.9%) 41/284 (14.4%) 1 more per 100 (from 3 fewer to 8 more) RR 1.08 (0.73 to 1.59) ⊕⊝⊝⊝ VERY LOW Adverse effects 101/394 (25.6%) 88/397 (22.2%) 3 more per 100 (from 4 fewer to 12 more) RR 1.13 (0.82 to 1.52) ⊕⊝⊝⊝ VERY LOW Link to detailed evidence profile Subgroup considerations: Link(s) to summary of findings and judgments for subgroups The data are indirect for all outcomes because they are primarily derived from studies that looked at mixed exposure in women during pregnancy and breastfeeding and of infants after birth. Only 1 RCT assessed the effect on eczema in pregnant women only: RR 0.88 (0.63 to 1.22); RD 5 fewer per 100 (from 14 fewer to 9 more) 5 RCTs included pregnant women + later breastfeeding mothers: RR 0.5 (0.4 to 0.63); RD 21 fewer per 100 (from 15 fewer to 25 fewer) 5 RCTs included pregnant women + infants after birth (follow-up 1 to 5 years): RR 0.87 (0.72 to 1.04), RD 4 fewer per 100 (from 8 fewer to 1 more) 3 RCTs included pregnant women + subsequently breastfeeding + infants (follow-up 3 to 4 years): RR 0.78 (0.49 to 1.24); RD 7 fewer per 100 (from 17 fewer to 8 more) No effects were observed on asthma/wheezing and food allergy. How substantial are the desirable anticipated effects? Don’t know Not important Somewhat important Moderately important Very important Varies X X Detailed judgements There was some disagreement among panel members whether the effect is somewhat or moderately important. How substantial are the undesirable anticipated effects? Don’t know Very important Moderately important Somewhat important Not important Varies X Detailed judgements No serious adverse effects, and no difference in mild adverse effects between the groups. Do the desirable effects outweigh the undesirable effects? No Probably No Don’t know Probably Yes Yes Varies X Detailed judgements Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
Explanations Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 2 CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL CONSIDERATIONS VALUES Is there important uncertainty or variability about how much people value the main outcomes? Important uncertainty or variability Possibly important uncertainty or variability Probably no important uncertainty or variability No important uncertainty or variability No known undesirable outcomes X Detailed judgements The relative importance or values of the main outcomes of interest: Outcome Relative importance Certainty of the evidence Eczema critical low Asthma/wheezing critical low Food allergy critical low Adverse effects critical low We judged that the outcomes eczema, asthma and food allergy are critical for people. The adverse outcomes are probably of high importance and the burden of taking daily pills is limited. Some immunocompromised women might not accept the risk. BENEFITS&HARMSOFTHEOPTIONS What is the overall certainty of the evidence of effectiveness? No included studies Very low Low Moderate High X Summary of findings: Outcome With [intervention] Without [intervention] Difference (per 100) (95%CI) Relative effect (RR) (95%CI) Certainty of the evidence (GRADE) Eczema (follow-up 1 to 5 years) 365/1520 (24%) 484/1515 (31.9%) 9 fewer per 100 (from 4 fewer to 13 fewer) RR 0.72 (0.6 to 0.86) ⊕⊝⊝⊝ VERY LOW Asthma/wheezing (follow-up 2 to 7 years) 143/992 (14.4%) 139/982 (14.2%) 0 fewer per 100 (from 3 fewer to 3 more) RR 0.97 (0.77 to 1.22) ⊕⊕⊝⊝ LOW Food allergy (follow-up 1 to 2 years) 36/279 (12.9%) 41/284 (14.4%) 1 more per 100 (from 3 fewer to 8 more) RR 1.08 (0.73 to 1.59) ⊕⊝⊝⊝ VERY LOW Adverse effects 101/394 (25.6%) 88/397 (22.2%) 3 more per 100 (from 4 fewer to 12 more) RR 1.13 (0.82 to 1.52) ⊕⊝⊝⊝ VERY LOW Link to detailed evidence profile Subgroup considerations: Link(s) to summary of findings and judgments for subgroups The data are indirect for all outcomes because they are primarily derived from studies that looked at mixed exposure in women during pregnancy and breastfeeding and of infants after birth. Only 1 RCT assessed the effect on eczema in pregnant women only: RR 0.88 (0.63 to 1.22); RD 5 fewer per 100 (from 14 fewer to 9 more) 5 RCTs included pregnant women + later breastfeeding mothers: RR 0.5 (0.4 to 0.63); RD 21 fewer per 100 (from 15 fewer to 25 fewer) 5 RCTs included pregnant women + infants after birth (follow-up 1 to 5 years): RR 0.87 (0.72 to 1.04), RD 4 fewer per 100 (from 8 fewer to 1 more) 3 RCTs included pregnant women + subsequently breastfeeding + infants (follow-up 3 to 4 years): RR 0.78 (0.49 to 1.24); RD 7 fewer per 100 (from 17 fewer to 8 more) No effects were observed on asthma/wheezing and food allergy. How substantial are the desirable anticipated effects? Don’t know Not important Somewhat important Moderately important Very important Varies X X Detailed judgements There was some disagreement among panel members whether the effect is somewhat or moderately important. How substantial are the undesirable anticipated effects? Don’t know Very important Moderately important Somewhat important Not important Varies X Detailed judgements No serious adverse effects, and no difference in mild adverse effects between the groups. Do the desirable effects outweigh the undesirable effects? No Probably No Don’t know Probably Yes Yes Varies X Detailed judgements Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
Explanations Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 3 CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL CONSIDERATIONS RESOURCEUSE How large are the resource requirements? Large costs Moderate costs Small Moderate savings Large savings Varies X Detailed judgements Prices are likely to vary substantially depending on the setting. This may be a particularly important consideration in low and middle-income countries. A level and type of insurance may play a substantial role as well. From a health systems point of view it might also be cost effective given that probiotic would be used for 9 months and cost of treatment of eczema may be distributed across many years. Extremely limited research evidence (internet searches of drug prices) Bifidobacterium bifidum (cost per person per year US$) Dose: 1 pill each day Lactobacillus gg (cost per person per year US$) 1 pill each day North- America Average $181.16 $341.6 South- America Average $174.3 $286 Europe Average $167.86 $251.56 Fewer office visits would occur as a result of eczema if the effects on eczema were true. How large is the incremental cost relative to the net benefit? Very large ICER Large ICER Moderate ICER Small ICER Savings Varies Detailed judgements No research evidence If eczema was reduced the intervention might be cost- effective given fewer office visits (between $17,400 and $34,100 to treat 100 people for 1 year or ¾ of that for 9 months) preventing 9 cases of eczema. In most studies probiotics were used in the last trimester of pregnancy, which, if used this same way, might reduce the cost per pregnant woman. EQUITY What would be the impact on health inequities? Increased Probably increased Uncertain Probably reduced Reduced Varies X Detailed judgements No research evidence In some settings it may be important to consider equity as the access may depend on socioeconomic status of the country or setting where coverage will depend on policymakers. ACCEPTABILITY Is the option acceptable to key stakeholders? No Probably No Uncertain Probably Yes Yes Varies X Detailed judgements No research evidence Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
Explanations Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 4 CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL CONSIDERATIONS FEASIBILITY Is the option feasible to implement? No Probably No Uncertain Probably Yes Yes Varies X Detailed judgements No research evidence Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
Explanations Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 5 Recommendation Should probiotics vs. no probiotics be used in pregnant women (exposing their children in utero)? Overall balance of consequences Undesirable consequences clearly outweigh desirable consequences Undesirable consequences probably outweigh desirable consequences The balance of desirable and undesirable consequences indicates they are very similar* Desirable consequences probably outweigh undesirable consequences Desirable consequences clearly outweigh undesirable consequences ! ! ! X ! We recommend against the option or for the alternative We suggest not to use the option or to use the alternative We suggest using the option We recommend the option ! ! X ! Panel decisions 3 panel members with potential COI recused themselves from participating in formulating the recommendation. Consensus was obtained from the rest of the team. Recommendation (text) The guideline panel suggests using probiotics in pregnant women at high risk for allergy in their children (conditional recommendation, very low quality evidence). Remarks and justification Most studies commenced probiotics in the last trimester of pregnancy. The very low quality evidence for adverse effects indicates that our confidence in the absence of increased adverse effects is low. Future research is needed (see definitions of very low quality) e.g., generalizing to immune-compromised children Subgroup considerations Women with high risk of allergy in their children Women with average risk of allergy in their children Implementation considerations This recommendation is based on trials investigating the probiotics or mixtures of probiotic listed below. We have not found a difference between these different probiotics, but that does not mean there is no difference Monitoring and evaluation considerations Research priorities Develop instruments for evaluating the risk of allergy in children as the family history predicts only about 30% of the population risk. There is some evidence that first child is at higher risk for allergy than subsequent children. Long-term follow-up of long-term effects. No direct evidence for the use of probiotics in formula – this should be evaluated in future research and is an unmet need. Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
OUTLINE • what are clinical practice guidelines? • why should we use them? • what are recommendations and levels of evidence? • how to interpret a CPG recommendation?
tools / more readings • guidelinedevelopment.org • cebgrade.mcmaster.ca • gradeing.cochrane.org GRADEing Methods
Thank you! cuelloca@mcmaster.ca @CharlieNeck

More Related Content

PPT
Clinical Audit Overview
byFastbleep
 
PPT
Quality and reliability in health care
byKiran Hanjar
 
PPTX
The journal club (education purpose)
byHarikesh Maurya
 
PPTX
Clinical audit presentation
byfaheta
 
PPTX
Clinical Governance Presentation by Michael Gorton AM - 21 July 2016
byRussell_Kennedy
 
PPT
Aun qa
byAkanit Ks
 
PDF
Patient recruitment into clinical trials presentation
byBeshr Nammouz
 
PPTX
NCCMT Spotlight Webinar - The Consolidated Framework for Implementation Resea...
byThe National Collaborating Centre for Methods and Tools
 
Clinical Audit Overview
byFastbleep
 
Quality and reliability in health care
byKiran Hanjar
 
The journal club (education purpose)
byHarikesh Maurya
 
Clinical audit presentation
byfaheta
 
Clinical Governance Presentation by Michael Gorton AM - 21 July 2016
byRussell_Kennedy
 
Aun qa
byAkanit Ks
 
Patient recruitment into clinical trials presentation
byBeshr Nammouz
 
NCCMT Spotlight Webinar - The Consolidated Framework for Implementation Resea...
byThe National Collaborating Centre for Methods and Tools
 

What's hot

PPT
Presentation on Good Clinical Practices (GCP) By Anubhav Singh m.pharm 1st year
byAnubhav Singh
 
PPT
Clinical Audit Presentation
byJohn O'Connor
 
PPTX
Quality Objectives
byGanapathi subramania sreedharan
 
PPT
Writing A Health Research Proposal
bySoha Rashed
 
PPTX
Vencouver styleppt
bychanu Bhattacharya
 
PPTX
Validity of a Research Tool
byjobyVarghese22
 
PPTX
Writing your thesis chapter by chapter
byRoger Watson
 
PPTX
Clinical audit by Dr A. K. Khandelwal
byDr.Ashok Khandelwal
 
PPTX
Medical Manuscript Writing
byCognibrain Healthcare
 
PPTX
Quality Assurance in Hospitals
byNc Das
 
PPT
Overview of Audits and Inspections in Clinical Research
byAshish K Awadhiya
 
PPTX
Interim analysis in clinical trials (1)
byADITYA CHAKRABORTY
 
PPTX
Clinical audit
byfaheta
 
PPTX
Clinilal audit
byNc Das
 
PPTX
Medical Writing: Answering Your Questions
byCognibrain Healthcare
 
PPTX
Critical analysis of research article
byDr.Nilima Sonawane
 
PPTX
Presentation how to write a research protocol
bySushma Sharma
 
PPTX
Qaulity improvement
byMahmoud Shaqria
 
PDF
Real-World Evidence: The Future of Data Generation and Usage
byApril Bright
 
PPT
Research Proposals
bylcslidepresentations
 
Presentation on Good Clinical Practices (GCP) By Anubhav Singh m.pharm 1st year
byAnubhav Singh
 
Clinical Audit Presentation
byJohn O'Connor
 
Quality Objectives
byGanapathi subramania sreedharan
 
Writing A Health Research Proposal
bySoha Rashed
 
Vencouver styleppt
bychanu Bhattacharya
 
Validity of a Research Tool
byjobyVarghese22
 
Writing your thesis chapter by chapter
byRoger Watson
 
Clinical audit by Dr A. K. Khandelwal
byDr.Ashok Khandelwal
 
Medical Manuscript Writing
byCognibrain Healthcare
 
Quality Assurance in Hospitals
byNc Das
 
Overview of Audits and Inspections in Clinical Research
byAshish K Awadhiya
 
Interim analysis in clinical trials (1)
byADITYA CHAKRABORTY
 
Clinical audit
byfaheta
 
Clinilal audit
byNc Das
 
Medical Writing: Answering Your Questions
byCognibrain Healthcare
 
Critical analysis of research article
byDr.Nilima Sonawane
 
Presentation how to write a research protocol
bySushma Sharma
 
Qaulity improvement
byMahmoud Shaqria
 
Real-World Evidence: The Future of Data Generation and Usage
byApril Bright
 
Research Proposals
bylcslidepresentations
 

Viewers also liked

PDF
Biomarcadores para neuroinfecciones en pediatría
byCarlos Cuello
 
PPTX
Towards evidence-based clinical practice guidelines implementation at King Sa...
byYasser Sami Abdel Dayem Amer
 
PDF
EVIDENCE-BASED CLINICAL PRACTICE GUIDELINES for the Staff of NICU of King Sau...
byYasser Sami Abdel Dayem Amer
 
PPTX
Theories and evidenced based practice
byNathan Loynes
 
PPTX
Evidenced Based Practice (EVP): GRADE Approach to Evidenced Based Guideline D...
byMichael Changaris
 
PPTX
Evidence-Based Clinical Practice Guidelines (CPGs) Adaptation & Implementati...
byYasser Sami Abdel Dayem Amer
 
PPTX
Breast cancer - current concepts
bymadurai
 
PPTX
JNC VIII GUIDELINES FOR MANAGEMENT OF BLOOD PRESSURE 2013
byPraveen Nagula
 
PPT
Overweight/obesity prevention, treatment, and maintenance from childhood to a...
byHealth Evidence™
 
PPT
Understanding Breast Cancer Guidelines
byfondas vakalis
 
Biomarcadores para neuroinfecciones en pediatría
byCarlos Cuello
 
Towards evidence-based clinical practice guidelines implementation at King Sa...
byYasser Sami Abdel Dayem Amer
 
EVIDENCE-BASED CLINICAL PRACTICE GUIDELINES for the Staff of NICU of King Sau...
byYasser Sami Abdel Dayem Amer
 
Theories and evidenced based practice
byNathan Loynes
 
Evidenced Based Practice (EVP): GRADE Approach to Evidenced Based Guideline D...
byMichael Changaris
 
Evidence-Based Clinical Practice Guidelines (CPGs) Adaptation & Implementati...
byYasser Sami Abdel Dayem Amer
 
Breast cancer - current concepts
bymadurai
 
JNC VIII GUIDELINES FOR MANAGEMENT OF BLOOD PRESSURE 2013
byPraveen Nagula
 
Overweight/obesity prevention, treatment, and maintenance from childhood to a...
byHealth Evidence™
 
Understanding Breast Cancer Guidelines
byfondas vakalis
 

Similar to GRADE approach for making recommendations in clinical practice guidelines #wac2015

PDF
L11 EBM.pdfhshshshhshshshshshhshshshshss
bymustafaroyale203
 
PDF
Lecture 2 EBM Final.pdf for medical management
byJosephmwanika
 
PDF
Evidence to Decision tables – 11th CE&B's research day presentation
byCarlos Cuello
 
PPTX
Growing Evidence in clinical practices.pptx
byMedha Sharma
 
PPT
ebm_3.ppt
byGokulThaamass
 
PPT
ebm_3.ppt
byYANIMULYANI18
 
PPTX
How to search the medical literature on the net
bySamir Haffar
 
PPT
Evidence-based medicine
byDeveloping
 
PPTX
Evidence based practice
byYouttam Laudari
 
PPTX
Introduction to peer group learning through EBM
byAungPyiSoe86
 
PPTX
Evidence-based Medicine (EBM) Applications
byuzairktk
 
PPTX
Evidence-Based Practice_Lecture 3_slides
byCMDLearning
 
PDF
K7 - Critical Appraisal.pdf
byJeslynTengkawan1
 
PPTX
HLinc presentation: levels of evidence
byCatherineVoutier
 
PPTX
Evidence Based Practice -By Ritika Soni
byShimla
 
PPT
EBM Part 1
byUSA Biomedical Library
 
PPTX
240119-Evidence Based Medicine nnnnn.pptx
byMyThaoAiDoan
 
PPTX
evidence- based medicine cpg_PHARMD.pptx
byabrharedae
 
PPTX
Evidence Based Practice_lecture 5_slides
byZakCooper1
 
PPTX
Evidence and Science Based Medicine A Primer.pptx
byKaushik Banerjee
 
L11 EBM.pdfhshshshhshshshshshhshshshshss
bymustafaroyale203
 
Lecture 2 EBM Final.pdf for medical management
byJosephmwanika
 
Evidence to Decision tables – 11th CE&B's research day presentation
byCarlos Cuello
 
Growing Evidence in clinical practices.pptx
byMedha Sharma
 
ebm_3.ppt
byGokulThaamass
 
ebm_3.ppt
byYANIMULYANI18
 
How to search the medical literature on the net
bySamir Haffar
 
Evidence-based medicine
byDeveloping
 
Evidence based practice
byYouttam Laudari
 
Introduction to peer group learning through EBM
byAungPyiSoe86
 
Evidence-based Medicine (EBM) Applications
byuzairktk
 
Evidence-Based Practice_Lecture 3_slides
byCMDLearning
 
K7 - Critical Appraisal.pdf
byJeslynTengkawan1
 
HLinc presentation: levels of evidence
byCatherineVoutier
 
Evidence Based Practice -By Ritika Soni
byShimla
 
EBM Part 1
byUSA Biomedical Library
 
240119-Evidence Based Medicine nnnnn.pptx
byMyThaoAiDoan
 
evidence- based medicine cpg_PHARMD.pptx
byabrharedae
 
Evidence Based Practice_lecture 5_slides
byZakCooper1
 
Evidence and Science Based Medicine A Primer.pptx
byKaushik Banerjee
 

More from Carlos Cuello

PDF
ROBINS I: riesgo de sesgo en estudios no aleatorios
byCarlos Cuello
 
PDF
Guias de practica clinica 2016 (3a parte)
byCarlos Cuello
 
PDF
Guias de practica clinica 2016 2a parte
byCarlos Cuello
 
PDF
Guias de practica clinica 2016 (primera parte): Introducción, alcances, objet...
byCarlos Cuello
 
PDF
El Ensayo Clínico Aleatorio: introducción
byCarlos Cuello
 
PDF
Preguntas CASP México Tratamiento
byCarlos Cuello
 
PDF
Preguntas CASP México Revisiones Sistemáticas
byCarlos Cuello
 
PDF
Introduccion a la colaboracion cochrane
byCarlos Cuello
 
PDF
Pruebas diagnósticas en guías de práctica clínica
byCarlos Cuello
 
PDF
Estudios Observacionales
byCarlos Cuello
 
PDF
Introducción y pregunta clínica estructurada
byCarlos Cuello
 
KEY
EBM Lecture in Los Cabos
byCarlos Cuello
 
PDF
Trauma craneal en pediatría
byCarlos Cuello
 
PDF
3 Redes sociales y el investigador clínico
byCarlos Cuello
 
PDF
2 Los filtros para la informacion
byCarlos Cuello
 
PDF
1 Clinico, investigador, informador y cientifico
byCarlos Cuello
 
PDF
Ensayo clínico de no inferioridad
byCarlos Cuello
 
PDF
BUSQUEDA DE INFORMACION MEDICA
byCarlos Cuello
 
PDF
El efecto placebo
byCarlos Cuello
 
PDF
Ensayo Clínico Aleatorio
byCarlos Cuello
 
ROBINS I: riesgo de sesgo en estudios no aleatorios
byCarlos Cuello
 
Guias de practica clinica 2016 (3a parte)
byCarlos Cuello
 
Guias de practica clinica 2016 2a parte
byCarlos Cuello
 
Guias de practica clinica 2016 (primera parte): Introducción, alcances, objet...
byCarlos Cuello
 
El Ensayo Clínico Aleatorio: introducción
byCarlos Cuello
 
Preguntas CASP México Tratamiento
byCarlos Cuello
 
Preguntas CASP México Revisiones Sistemáticas
byCarlos Cuello
 
Introduccion a la colaboracion cochrane
byCarlos Cuello
 
Pruebas diagnósticas en guías de práctica clínica
byCarlos Cuello
 
Estudios Observacionales
byCarlos Cuello
 
Introducción y pregunta clínica estructurada
byCarlos Cuello
 
EBM Lecture in Los Cabos
byCarlos Cuello
 
Trauma craneal en pediatría
byCarlos Cuello
 
3 Redes sociales y el investigador clínico
byCarlos Cuello
 
2 Los filtros para la informacion
byCarlos Cuello
 
1 Clinico, investigador, informador y cientifico
byCarlos Cuello
 
Ensayo clínico de no inferioridad
byCarlos Cuello
 
BUSQUEDA DE INFORMACION MEDICA
byCarlos Cuello
 
El efecto placebo
byCarlos Cuello
 
Ensayo Clínico Aleatorio
byCarlos Cuello
 

Recently uploaded

PPTX
Central Venous Line: Insertion Technique & Complications
byDr Jebish Pradhan
 
PPTX
Phytochemical Analysis And Antimicrobial Screening Studies Of Calotropis Giga...
bySoukat Garain
 
PPTX
CPCSEA GUIDELINE (COMMITTEE FOR THE PURPOSE OF CONTROL AND SUPERVISION OF EXP...
byVinayPolpelliwar
 
PPTX
Research Methodology_UNIT_V_Declaration of Helsinki M. Pharm (IIIrd Sem.).pptx
byDHANASHREE KOLHEKAR
 
PDF
Lenvatinib Tablets: Uses, Benefits, Diseases Treated & Availability in the Ph...
byLetsMeds
 
PDF
Tarsal Tunnel Syndrome anatomy and management
byMohamed E Elsebaey
 
PPTX
Holistic Oral Care- Danta Davana and Jihwa nilekhana in Ayurveda: Classical ...
bySudha Sumathy
 
PDF
Partograph: It is a composite graphical recording of cervical dilatation and ...
byMayuriGamit2
 
PDF
Ibrutinib 140 mg Capsule Treatment Guide
byLetsMeds
 
PDF
Next-Generation Solutions to Optimize Glioma Care: Applying The Latest Eviden...
byPVI, PeerView Institute for Medical Education
 
PDF
Short notes in Radiation Oncology by SCOPE
byKanhu Charan
 
PPTX
LOW BACK PAIN_BY. R.YUVASHRI BPT FINAL YEAR
byarunmanikandan6
 
PDF
EFRUZHU INFLAMMATION MECHANISM THEOREM AND ADVOCATES.pdf
byEFRUZHUCANCERTHERAPY
 
PPTX
Cardiac tamponade - Recognition & Emergency Management.pptx
byAsokan R
 
PDF
Novo protocolo (7) da ABM sobre Políticas de Apoio à AMAMENTAÇÃO em MATERNIDADES
byProf. Marcus Renato de Carvalho
 
PDF
Prosthetic Solutions Cranio-Maxillo-Facial
byDavide Decesari
 
PPTX
Foundations of Pharmaceutical Quality Assurance and Quality Management System...
byDr. Smita Kumbhar
 
PPTX
Emerging Targets in NSCLC beyond EGFR, ALK & ROS .pptx
bySumitKumar452
 
PPTX
Development of cleft lip and palate.pptx
byMohammed Haneef Farooq
 
PPTX
Surgery instruments for MBBS Students, pptx
bycbmesurgery
 
Central Venous Line: Insertion Technique & Complications
byDr Jebish Pradhan
 
Phytochemical Analysis And Antimicrobial Screening Studies Of Calotropis Giga...
bySoukat Garain
 
CPCSEA GUIDELINE (COMMITTEE FOR THE PURPOSE OF CONTROL AND SUPERVISION OF EXP...
byVinayPolpelliwar
 
Research Methodology_UNIT_V_Declaration of Helsinki M. Pharm (IIIrd Sem.).pptx
byDHANASHREE KOLHEKAR
 
Lenvatinib Tablets: Uses, Benefits, Diseases Treated & Availability in the Ph...
byLetsMeds
 
Tarsal Tunnel Syndrome anatomy and management
byMohamed E Elsebaey
 
Holistic Oral Care- Danta Davana and Jihwa nilekhana in Ayurveda: Classical ...
bySudha Sumathy
 
Partograph: It is a composite graphical recording of cervical dilatation and ...
byMayuriGamit2
 
Ibrutinib 140 mg Capsule Treatment Guide
byLetsMeds
 
Next-Generation Solutions to Optimize Glioma Care: Applying The Latest Eviden...
byPVI, PeerView Institute for Medical Education
 
Short notes in Radiation Oncology by SCOPE
byKanhu Charan
 
LOW BACK PAIN_BY. R.YUVASHRI BPT FINAL YEAR
byarunmanikandan6
 
EFRUZHU INFLAMMATION MECHANISM THEOREM AND ADVOCATES.pdf
byEFRUZHUCANCERTHERAPY
 
Cardiac tamponade - Recognition & Emergency Management.pptx
byAsokan R
 
Novo protocolo (7) da ABM sobre Políticas de Apoio à AMAMENTAÇÃO em MATERNIDADES
byProf. Marcus Renato de Carvalho
 
Prosthetic Solutions Cranio-Maxillo-Facial
byDavide Decesari
 
Foundations of Pharmaceutical Quality Assurance and Quality Management System...
byDr. Smita Kumbhar
 
Emerging Targets in NSCLC beyond EGFR, ALK & ROS .pptx
bySumitKumar452
 
Development of cleft lip and palate.pptx
byMohammed Haneef Farooq
 
Surgery instruments for MBBS Students, pptx
bycbmesurgery
 

GRADE approach for making recommendations in clinical practice guidelines #wac2015

  • 1.
    Carlos A. CuelloGarcia MD, PhD(c) Health Research Methodology Program Department of Clinical Epidemiology and Biostatistics Guideline development Levels of evidence and the GRADE approach
  • 2.
    DISCLOSURE • currently workingat MacGRADE centre & Cochrane GRADEing group • Cochrane author • I have received support from the WAO for travel / meetings • no other COIs related to this talk GRADEing Methods
  • 3.
    OUTLINE • what areclinical practice guidelines? • why should we use them? • what are recommendations and levels of evidence? • how to interpret a CPG recommendation?
  • 5.
  • 6.
  • 7.
  • 8.
    WE BETTER STAYWITH TOPICAL CORTICOSTEROIDS
  • 9.
    WE BETTER STAYWITH TOPICAL CORTICOSTEROIDS PIMECROLIMUS IS BETTER THAN TCS
  • 10.
  • 11.
    PIMECROLIMUS OR TOPICALCORTICOSTEROIDS? Meanwhile, at the Ministry of Health…
  • 12.
    CLINICAL PRACTICE GUIDELINES ๏statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options. IOM · AHRQ 2011
  • 13.
  • 14.
  • 15.
    guidelines we cantrust ๏ Based on a systematic review of the existing evidence 1
  • 16.
    guidelines we cantrust ๏ Developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups 2
  • 17.
    guidelines we cantrust ๏ Considers important patient subgroups and patient preferences 3
  • 18.
    guidelines we cantrust ๏ Based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest 4
  • 19.
    guidelines we cantrust ๏ Provides a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of recommendations 5
  • 20.
    guidelines we cantrust ๏ It is revised as appropriate when important new evidence warrants modifications of recommendations. 6
  • 21.
  • 22.
  • 23.
  • 24.
  • 25.
  • 26.
  • 27.
  • 28.
  • 29.
    CHALLENGES AHEAD why weneed trustworthy clinical practice guidelines?
  • 30.
    ๏ Every year$100 billion dollars are invested in biomedical research ๏ Only 10% is used to test treatments Chalmers 2009 $
  • 31.
    ๏ only halfof researchers use a systematic review that has been previously done on their topic Chalmers I, Glasziou P. Lancet 2009;374:86-9 Cooper NJ, et al Clin Trials 2005;2:260–4. 50%
  • 32.
    ๏ only halfof researchers use adequate tools to ensure the quality of their work Chalmers I, Glasziou P. Lancet 2009;374:86-9 Cooper NJ, et al Clin Trials 2005;2:260–4. CONSORT
  • 33.
    ๏ not allresearch is registered, reported, or published… Cressey D. Secrets of trial data revealed. Nature. 2013 Oct 10;502(7470):154-5 RESEARCH GENERATION
  • 34.
    and many timeswe overlook research priorities and patient important outcomes Chalmers I, Glasziou P. Lancet 2009;374:86-9 Cooper NJ, et al Clin Trials 2005;2:260–4.
  • 35.
  • 36.
    100 participants 36.5 (2.1) 36.7(1.9) 4.3 (1.2) 6.1(1.3) 25 (3) 22(2.4) Temp ºC –mean (SD) Fear scale –mean (IQR) Serum adrenaline (mcg/dL) – mean (SD) parachute placebo
  • 37.
    100 participants 36.5 (2.1) 36.7(1.9) 4.3 (1.2) 6.1(1.3) 25 (3) 22(2.4) Temp ºC –mean (SD) Fear scale –mean (IQR) Serum adrenaline (mcg/dL) – mean (SD) p=0.00001 parachute placebo
  • 38.
  • 39.
    outcomes Patients with asthma nitrous oxide placebo O2sat FEV1 Length of hospital stay Death
  • 40.
    so, do yourecommend..?
  • 41.
    so, do yourecommend..? ๏ pimecrolimus or topical corticosteroids for atopic dermatitis?
  • 42.
    so, do yourecommend..? ๏ pimecrolimus or topical corticosteroids for atopic dermatitis? ๏ nitrous oxide for severe asthma?
  • 43.
    so, do yourecommend..? ๏ pimecrolimus or topical corticosteroids for atopic dermatitis? ๏ nitrous oxide for severe asthma? ๏ probiotics for allergy prevention?
  • 44.
    so, do yourecommend..? ๏ pimecrolimus or topical corticosteroids for atopic dermatitis? ๏ nitrous oxide for severe asthma? ๏ probiotics for allergy prevention? ๏ epinephrine for anaphylactic shock?
  • 45.
    expert recommendations cross-sectional case series,case reports case-control cohort Randomized trial Systematic review CTFPHE 1979
  • 46.
    expert recommendations cross-sectional case series,case reports case-control cohort Randomized trial Systematic review CTFPHE 1979 I good II fair III poor EVIDENCE
  • 47.
    expert recommendations cross-sectional case series,case reports case-control cohort Randomized trial Systematic review CTFPHE 1979 I good II fair III poor EVIDENCE A B C RECOMMENDATION D
  • 48.
    I want tojump from this airplane
  • 49.
    do you recommenda parachute?
  • 51.
    Grading of RecommendationsAssessment, Development and Evaluation
  • 52.
  • 53.
  • 54.
  • 55.
    QUALITY OF EVIDENCE ๏You can also call it: ๏ certainty of evidence ๏ confidence in the estimates Balshem H, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401-6
  • 56.
    QUALITY OF EVIDENCE BalshemH, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol. 2011;64(4):401-6 SYSTEMATIC REVIEW CLINICAL PRACTICE GUIDELINE the extent of our confidence that the estimates of the effect are correct the extent of our confidence that the estimates of the effect are adequate to support a particular decision or recommendation
  • 57.
    QUALITY OF EVIDENCE HIGH Weare very confident that the true effect lies close to that of the estimate of the effect We are moderately confident in the effect estimate: The true effect is likely to be close to the estimate of the effect, but there is a possibility that it is substantially different Our confidence in the effect estimate is limited: The true effect may be substantially different from the estimate of the effect We have very little confidence in the effect estimate: The true is likely to be substantially different from the estimate of effect MODERATE LOW VERY LOW
  • 58.
    how is thequality (confidence) of the evidence determined?
  • 59.
    what would makeyou loose confidence? ๏ Overall, probiotics reduced the risk of eczema in infants when given to their pregnant mothers ๏ RR=0.72 (95% CI 0.61 to 0.85) ๏ 3’509 participants, 15 trials
  • 60.
    QUALITY OF EVIDENCE riskof bias STUDY 1 STUDY 2 STUDY 3
  • 61.
    QUALITY OF EVIDENCE inconsistencyriskof bias STUDY 1 STUDY 2 STUDY 3
  • 62.
  • 63.
  • 64.
  • 65.
    By default... cross-sectional case series,etc. case-control cohort RCT RCT Observational HIGH QUALITY LOW QUALITY
  • 66.
  • 67.
  • 68.
  • 69.
    HIGH MODERATE LOW VERY LOW QUALITY OFEVIDENCE RCT risk of bias
  • 70.
    HIGH MODERATE LOW VERY LOW QUALITY OFEVIDENCE RCT risk of bias inconsistency
  • 71.
    HIGH MODERATE LOW VERY LOW QUALITY OFEVIDENCE RCT risk of bias inconsistency imprecision
  • 72.
    HIGH MODERATE LOW VERY LOW QUALITY OFEVIDENCE RCT risk of bias inconsistency imprecision indirectness
  • 73.
    HIGH MODERATE LOW VERY LOW QUALITY OFEVIDENCE RCT risk of bias inconsistency imprecision indirectness publication bias
  • 74.
    VERY LOW QUALITY OFEVIDENCE RCT risk of bias inconsistency imprecision indirectness publication bias
  • 75.
  • 76.
  • 77.
    HIGH MODERATE LOW VERY LOW QUALITY OFEVIDENCE OBSERVATIONAL strong association dose response
  • 78.
    HIGH MODERATE LOW VERY LOW QUALITY OFEVIDENCE OBSERVATIONAL strong association dose response opposingresidual confounding
  • 79.
  • 80.
  • 81.
    Quality of evidence Strengthof a recommendation the extent of our confidence that the estimates of an effect are adequate to support a particular decision or recommendation the extent to which one can be confident that the desirable consequences of an intervention outweigh its undesirable consequences
  • 82.
  • 83.
    NO YES YES NO strengthofarecommendation direction ofa recommendation for against STRONG
  • 84.
    NO YES YES NO strengthofarecommendation direction ofa recommendation for against CONDITIONAL STRONG
  • 85.
    patients clinicians policy makers Most individualsin this situation would want the recommended course of action, and only a small proportion would not Most individuals should receive the intervention. Adherence to this recommendation could be used as a quality criterion or performance indicator The recommendation can be adopted as policy in most situations. STRONG CONDITIONAL The majority of individuals in this situation would want the suggested course of action, but many would not Different choices will be appropriate for individual patients. Must help each patient arrive at a management decision consistent with his or her values and preferences. Policy making will require substantial debate and involvement of various stakeholders.
  • 86.
  • 87.
  • 88.
  • 89.
    CRITERIA Problem Quality of evidence Benefits& harms Values Resource use Equity Acceptability Feasibility
  • 90.
    CRITERIA Problem Quality of evidence Benefits& harms Values Resource use Equity Acceptability Feasibility JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION
  • 91.
    CRITERIA Problem Quality of evidence Benefits& harms Values Resource use Equity Acceptability Feasibility JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION what is to be considered?
  • 92.
    CRITERIA Problem Quality of evidence Benefits& harms Values Resource use Equity Acceptability Feasibility JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION what do we consider about it?
  • 93.
    CRITERIA Problem Quality of evidence Benefits& harms Values Resource use Equity Acceptability Feasibility JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION what do we consider about it?
  • 94.
    CRITERIA Problem Quality of evidence Benefits& harms Values Resource use Equity Acceptability Feasibility JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION based on what evidence?
  • 95.
    CONCLUSIONS Balance of consequences Decision/recommendation Justification Implementation considerations Monitoring Evaluation Research priorities
  • 96.
  • 97.
    Explanations Prepared by:Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 1 Evidence to decision framework Question 1: Should probiotics vs. no probiotics be used in pregnant women? Population:!pregnant!women! Option:!probiotics! Comparison:!no!probiotics! Setting:3outpatient! Perspective:3individual!patient Background: The$intestinal$microbiome$could$play$an$important$role$in$the$immune$system$maturation,$ and$it$has$been$suggested$that$early6life$probiotic$administration,$whether$directly$to$the$infant$or$in$ their$mothers$breast$milk,$may$reduce$the$risk$of$allergies$in$childhood.$The$objective$of$this$question$is$ to$evaluate$the$impact$of$probiotics$administered$to$the$expecting$mothers$on$their$infant. Subgroup considerations: subpopulation of women at high risk for allergy in a child CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL INFORMATION PROBLEM Is the problem a priority? No Probably no Uncertain Probably yes Yes X Allergic diseases represent a spectrum of health conditions and a worldwide burden in different populations. (1) Are a large number of people affected? No Probably no Uncertain Probably yes Yes X As many as 40% of the worldwide population is affected by any type of allergy. In infants prevalence depends highly on the allergic status of their parents, being approximately of 10% in those without an allergic parent or sibling, versus 20% to 30% in those with the atopic background in their relatives. (2) Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
  • 98.
    Explanations Prepared by:Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 2 CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL CONSIDERATIONS VALUES Is there important uncertainty or variability about how much people value the main outcomes? Important uncertainty or variability Possibly important uncertainty or variability Probably no important uncertainty or variability No important uncertainty or variability No known undesirable outcomes X Detailed judgements The relative importance or values of the main outcomes of interest: Outcome Relative importance Certainty of the evidence Eczema critical low Asthma/wheezing critical low Food allergy critical low Adverse effects critical low We judged that the outcomes eczema, asthma and food allergy are critical for people. The adverse outcomes are probably of high importance and the burden of taking daily pills is limited. Some immunocompromised women might not accept the risk. BENEFITS&HARMSOFTHEOPTIONS What is the overall certainty of the evidence of effectiveness? No included studies Very low Low Moderate High X Summary of findings: Outcome With [intervention] Without [intervention] Difference (per 100) (95%CI) Relative effect (RR) (95%CI) Certainty of the evidence (GRADE) Eczema (follow-up 1 to 5 years) 365/1520 (24%) 484/1515 (31.9%) 9 fewer per 100 (from 4 fewer to 13 fewer) RR 0.72 (0.6 to 0.86) ⊕⊝⊝⊝ VERY LOW Asthma/wheezing (follow-up 2 to 7 years) 143/992 (14.4%) 139/982 (14.2%) 0 fewer per 100 (from 3 fewer to 3 more) RR 0.97 (0.77 to 1.22) ⊕⊕⊝⊝ LOW Food allergy (follow-up 1 to 2 years) 36/279 (12.9%) 41/284 (14.4%) 1 more per 100 (from 3 fewer to 8 more) RR 1.08 (0.73 to 1.59) ⊕⊝⊝⊝ VERY LOW Adverse effects 101/394 (25.6%) 88/397 (22.2%) 3 more per 100 (from 4 fewer to 12 more) RR 1.13 (0.82 to 1.52) ⊕⊝⊝⊝ VERY LOW Link to detailed evidence profile Subgroup considerations: Link(s) to summary of findings and judgments for subgroups The data are indirect for all outcomes because they are primarily derived from studies that looked at mixed exposure in women during pregnancy and breastfeeding and of infants after birth. Only 1 RCT assessed the effect on eczema in pregnant women only: RR 0.88 (0.63 to 1.22); RD 5 fewer per 100 (from 14 fewer to 9 more) 5 RCTs included pregnant women + later breastfeeding mothers: RR 0.5 (0.4 to 0.63); RD 21 fewer per 100 (from 15 fewer to 25 fewer) 5 RCTs included pregnant women + infants after birth (follow-up 1 to 5 years): RR 0.87 (0.72 to 1.04), RD 4 fewer per 100 (from 8 fewer to 1 more) 3 RCTs included pregnant women + subsequently breastfeeding + infants (follow-up 3 to 4 years): RR 0.78 (0.49 to 1.24); RD 7 fewer per 100 (from 17 fewer to 8 more) No effects were observed on asthma/wheezing and food allergy. How substantial are the desirable anticipated effects? Don’t know Not important Somewhat important Moderately important Very important Varies X X Detailed judgements There was some disagreement among panel members whether the effect is somewhat or moderately important. How substantial are the undesirable anticipated effects? Don’t know Very important Moderately important Somewhat important Not important Varies X Detailed judgements No serious adverse effects, and no difference in mild adverse effects between the groups. Do the desirable effects outweigh the undesirable effects? No Probably No Don’t know Probably Yes Yes Varies X Detailed judgements Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
  • 99.
    Explanations Prepared by:Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 2 CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL CONSIDERATIONS VALUES Is there important uncertainty or variability about how much people value the main outcomes? Important uncertainty or variability Possibly important uncertainty or variability Probably no important uncertainty or variability No important uncertainty or variability No known undesirable outcomes X Detailed judgements The relative importance or values of the main outcomes of interest: Outcome Relative importance Certainty of the evidence Eczema critical low Asthma/wheezing critical low Food allergy critical low Adverse effects critical low We judged that the outcomes eczema, asthma and food allergy are critical for people. The adverse outcomes are probably of high importance and the burden of taking daily pills is limited. Some immunocompromised women might not accept the risk. BENEFITS&HARMSOFTHEOPTIONS What is the overall certainty of the evidence of effectiveness? No included studies Very low Low Moderate High X Summary of findings: Outcome With [intervention] Without [intervention] Difference (per 100) (95%CI) Relative effect (RR) (95%CI) Certainty of the evidence (GRADE) Eczema (follow-up 1 to 5 years) 365/1520 (24%) 484/1515 (31.9%) 9 fewer per 100 (from 4 fewer to 13 fewer) RR 0.72 (0.6 to 0.86) ⊕⊝⊝⊝ VERY LOW Asthma/wheezing (follow-up 2 to 7 years) 143/992 (14.4%) 139/982 (14.2%) 0 fewer per 100 (from 3 fewer to 3 more) RR 0.97 (0.77 to 1.22) ⊕⊕⊝⊝ LOW Food allergy (follow-up 1 to 2 years) 36/279 (12.9%) 41/284 (14.4%) 1 more per 100 (from 3 fewer to 8 more) RR 1.08 (0.73 to 1.59) ⊕⊝⊝⊝ VERY LOW Adverse effects 101/394 (25.6%) 88/397 (22.2%) 3 more per 100 (from 4 fewer to 12 more) RR 1.13 (0.82 to 1.52) ⊕⊝⊝⊝ VERY LOW Link to detailed evidence profile Subgroup considerations: Link(s) to summary of findings and judgments for subgroups The data are indirect for all outcomes because they are primarily derived from studies that looked at mixed exposure in women during pregnancy and breastfeeding and of infants after birth. Only 1 RCT assessed the effect on eczema in pregnant women only: RR 0.88 (0.63 to 1.22); RD 5 fewer per 100 (from 14 fewer to 9 more) 5 RCTs included pregnant women + later breastfeeding mothers: RR 0.5 (0.4 to 0.63); RD 21 fewer per 100 (from 15 fewer to 25 fewer) 5 RCTs included pregnant women + infants after birth (follow-up 1 to 5 years): RR 0.87 (0.72 to 1.04), RD 4 fewer per 100 (from 8 fewer to 1 more) 3 RCTs included pregnant women + subsequently breastfeeding + infants (follow-up 3 to 4 years): RR 0.78 (0.49 to 1.24); RD 7 fewer per 100 (from 17 fewer to 8 more) No effects were observed on asthma/wheezing and food allergy. How substantial are the desirable anticipated effects? Don’t know Not important Somewhat important Moderately important Very important Varies X X Detailed judgements There was some disagreement among panel members whether the effect is somewhat or moderately important. How substantial are the undesirable anticipated effects? Don’t know Very important Moderately important Somewhat important Not important Varies X Detailed judgements No serious adverse effects, and no difference in mild adverse effects between the groups. Do the desirable effects outweigh the undesirable effects? No Probably No Don’t know Probably Yes Yes Varies X Detailed judgements Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
  • 100.
    Explanations Prepared by:Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 3 CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL CONSIDERATIONS RESOURCEUSE How large are the resource requirements? Large costs Moderate costs Small Moderate savings Large savings Varies X Detailed judgements Prices are likely to vary substantially depending on the setting. This may be a particularly important consideration in low and middle-income countries. A level and type of insurance may play a substantial role as well. From a health systems point of view it might also be cost effective given that probiotic would be used for 9 months and cost of treatment of eczema may be distributed across many years. Extremely limited research evidence (internet searches of drug prices) Bifidobacterium bifidum (cost per person per year US$) Dose: 1 pill each day Lactobacillus gg (cost per person per year US$) 1 pill each day North- America Average $181.16 $341.6 South- America Average $174.3 $286 Europe Average $167.86 $251.56 Fewer office visits would occur as a result of eczema if the effects on eczema were true. How large is the incremental cost relative to the net benefit? Very large ICER Large ICER Moderate ICER Small ICER Savings Varies Detailed judgements No research evidence If eczema was reduced the intervention might be cost- effective given fewer office visits (between $17,400 and $34,100 to treat 100 people for 1 year or ¾ of that for 9 months) preventing 9 cases of eczema. In most studies probiotics were used in the last trimester of pregnancy, which, if used this same way, might reduce the cost per pregnant woman. EQUITY What would be the impact on health inequities? Increased Probably increased Uncertain Probably reduced Reduced Varies X Detailed judgements No research evidence In some settings it may be important to consider equity as the access may depend on socioeconomic status of the country or setting where coverage will depend on policymakers. ACCEPTABILITY Is the option acceptable to key stakeholders? No Probably No Uncertain Probably Yes Yes Varies X Detailed judgements No research evidence Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
  • 101.
    Explanations Prepared by:Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 4 CRITERIA JUDGEMENTS RESEARCH EVIDENCE ADDITIONAL CONSIDERATIONS FEASIBILITY Is the option feasible to implement? No Probably No Uncertain Probably Yes Yes Varies X Detailed judgements No research evidence Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
  • 102.
    Explanations Prepared by:Holger, Jan, Carlos, Juan Date: December 10, 2013 Generic EtD framework 5 Recommendation Should probiotics vs. no probiotics be used in pregnant women (exposing their children in utero)? Overall balance of consequences Undesirable consequences clearly outweigh desirable consequences Undesirable consequences probably outweigh desirable consequences The balance of desirable and undesirable consequences indicates they are very similar* Desirable consequences probably outweigh undesirable consequences Desirable consequences clearly outweigh undesirable consequences ! ! ! X ! We recommend against the option or for the alternative We suggest not to use the option or to use the alternative We suggest using the option We recommend the option ! ! X ! Panel decisions 3 panel members with potential COI recused themselves from participating in formulating the recommendation. Consensus was obtained from the rest of the team. Recommendation (text) The guideline panel suggests using probiotics in pregnant women at high risk for allergy in their children (conditional recommendation, very low quality evidence). Remarks and justification Most studies commenced probiotics in the last trimester of pregnancy. The very low quality evidence for adverse effects indicates that our confidence in the absence of increased adverse effects is low. Future research is needed (see definitions of very low quality) e.g., generalizing to immune-compromised children Subgroup considerations Women with high risk of allergy in their children Women with average risk of allergy in their children Implementation considerations This recommendation is based on trials investigating the probiotics or mixtures of probiotic listed below. We have not found a difference between these different probiotics, but that does not mean there is no difference Monitoring and evaluation considerations Research priorities Develop instruments for evaluating the risk of allergy in children as the family history predicts only about 30% of the population risk. There is some evidence that first child is at higher risk for allergy than subsequent children. Long-term follow-up of long-term effects. No direct evidence for the use of probiotics in formula – this should be evaluated in future research and is an unmet need. Prepared by: Holger, Jan, Carlos, Juan Date: December 10, 2013
  • 103.
    OUTLINE • what areclinical practice guidelines? • why should we use them? • what are recommendations and levels of evidence? • how to interpret a CPG recommendation?
  • 104.
    tools / morereadings • guidelinedevelopment.org • cebgrade.mcmaster.ca • gradeing.cochrane.org GRADEing Methods
  • 105.