Elaboración de
recomendaciones en las GPC
Sistema GRADE
Nicola Magrini
NHS CeVEAS, Centre for the Evaluation of the Effect...
Contents of the presentation
• What are the defects of existing guidelines and
systems of grading
• Why GRADE could help …...
Contents of the presentation
• What are the defects of existing guidelines and
systems of grading
• Why GRADE could help …...
http://clinicalevidence.bmj.com/ceweb/about/knowledge.jsp 2011
Tendency of recent guidelines
Trends in guideline production
(AHA guidelines, Tricoci JAMA 2009)
• Recommendations are increasing in size with every upd...
Guidelines reassessment …
• … in ACC/AHA guidelines with at least 1 revision, the number of
recommendations increased 48% ...
How to improve guideline quality
Present limitations:
• Governance and composition of the guideline
committee (“what is to...
Too many grading systems?
Who is confused?
Evidence Recommendation
B Class I
C+ 1
IV C
Organization
AHA
ACCP
SIGN
Recommen...
Contents of the presentation
• What are the defects of existing guidelines and
systems of grading
• Why GRADE could help …...
Why using GRADE
GRADE is much more than a rating system
• offers a transparent and structured process for
developing and p...
WHO guideline
development processes
update 2010
1. Scoping the document: reasons for choosing the topic, problems
with existing guidelines, variations and gaps,
2. Group ...
GRADE Working Group website
and publications
www.gradeworkinggroup.org
Contents of the presentation
• What are the defects of existing guidelines and
systems of grading
• Why GRADE could help …...
GRADE: a 3 pillars approach
1. Formulate the question, choose and rate your
outcomes of interest and perform a systematic
...
GRADE: a 3 pillars approach
1. Formulate the question, choose and rate
the outcomes of interest and perform a
systematic r...
Figure 1: Hierarchy of outcomes according to
their patient-importance to assess the effect of
enteral supplement nutrition...
WHO Recommendations for the Prevention of PPH, 2007
WHO Recommendations for the Prevention of PPH, 2007
Study design is important
 Early systems of grading the quality of evidence
focused almost exclusively on study design
 ...
Quality assessment criteria:
the big start
Factors that may decrease the
quality of evidence
Study limitations (risk of bias)
 well established
– concealment
– inte...
Factors that may decrease the
quality of evidence
 Study limitations (risk of bias)
 Inconsistency among studies
 Indir...
Evidence synthesis (systematic review)
P
I
C
O
Outcome
Outcome
Outcome
Outcome
Critical
Important
Critical
Not
Summary of ...
GRADE: a 3 pillars approach
1. Formulate the question, choose and rate
your outcomes of interest and perform a
systematic ...
Determining the benefit risk profile:
positive/uncertain/unfavourable
Factors Impact on the strength of a
recommendation
B...
GRADE Step 2: risk benefit profile,
values and preferences (1/3)
GRADE Step 2: risk benefit profile,
values and preferences (2/3)
GRADE Step 2: risk benefit profile,
values and preferences (3/3)
GRADE: a 3 pillars approach
1. Formulate the question, choose and rate
your outcomes of interest and perform a
systematic ...
Strength of recommendation
The degree of confidence that the
desirable effects of adherence to a
recommendation outweigh t...
Categories of recommendations
Although the degree of confidence is a
continuum, we suggest using two
categories: strong an...
Why Grade Recommendations?
 Strong recommendations
– strong methods
– large precise effect
– few down sides of therapy
 ...
Evidence synthesis (systematic review)
Making recommendations (guidelines)
P
I
C
O
Outcome
Outcome
Outcome
Outcome
Critica...
Contents of the presentation
• What are the defects of existing guidelines and
systems of grading
• Why GRADE could help …...
Recommendations using
GRADE: Example 1
A flexible method:
quality of evidence independent
from strength of recommendation
WHO avian flu guideline 2006
Schünemann HJ et al. Lancet Infect Dis 2007;7:21-31
For opioid agonist maintenance treatment, most patients
should be advised to use methadone in adequate doses
in preference...
Recommendations using
GRADE: Example 2
Taking into account values and
preferences … and local context
Values and preferences
Stroke guideline: patients with TIA clopidogrel
over aspirin (Grade 2B).
Underlying values and pref...
Values and preferences
peripheral vascular disease: aspirin be
used instead of clopidogrel (Grade 2A).
Underlying values a...
Recommendations using
GRADE: Example 3
Weak recommendations …
a blurred vision or a clear one?
Recommendations and expected adoption rate
Strength
Definition and implications
Expected
adoption
rate
Strong
positive
The...
Contents of the presentation
• What are the defects of existing guidelines and
systems of grading
• Why GRADE could help …...
GRADE … in short
• Have an overall view of the process (see WHO), a good-
enough mandate and some governance of relevant C...
Where
am I?
You’re 30
metres
above the
ground in a
balloon
You must
be a
researcher
Yes.
How
did you
know?
Because what
yo...
Elaboración de recomendaciones en GPC. Sistema GRADE
Upcoming SlideShare
Loading in...5
×

Elaboración de recomendaciones en GPC. Sistema GRADE

901

Published on

Presentación realizada por Nicola Magrini, Director del Centro de evaluación de efectividad de cuidados en salud del Sistema Nacional de Salud de Italia, sobre el uso del Sistema GRADE para la elaboración de guías de práctica clínica. Presentación realizada en la Jornada Cienfífica de GuíaSalud 2011 "Avances en el desarrollo de Guías de Práctica Clínica".
Portal GuíaSalud http://www.guiasalud.es

Published in: Health & Medicine, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
901
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Elaboración de recomendaciones en GPC. Sistema GRADE"

  1. 1. Elaboración de recomendaciones en las GPC Sistema GRADE Nicola Magrini NHS CeVEAS, Centre for the Evaluation of the Effectiveness of Health Care, Modena, Italy WHO Collaborating Centre for Evidence Based Research Synthesis and Guideline Development
  2. 2. Contents of the presentation • What are the defects of existing guidelines and systems of grading • Why GRADE could help … • A three pillar method: the GRADE system to evaluate quality of evidence and define the strength of a recommendation • Three examples • Conclusions
  3. 3. Contents of the presentation • What are the defects of existing guidelines and systems of grading • Why GRADE could help … • A three pillar method: the GRADE system to evaluate quality of evidence and define the strength of a recommendation • Three examples • Conclusions
  4. 4. http://clinicalevidence.bmj.com/ceweb/about/knowledge.jsp 2011
  5. 5. Tendency of recent guidelines
  6. 6. Trends in guideline production (AHA guidelines, Tricoci JAMA 2009) • Recommendations are increasing in size with every update (+48% form first version) • Quality of evidence: only a minority of recommendations are based on good evidence (11%) and half (48%) on low quality evidence • Recommendations with high quality evidence are mostly concentrated in class I (strong recommendation) but only 245 of 1305 class I recommendations have high quality evidence (median, 19%)
  7. 7. Guidelines reassessment … • … in ACC/AHA guidelines with at least 1 revision, the number of recommendations increased 48% from the first guideline to the most recent version. If there is a main message in such guidelines, it is likely to be lost in the minutiae. • Within a guideline document, individual recommendations also need to be prioritized. • Finally, guidelines need flexibility. Recommendations should vary based on patient comorbidities, the health care setting, and patient values and preferences. • Physicians would be better off making clinical decisions based on valid primary data. Shaneyfelt TM, Centor RM. Reassessment of clinical practice guidelines JAMA 2009
  8. 8. How to improve guideline quality Present limitations: • Governance and composition of the guideline committee (“what is to be decided is often already decided with the selection of the deciders”) • Unanimity in guideline (not a natural component in research) • Lack of independent review (outside the accepted procedures of scientific publications) • Suboptimal management of Conflicts of interests Sniderman AD, Furberg CD. Why guidelines making requires reform JAMA 2009
  9. 9. Too many grading systems? Who is confused? Evidence Recommendation B Class I C+ 1 IV C Organization AHA ACCP SIGN Recommendation for use of oral anticoagulation in patients with atrial fibrillation and rheumatic mitral valve disease
  10. 10. Contents of the presentation • What are the defects of existing guidelines and systems of grading • Why GRADE could help … • A three pillar method: the GRADE system to evaluate quality of evidence and define the strength of a recommendation • Three examples • Conclusions
  11. 11. Why using GRADE GRADE is much more than a rating system • offers a transparent and structured process for developing and presenting summaries of quality of evidence • provides guideline developers with a comprehensive and transparent framework for carrying out the steps involved in developing recommendations • specifies an approach to framing questions, choosing outcomes of interest and rating their importance, evaluating the evidence, and incorporating evidence with considerations of values and preferences of patients and society to arrive at recommendations
  12. 12. WHO guideline development processes update 2010
  13. 13. 1. Scoping the document: reasons for choosing the topic, problems with existing guidelines, variations and gaps, 2. Group composition (or consultations) 3. Conflict of interest 4. Formulations of the questions and choice of the relevant outcomes 5. Evidence retrieval, evaluation and synthesis (balance sheet, evidence table) 6. Benefit/risk profile: integrating evidence with values and preferences, equity and costs 7. Formulation of the recommendations 8. Implementation and evaluation of impact 9. Research needs or areas of further research 10. Peer-review process and updating Title, responsible person, WHO Department - responsible of the clearance process, WHO Departments involved, CC involved, Standards for evidence: GRADE system Reporting standard and process Reporting standard and process
  14. 14. GRADE Working Group website and publications www.gradeworkinggroup.org
  15. 15. Contents of the presentation • What are the defects of existing guidelines and systems of grading • Why GRADE could help … • A three pillar method: the GRADE system to evaluate quality of evidence and define the strength of a recommendation • Three examples • Conclusions
  16. 16. GRADE: a 3 pillars approach 1. Formulate the question, choose and rate your outcomes of interest and perform a systematic review (quality of evidence) 2. Risk benefit evaluation, consider patients values and preferences and also resource use and feasibility 3. Direction (positive/negative) and strength (strong/weak) of the recommendation
  17. 17. GRADE: a 3 pillars approach 1. Formulate the question, choose and rate the outcomes of interest and perform a systematic review (quality of evidence) 2. Risk benefit evaluation, consider patients values and preferences and also resource use and feasibility 3. Strength of the recommendation
  18. 18. Figure 1: Hierarchy of outcomes according to their patient-importance to assess the effect of enteral supplement nutrition for geriatric patients with bed sores Nutritional status 4 Importance of endpoints Microcirculation of the wound 1 2 Energy supply 3 5 Function 6 Quality of life 7 Healing of the 8 bedsore Mortality 9 Critical for decision making Important, but not critical for decision making Not patient-important Rating of outcomes … example: patient with bed sores
  19. 19. WHO Recommendations for the Prevention of PPH, 2007
  20. 20. WHO Recommendations for the Prevention of PPH, 2007
  21. 21. Study design is important  Early systems of grading the quality of evidence focused almost exclusively on study design  Randomised trials provide, in general, stronger evidence than observational studies: –RCTs start at High Quality –Observational studies start at Low Quality  However, other factors may decrease or increase the quality of evidence
  22. 22. Quality assessment criteria: the big start
  23. 23. Factors that may decrease the quality of evidence Study limitations (risk of bias)  well established – concealment – intention to treat principle observed – blinding – completeness of follow-up – Choice of comparator (standard/optimal treatment)  more recent – early stopping for benefit – selective outcome reporting bias
  24. 24. Factors that may decrease the quality of evidence  Study limitations (risk of bias)  Inconsistency among studies  Indirectness of evidence  Imprecise results  Reporting bias
  25. 25. Evidence synthesis (systematic review) P I C O Outcome Outcome Outcome Outcome Critical Important Critical Not Summary of findings & estimate of effect for each outcome Rate overall quality of evidence across outcomes based on lowest quality of critical outcomes RCT start high, obs. data start low 1. Risk of bias 2. Inconsistency 3. Indirectness 4. Imprecision 5. Publication bias GradedownGradeup 1. Large effect 2. Dose response 3. Confounders Very low Low Moderate High
  26. 26. GRADE: a 3 pillars approach 1. Formulate the question, choose and rate your outcomes of interest and perform a systematic review (quality of evidence) 2. Risk benefit evaluation, consider patients values and preferences and also resource use and feasibility 3. Strength of the recommendation
  27. 27. Determining the benefit risk profile: positive/uncertain/unfavourable Factors Impact on the strength of a recommendation Balance between desirable and undesirable effects Larger the difference between the desirable and undesirable effects, more likely a favourable benefit But differences can arise depending on the severuty of adverse events Values and preferences More variability in values and preferences, or more uncertainty in values and preferences, more likely an unfavourable profile. Costs (resource use) Higher the costs of an intervention – that is, the more resources consumed – less likely a favourable profile.
  28. 28. GRADE Step 2: risk benefit profile, values and preferences (1/3)
  29. 29. GRADE Step 2: risk benefit profile, values and preferences (2/3)
  30. 30. GRADE Step 2: risk benefit profile, values and preferences (3/3)
  31. 31. GRADE: a 3 pillars approach 1. Formulate the question, choose and rate your outcomes of interest and perform a systematic review (quality of evidence) 2. Risk benefit evaluation, consider patients values and preferences and also resource use and feasibility 3. Strength of the recommendation
  32. 32. Strength of recommendation The degree of confidence that the desirable effects of adherence to a recommendation outweigh the undesirable effects. Desirable effects •health benefits •less burden •savings Undesirable effects •harms •more burden •costs
  33. 33. Categories of recommendations Although the degree of confidence is a continuum, we suggest using two categories: strong and weak.  Strong recommendation: the panel is confident that the desirable effects of adherence to a recommendation outweigh the undesirable effects.  Weak recommendation: the panel concludes that the desirable effects of adherence to a recommendation probably outweigh the undesirable effects, but is not confident. Recommend   Suggest  
  34. 34. Why Grade Recommendations?  Strong recommendations – strong methods – large precise effect – few down sides of therapy  Weak recommendations – weak methods – imprecise estimate – small effect – substantial down sides
  35. 35. Evidence synthesis (systematic review) Making recommendations (guidelines) P I C O Outcome Outcome Outcome Outcome Critical Important Critical Not Summary of findings & estimate of effect for each outcome Rate overall quality of evidence across outcomes based on lowest quality of critical outcomes RCT start high, obs. data start low 1. Risk of bias 2. Inconsistency 3. Indirectness 4. Imprecision 5. Publication bias GradedownGradeup 1. Large effect 2. Dose response 3. Confounders Very low Low Moderate High Formulate recommendations: •For or against (direction) •Strong or weak (strength) By considering: Quality of evidence Balance benefits/harms Values and preferences Revise if necessary by considering: Resource use (cost) • “We recommend using…” • “We suggest using…” • “We recommend against using…” • “We suggest against using…”
  36. 36. Contents of the presentation • What are the defects of existing guidelines and systems of grading • Why GRADE could help … • A three pillar method: the GRADE system to evaluate quality of evidence and define the strength of a recommendation • Three examples • Conclusions
  37. 37. Recommendations using GRADE: Example 1 A flexible method: quality of evidence independent from strength of recommendation
  38. 38. WHO avian flu guideline 2006 Schünemann HJ et al. Lancet Infect Dis 2007;7:21-31
  39. 39. For opioid agonist maintenance treatment, most patients should be advised to use methadone in adequate doses in preference to buprenorphine. – Strength of recommendation – Strong – Quality of evidence – High WHO Guidelines for the Psychosocially Assisted Pharmacological Treatment of Opioid Dependence (2009) On average, methadone maintenance doses should be in the range of 60–120 mg per day. – Strength of recommendation – Strong – Quality of evidence – Low
  40. 40. Recommendations using GRADE: Example 2 Taking into account values and preferences … and local context
  41. 41. Values and preferences Stroke guideline: patients with TIA clopidogrel over aspirin (Grade 2B). Underlying values and preferences: This recommendation to use clopidogrel over aspirin places a relatively high value on a small absolute risk reduction in stroke rates, and a relatively low value on minimizing drug expenditures.
  42. 42. Values and preferences peripheral vascular disease: aspirin be used instead of clopidogrel (Grade 2A). Underlying values and preferences: This recommendation places a relatively high value on avoiding large expenditures to achieve small reductions in vascular events.
  43. 43. Recommendations using GRADE: Example 3 Weak recommendations … a blurred vision or a clear one?
  44. 44. Recommendations and expected adoption rate Strength Definition and implications Expected adoption rate Strong positive The drugs/interventions should offered to the vast majority of patients and could be used as an indicator of good quality of care > 60-70% Weak positive It has the wider range of uncertainty since it could mean only for a minority of patients (30%) or for a good proportion of them (50- 60%). It is necessary to inform patients of the expected benefits and risks (and their magnitude), explore patients values and discuss potential alternative treatments 30-60% Weak negative In selected cases or a defined minority. The decision should go along with a detailed information to patient of the benefit risk profile (magnitude), patients values and expectations and the presentation of potential alternative treatments 5-30% Strong negative It should not be used neither routinely nor for a subgroup. Only in few very selected (and documented) cases can be used since the benefit/risk balance is negative/unknown and available alternative are preferable < 5%
  45. 45. Contents of the presentation • What are the defects of existing guidelines and systems of grading • Why GRADE could help … • A three pillar method: the GRADE system to evaluate quality of evidence and define the strength of a recommendation • Three examples • Conclusions
  46. 46. GRADE … in short • Have an overall view of the process (see WHO), a good- enough mandate and some governance of relevant CoI • Make just a few (a reasonable number of) recommendations • Use systematic reviews (if not available, review key, accessible evidence) – DO NOT meta-analyse if not done • Use GRADE criteria for quality of evidence • Explain the reasons supporting the strength of recommendations, including the benefit/risk profile and values and preferences • … just be (more) transparent
  47. 47. Where am I? You’re 30 metres above the ground in a balloon You must be a researcher Yes. How did you know? Because what you told me is absolutely correct but completely useless You must be a policy maker Yes, how did you know? Because you don’t know where you are, you don’t know where you’re going, and now you’re blaming me from: Jonathan Lomas, 2008
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×