Defining the Inclusion and Exclusion Criteria for Systematic Reviews

1. Uzairue Leonard Ighodalo
Federal University Oye-Ekiti-Centre for Impact AMR and Policy Research (CIAMPPR)
Founder-Research Hub Nigeria and Frontier Lead Consulting
SYSTEMATIC REVIEW ELIGIBILITY
CRITERIA

2. DISCLOSURE
Some Information on this presentation is part of the trainings I received from other
Institutions.

3. SYSTEMATIC REVIEW PROCESS
OVERVIEW

4. LEARNING OBJECTIVES
To understand the role of study eligibility criteria (inclusion/exclusion) in framing a
systematic review
To know when and how to set study eligibility criteria
To understand the effect of study eligibility criteria on interpretation of a review

5. STUDY ELIGIBILITY CRITERIA
Eligibility criteria
▪Define which studies or articles are included in a systematic review.
▪Ensure the review's relevance, reliability, and validity.
▪Minimize bias and increase transparency.
▪Function the same in systematic reviews as in primary research
▪Reflect the analytic framework and key questions
▪Are powerful tools for widening or narrowing the scope of a review
▪Provide information to determine whether reviews can be compared or combined

6. SOME SAMPLE CRITERIA
▪Population type (adult, community-dwelling females)
▪Intervention (screening tool to identify depression risk)
▪Country (Nigeria only)
▪Setting (hospital-based studies or Community based
studies)
▪Study population size (N > 200)
▪Study design (randomized controlled trials)

7. USING BROAD CRITERIA
▪Can be as broad as “all related studies”
▪Can be helpful for exploring “what is known”
▪May result in too much literature to feasibly review
▪Could pull in disparate literature that cannot be
compared

8. USING NARROW CRITERIA
▪Can be helpful in culling homogenous literature
▪Can reduce size of the literature to a manageable scope
▪Could be appropriate for a narrow research question
▪May reduce applicability of the review and sometimes
risks introducing bias

9. REFINING CRITERIA
▪Choosing how broad or how narrow to set criteria requires a
balance of obtaining adequate information to answer a given
question without obscuring the results with irrelevant literature.
▪Review teams should work together to find this balance.
▪The overarching goal is to minimize bias related to which studies
are selected.

10. BIAS IN THIS CONTEXT
▪Distortion of the estimate of effect that comes from
how studies are selected for inclusion
▪Affects the applicability, or “external validity,” of
the review itself

11. EXAMPLES OF BIAS IN THIS
CONTEXT
Inappropriate eligibilty criteria may limit applicability of the review.
Example: the use of studies of twin pregnancies in a review of preterm labor management for low-risk
women
Eligibility criteria may result in the inclusion of more of a certain study type that either
overestimates or underestimates effectiveness.

12. SELECTING CRITERIA
▪Review study goals
▪Assess analytic framework and key questions
▪Set criteria before beginning abstract review

13. TIE CRITERIA TO PICOTS
Population — condition, disease severity and stage, comorbidities,
patient demographics
Intervention — dosage, frequency, method of administration
Comparator — placebo, usual care, or active control
Outcome — health outcomes, morbidity, mortality, quality of life
Timing — Duration of followup
Setting — Primary, specialty, inpatient, cointerventions

14. CASE EXAMPLES
▪Population
Research Question: Does vitamin C supplementation reduce the
incidence of common cold in adults?
Eligibility Criteria for Population: Adults (age 18-65)
▪Intervention
Research Question: Does a specific diet plan improve weight loss in
obese individuals?
Eligibility Criteria for Intervention: Specific diet plan (e.g.,
Mediterranean diet)

15. ▪Comparison
Research Question: Does Drug A outperform Drug B in reducing
blood pressure?
Eligibility Criteria for Comparison: Drug B as the comparator
▪Outcomes
Research Question: Does exercise enhance quality of life in cancer
survivors?
Eligibility Criteria for Outcomes: Primary outcome - quality of life

16. ▪Study Design
Research Question: Is acupuncture effective in managing chronic
pain?
Eligibility Criteria for Study Design: RCTs only
▪Time Frame
Research Question: What is the trend in air pollution levels over the
past decade?
Eligibility Criteria for Time Frame: Studies published between
2010-2020

17. OTHER CONSIDERATIONS FOR
SETTING CRITERIA
▪What study designs should be included?
Include studies in foreign languages?
Include studies conducted in other countries but published in
English?
Include “grey” or “fugitive” literature?
Include year of publication?

18. TYPES OF STUDIES TO BE INCLUDED
▪Limit to randomized controlled trials?
▪Include observational studies?
If so, what specific types?
▪What is the value of a case series?
▪How is the definition of case series operationalized?

19. OBSERVATIONAL STUDIES (I)
There are four main types of observational studies:
Cohorts (with comparisons)
Case controls
Case series
Registries/databases

20. OBSERVATIONAL STUDIES (II)
Well-conducted observational studies can address issues of
applicability and the need for longer-term outcomes if they:
include more representative patient populations,
have relevant comparators, and
report more meaningful clinical outcomes over longer time frames.
Observational studies may be a better source of information about
harms.

21. NON-ENGLISH–LANGUAGE STUDY
REPORTS
Positive findings may be more likely to be published in high-profile
English-language journals.
Therefore, to include only English-language journals may
overestimate the positive effect of an intervention.
Empirically, the bias associated with limiting a review to
English-language reports has been shown to be small.
Gregoire G, et al. J Clin Epidemiol 1995;48:159-63; Moher D, et al. J Clin Epidemiol 2000;53:964-72.

22. GREY OR “FUGITIVE “LITERATURE
Material that may be difficult to identify and retrieve:
Government reports and regulatory sites (Center for Drug Evaluation and Research/U.S. Food and Drug
Administration)
Pharmacoepidemiologic databases, including postmarketing surveillance
Book chapters
Conference proceedings
Published dissertations
Scientific information packets (Agency for Healthcare Research and Quality)
Rationale for a search:
Because journals may publish positive or statistically significant results, finding grey literature of unpublished
nonsignificant or null results may indicate the presence of publication bias.

23. YEAR OF PUBLICATION
▪Literature searches typically have date parameters (i.e.,
earliest or latest date of publication).
▪Date parameters are particularly important when there
has been a change in policy, practice, or formulation that
makes older studies less applicable.

24. EXERCISE 1
What would you do if you were asked to review the literature on
transition support for adolescents with autism spectrum disorder who
are entering adulthood?
■ Before seeing the key questions, consider the categories of criteria
that you will want to apply.

25. EXERCISE 1: BASIC PICOTS
QUESTIONS
▪What is the relevant population?
▪What is the intervention of interest?
▪To what exposure is the intervention being compared?
▪What outcomes are relevant?
▪What short-term or long-term outcomes should be considered?
▪In what setting would the results be applicable?
PICOTS = population, intervention, comparators, outcomes, timing, and setting

26. EXERCISE 1: PICOTS
Population
What constitutes an adolescent?
What constitutes a diagnosis of autism spectrum disorder (ASD)?
Intervention How is transition support defined?
Comparator
Do we compare to no transition support or directly compare types
of support?
Outcome
What are the goals for adolescents with ASD as they transition to
adulthood?
Timing How quickly should the outcomes be apparent?
Setting
Is transition support provided in multiple settings, such as
schools, clinics, and the community?

27. EXERCISE 1: WHAT WOULD YOU
DO WITH . . .
A study that included “individuals over 10 years of age”?
A paper about an intervention for individuals with a range of
developmental disabilities, not only autism spectrum disorder
(ASD)?
A study of children with Asperger’s syndrome but not other forms of
ASD?

28. EXERCISE 1: NO “RIGHT” ANSWERS
There are no “right” answers to these questions.
The team should consider how selection criteria may bias the review
or affect applicability.
The team should clearly define how eligibility criteria are
operationalized and record these decisions throughout the review.

29. REFINING ELIGIBILITY CRITERIA
▪Review protocols should specify eligibility criteria in detail.
▪Criteria may need to be adapted during the review
process based on emerging evidence.

30. CONCLUSION
▪Eligibility criteria are the foundation of systematic
reviews.
▪They ensure research relevance and minimize bias.
▪Real-world case examples demonstrate their
application.

31. Thank you for your attention.
CONTACT
Email: uzairue.leonard@gmail.com or leonard.uzairue@fuoye.edu.ng
LinkedIn: Uzairue Leonard
Twitter: @Uzairue1
Research gate: Uzairue Leonard