The document discusses key aspects of systematic reviews that should be addressed to minimize bias, including clearly specifying inclusion/exclusion criteria, conducting comprehensive searches to identify most relevant studies, and accounting for quality of reviewed studies. It emphasizes the importance of transparently reporting how trials were selected and quality assessed to strengthen the evidence provided by the systematic review.

1. Systematic reviews of randomized
trials
Dr. Maria Idrees: PT

2. 1. Was it clear which trials were to be
reviewed?
2. Were most relevant studies reviewed?
3. Was the quality of the reviewed
studies taken into account?

3. Was it clear which trials were to be
reviewed?
• When we read systematic reviews we need to
be satisfied that the reviewer has not
selectively reviewed those trials that support
his or her own point of view.
• One of the strengths of properly conducted
systematic reviews is that the possibility of
selective reviewing is reduced.

4. • To reduce the possibility of selective reviewing,
reviewers should clearly define the scope of the
review prior to undertaking a search for relevant
trials.
• The best way to do this is to describe clearly the
criteria that are used to decide what sorts of trial
will be included in the review, and perhaps also
which trials will not.
• Systematic reviews that specify clear inclusion
and exclusion criteria provide stronger evidence
of effects of therapy than those that do not.

5. Were most relevant studies
reviewed?
• There are two reasons why it is important that
reviews identify most relevant trials.
1) First, if the review does not identify all relevant
trials it may conclude that there is less evidence
than there really is.
2) More seriously, when not all relevant trials are
found there is the possibility that those trials
that were not found had systematically different
conclusions from those included in the review.

6. • It has been shown that unpublished studies
and studies published in languages other than
English tend to have more negative estimates
of the effects of interventions than trials
published in English
• Hence systematic reviews that search only for
published trials are said to be exposed to
‘publication bias’, and systematic reviews that
search only for trials reported in English are
said to be exposed to ‘language bias’.

7. What constitutes an adequate search?
• It is clearly insufficient to search only
MEDLINE.
• It is desirable that the reviewers perform
sensitive searches of several medical literature
databases (say, at least two of MEDLINE,
Embase, CINAHL and PsychINFO) and at least
one of the specialist databases such as the
Cochrane Collaboration’s Central Register of
Clinical Trials (CENTRAL) or PEDro.

8. • A further consideration is the recency of the
review.
• Systematic reviews tend to date rather quickly
because, in most fields of physiotherapy, new
trials are being published all the time
• A systematic review that involved a
comprehensive search but that was published
5 years ago is unlikely to provide a
comprehensive overview of the findings of all
relevant trials.

9. • In fact, there is often a lag of several years
between when a search was conducted and
the review was eventually published, so the
search may be considerably older than the
year of publication of the review suggests.
• The year in which the search was conducted is
usually given in the Methods section of the
review.

10. Was the quality of the reviewed studies taken
into account?
• Many randomized trials are poorly designed
and provide potentially seriously biased
estimates of the effects of intervention.
• The simplest way to incorporate quality
assessments into the findings of a systematic
review is to list minimum quality criteria for
trials that are to be considered in a review.

11. • Most (but not all) reviews specify that trials must
be randomized.
• The consequence is that non-randomized trials
are effectively ignored.
• Excluding non-randomized trials protects against
the allocation bias that potentially distorts
findings of non-randomized trials.
• However, as we have seen, randomization alone
does not guarantee protection from bias.
• Even randomized trials are exposed to other
sources of bias, so it is not sufficient to require
only that trials be randomized; it is necessary to
apply additional quality criteria

12. • Some systematic reviewers stipulate that a
trial must also be participant- and assessor-
blinded if it is to be considered in the review.
• An alternative way to take into account trial
quality in a review is to assess the quality of
the trial using a checklist or scale.
• Jadad scale (Checklist)
• Maastricht scale (Scale)
• PEDro scale (Scale)
• Quality score can be used to set a quality
threshold

13. • There is the risk that quality thresholds are
too low (biased trials are still given too much
weight) or too high (important trials are
ignored).