non rct, quasi trial , time series design, uncontrolled trials, before and after trials

2. NON-RANDOMIZED
CONTROL TRIALS
P R E S E N T E D BY
- D R K A R I S H M A S H A L A G E R I

3. Contents
Introduction
Reasons For The Use Of Nonrandomized Studies
Examples Of Nonrandomized Studies
Types Of Non Randomized Trials
Quasi-experimental Designs
Threats To Establishing Causality
When Using Quasi-experimental Designs
Threats To Internal Validity
Sources Of Bias In Non Randomized Trials
Case Mix Adjustment Methods
Implications For Using Non Randomized Trials
Conclusion
References

4. INTRODUCTION
• The ultimate goal of the evaluation of healthcare interventions is to
produce a valid estimate of effectiveness, in terms of both internal and
external validity.
• Internal validity concerns the extent to which the results of a study can be
reliably attributed to the intervention under evaluation
• Whereas external validity concerns the extent to which a study’s results
can be generalized beyond the given study context.

5. • Although experimental method is almost always to be preferred, it is not always
possible for ethical, administrative and other reasons to resort to randomized control
trial in human beings.
• Secondly, some preventive measures can be applied only to groups or on community
wide basis .
• Thirdly when disease frequency is low and natural history long, RCT require follow up
of thousands of people for a decade or more.
• The cost and logistics are often prohibitive.
• In such situations we must depend on other study designs such as Non-Randomized
trials
Ex: community trials on water fluoridation.
Ex: cancer cervix
INTRODUCTION

6. Validity of causal inference remains largely a
matter of extra statistical judgement.
Nevertheless, Vital decisions
affecting public health and
preventive medicine have been
made by non-experimental studies .
As there is no randomization in non-experimental trials, the degree of comparability will be
low and chances of spurious result higher than where randomization had taken place.

7. REASONS FOR THE USE OF
NONRANDOMIZED STUDIES
It tends to balance subject characteristics between the groups and facilitate causal inference.
It eliminates selection effects
It provides a basis for statistical inference

8. 1. NONRANDOMIZED STUDIES ARE SOMETIMES THE
ONLY ETHICAL WAY TO CONDUCT AN INVESTIGATION
• If the treatment is potentially harmful, it is generally unethical for an
investigator to assign people to this treatment.
An example of this is,
1. A study of the effects of malnutrition, where we simply cannot assign subjects to
intolerable
diets. Thus we compare malnourished populations with those on adequate diets.
2. A study of the effects of carbonated drinks on tooth erosion, where we cannot assign
subjects to such habits. Thus we compare population with regular consumption of
carbonated drinks and population who don’t consume such drinks.

9. 2. NONRANDOMIZED STUDIES ARE
SOMETIMES THE ONLY ONES POSSIBLE.
• Certain investigations require the implementation of treatments that
may affect people's lives. In a democratic society randomized
implementation of such treatments is not always feasible.
Example: The question of fluoridating a town's water supply.
We would have a series of towns, some of which have elected fluoridation and others
which have not. The dental experience of the children in these towns can provide a great
deal of useful information if properly analysed.

10. 3. NONRANDOMIZED STUDIES ARE
USUALLY LESS EXPENSIVE.
• An advantage of nonrandomized studies is that they usually cost less
per subject and may not require the extensive planning and control that
are needed for randomized studies.
• This makes nonrandomized studies particularly attractive in the early
stages of any research effort.

11. 4. NONRANDOMIZED STUDIES MAY BE
CLOSER TO REAL-LIFE SITUATIONS.
• To the extent that randomization differs from natural selection mechanisms, the
conditions of a randomized study might be quite different from those in which the
treatment would ordinarily be applied.
Example:
A program may be very successful for those who choose it themselves on the
basis of a media publicity campaign but ineffective when administered as a
social experiment.

12. EXAMPLES OF NON-RANDOMIZED TRIALS
Uncontrolled trials
Natural experiments
Before and after
comparison studies

13. 1. UNCONTROLLED TRIALS
• These are trials with no comparison group.
• Initially uncontrolled trials may be useful in evaluating
whether a specific therapy appears to have any value in particular disease
 to determine an appropriate dose
To investigate adverse reactions
Even in these uncontrolled trials, one is using implied “historical controls”,
i.e., the experience of earlier untreated patients affected by the same disease.

14. 1. UNCONTROLLED TRIALS
It is becoming increasingly common to employ the procedures of a double-blind
controlled clinical trial in which the effect of new drug are compared to some
concurrent experience.
(either placebo or currently utilized therapy)
 Uncontrolled trials may be useful in evaluating whether a specific therapy appears
to have any value in a particular disease, to determine an appropriate dose, to
investigate adverse reactions, etc.

15. 2. NATURAL EXPERIMENTS
• Where experimental studies are not possible in human populations, the epidemiologist
seeks to identify “natural circumstances” that mimic an experiment.
For example: in respect to cigarette smoking
People have separated themselves “naturally” into 2 groups, smokers an
non-smokers.
Other population involved in natural experiments comprise the following groups:
a) Migrants b) religious or social groups c) famines d) Earthquakes

16. 2. NATURAL EXPERIMENTS
• A major earthquakes in Athens in1981 provided a natural experiments to
epidemiologists who studied the effects of acute stress on cardiovascular mortality.
They showed an excess of deaths from cardiac and external causes on the days after
the major earthquake, but no excess deaths from other causes.
John Snows discovery that cholera is a water borne disease was the outcome of a natural
experiment.

17. EXAMPLE

18. QUASI-EXPERIMENTAL DESIGNS
• Quasi-experimental studies encompass a broad range of
nonrandomized intervention studies.
• These designs are frequently used when it is not logistically feasible or
ethical to conduct a randomized controlled trial.
• These studies aim to evaluate interventions but that do not use
randomization.
• Similar to randomized trials, quasi-experiments aim to demonstrate
causality between an intervention and an outcome.
• Quasi-experimental studies can use both pre intervention and post
intervention measurements as well as non randomly selected control
groups.

19. Researchers often choose not to randomize the intervention for one or more
reasons:
(1) Ethical Considerations
(2) Difficulty Of Randomizing Subjects
(3) Difficulty To Randomize By Locations (E.G., By Wards)
(4) Small Available Sample Size
QUASI-EXPERIMENTAL DESIGNS

20. WHEN IS IT APPROPRIATE TO USE QUASI-
EXPERIMENTAL METHODS?
• Quasi-experimental methods can be used,
i.e., after the intervention has taken place (at time t+1).
• In some cases, especially for interventions that are spread over a longer duration,
preliminary impact estimates may be made at mid-term (time t).
• It is always highly recommended that evaluation planning begins in advance of an
intervention, however. This is especially important as baseline data should be collected
before the intended recipients are exposed to the programme /policy activities (time t-
1).

21. DIFFERENT QUASI-EXPERIMENTAL
STUDY DESIGNS
(i) Before And After Studies without Control;
(Ii) Time Series Designs; And
(Iii) Before And After Studies with Control.

22. 3.BEFORE AND AFTER COMPARISON
STUDIES
Before and after comparison
studies without control
Before and after comparison
studies with control

23. BEFORE AND AFTER COMPARISON
STUDIES WITHOUT CONTROL .
• These studies centre round comparing the incidence of disease before and after
introduction of preventive measure.
• The experiment serves as its own control; this eliminates virtually all group differences .
The events which took place prior to the use of the new treatment or preventive procedure
are used as a standard for comparison.

24. Classic examples of “before and after comparison studies” were
The prevention of scurvy among sailors
James Lind in 1750 by providing fresh
Studies on the transmission of cholera by
John Snow in 1854
Prevention of polio by Salk and Sabin
vaccines.

25. This table gives an example of a "before and after comparison
study" in Victoria (Australia) following introduction of seat-belt
legislation for prevention of deaths and injuries caused by motor
vehicle accidents.

26. BEFORE AND AFTER STUDIES WITHOUT
CONTROL
• The intervention is confounded by the Hawthorne effect (the non-specific beneficial
effect on performance of taking part in research) which could lead to an overestimate
of the effectiveness of an intervention.
• In general, before and after studies without control should not be used to evaluate the
effects of guideline implementation strategies, and the results of studies using such
designs have to be interpreted with great caution.

27. IN ORDER TO ESTABLISH EVIDENCE IN BEFORE
AND AFTER COMPARISON STUDIES , WE NEED:
Data –regarding incidence of disease, before and after introduction of preventive measure must be
available.
Introduction or manipulation of only one factor or change relevant to the situation, other factors
remaining the same.
Ex; addition of fluoride to drinking water to prevent dental caries.
Diagnostic criteria of the disease should remain the same.
Adoption of preventive measures should be over a wide area
Reduction in the incidence must be large following the introduction of the preventive measure,
because there is no control .
Several trials may be needed before the evaluation is considered conclusive

28. TIME SERIES DESIGNS
• Time series designs attempt to detect whether an intervention has had an effect
significantly greater than the underlying trend.
• They are useful in guideline implementation research for evaluating the effects of
interventions when it is difficult to randomize or identify an appropriate control group.

29. • Data are collected at multiple time points before and after the intervention; the
multiple time points before the intervention allow the underlying trend to be
estimated, the multiple time points after the intervention allow the intervention effect
to be estimated accounting for the underlying trend.
TIME SERIES DESIGNS

30. • Time series designs increase the confidence with which the estimate of effect can be
attributed to the intervention, although the design does not provide protection against
the effects of other events occurring at the same time as the study intervention, which
might also improve performance.
• Furthermore, it is often difficult to collect sufficient data points unless routine data
sources are available.
• Currently, many published interrupted time series have been analysed inappropriately,
frequently overestimating the effect of the intervention.
TIME SERIES DESIGNS

31. SINGLE-GROUP INTERRUPTED TIME-
SERIES DESIGN
• In this design, the researcher records measure for a single group both before and after
a treatment.
• Group A O------O------O------O------- X ------O-----O-----O------O

32. CONTROL-GROUP INTERRUPTED TIME-
SERIES DESIGN
• This is a modification of Single-Group Interrupted Time-Series Design in which two
groups of participants, not randomly assigned, are observed over time. A treatment is
administered to one of the group (i.e. group A)
Group A O------O------O------O------- X ------O-----O-----O------O
Group B O------O------O------O------- O------O-----O-----O------O

33. C. BEFORE AND AFTER COMPARISON
STUDIES WITH CONTROL
• In the absence of control group, comparison between observations before and after
the use of a new treatment or procedure may be misleading.
In these situation, the epidemiologist tries to utilize a “natural” control group i.e., the one
provided by natural or natural circumstances.
If preventive programme is to be applied to an entire community, we would select another
community as similar as possible, particularly with respect to frequency and characteristics of the
disease to be prevented.

34. In this example, the existence of a control with which the results in
victoria could be compared strengthens the conclusion that there was
definite fall in the number of deaths and injuries in occupants of cars
after the introduction of compulsory seat-belt legislation.

35. • Data are collected in both populations contemporaneously using similar methods
before and after the intervention is introduced in the study population.
• A ‘between group’ analysis comparing performance in the study and control groups
following the intervention is undertaken, and any observed differences are assumed to
be due to intervention.

36. NON-EQUIVALENT (PRETEST AND POST-
TEST) CONTROL-GROUP DESIGN
• In this design, the experimental Group A and the control Group B are selected with
random assignment. Both groups take a pre-test and post-test. But only the
experimental group receives the treatment.
Group A O------- X ------O
Group B O----------------O

37. THREATS TO ESTABLISHING CAUSALITY
WHEN USING QUASI-EXPERIMENTAL
DESIGNS
• The lack of random assignment is the major weakness of the quasi-experimental study
design.

38. THREATS TO INTERNAL VALIDITY
Ambiguous temporal
precedence
Lack of clarity about whether intervention occurred before
outcome
Selection Systematic differences over conditions in respondent
characteristics that could also cause the observed effect
History Events occurring concurrently with intervention could
cause the observed effect
Maturation Naturally occurring changes over time could be
confused with a treatment effect
Regression When units are selected for their extreme scores,
they will often have less extreme subsequent scores, an
occurrence that can be confused with an intervention
effect
Attrition Loss of respondents can produce artifactual effects
if that loss is correlated with intervention
Testing Exposure to a test can affect scores on subsequent
exposures to that test
Instrumentation The nature of a measurement may change

39. SOURCES OF BIAS IN NONRANDOMIZED
STUDIES
• Four main sources of systematic bias in trials of the effects of healthcare as being:
Selection Bias
Performance Bias- if there are errors and inconsistencies in the allocation,
application and recording of interventions
Attrition Bias - will occur if there are dropouts,
Detection Bias - if the assessment of outcomes is not standardized and blinded
All of these biases can also occur in RCTs, but there is perhaps potential for their impact to be
greater in non-randomized studies which are usually undertaken without protocols specifying
standardised interventions, outcome assessments and data recording procedures

40. SELECTION BIAS
• Randomized and Non-Randomized studies is, the risk of selection bias, where
systematic differences in comparison groups arise at baseline.
It is sometimes referred to as case-mix bias, or
confounding.
The term selection bias can be misleading as it is used to describe both
1. Biased selection of participants for inclusion in a study (which applies to both
experimental and observational studies) - classified as an issue of external validity
2. Biased allocation of patients to a given intervention (which occurs where
randomization is not used) - is an issue of internal validity.

41. WHEN SELECTION BIAS WILL BE
INTRODUCED IN NON RANDOMIZED
CONTROL TRIALS ..
• when participants chosen for one intervention have different characteristics from those
allocated to the alternative intervention (or not treated).
• The choice of an intervention under these circumstances will be influenced not only by a
clinician’s own personal preference for one intervention over another but also by patient
preference, patient characteristics and clinical history.

42. • Protopathic bias is a term coined by Horwitz and Feinstein15 to describe
situations where the first symptoms of a given outcome are the reason for treatment
initiation: “Protopathic bias” occurs “when a pharmaceutical or other therapeutic
is inadvertently prescribed for an early manifestation of a disease that has not yet
been diagnostically detected” (our emphasis).
• For example, a drug given for abdominal pain may be wrongly associated with
injury, as abdominal pain may be one of the prodromal symptoms.
• A drug given for persistent mouth ulcer may be wrongly associated with oral cancer,
persistent mouth ulcer may be one of the prodromal symptoms.

43. CASE-MIX ADJUSTMENT METHODS
• In the absence of information on factors influencing allocation, the traditional solution
to removing selection bias in non-randomized studies has been to attempt to control
for known prognostic factors, either by design and/or by analysis.
STANDARDISATION
Participants are analysed in groups (strata)
which have
similar characteristics, the overall effect being
estimated
by averaging the effects seen in each of the
groups

44. REGRESSION
Relationships between prognostic factors and outcome
are estimated from the data in hand, and adjustments
calculated for the difference in average values of the
prognostic factor between the two groups. Linear
regression (or covariance analysis) is used for continuous
outcomes, logistic regression for binary outcomes.
Propensity scores
Propensity probabilities are calculated for each participant
from the data set, estimating their chance of receiving
treatment according to their characteristics. Treatment
effects are estimated either by comparing groups that
have similar propensity scores (using matching or
stratification methods), or by calculating a regression
adjustment based on the difference in average propensity

45. IMPLICATIONS FOR THOSE PRODUCING, REVIEWING
AND USING NONRANDOMIZED STUDIES
• An investigator planning to undertake a nonrandomized study should first make
certain that an RCT cannot be undertaken.
• The ability to eradicate bias at the design stage is crucial to establishing the validity of
a study. In particular, investigators should not assume that statistical methods can be
used reliably to compensate for biases introduced through suboptimal allocation
methods.
• A prospective non-randomized study should be undertaken according to a protocol
that is carefully followed to ensure consistent inclusion criteria, that all relevant factors
are measured accurately for each participant and that participants are all monitored in
a standard manner and blinded to treatment if possible.

46. • In some situations it may even be possible to match prospectively treated and control
patients on important prognostic factors
• Healthcare decision-makers should be cautious not to over-interpret results from non-
randomized studies.
• Importantly, checking that treated and control groups appear comparable does not
guarantee freedom from bias, and it should never be assumed that case-mix
adjustment methods can fully correct for observed differences between groups.

47. CONCLUSION
• Non-randomized studies are sometimes but not always biased,
The results of non-randomized studies can differ from the results of RCTs of the same
intervention.
• Statistical methods of analysis cannot properly correct for inadequacies of study
design.
• Systematic reviews of effectiveness often do not adequately assess the quality of non-
randomized studies.
• Non-randomized studies provide a poor basis for treatment or health policy decisions.

48. REFERENCES
• K Park. Park’s textbook of preventive and social medicine.2019;25th ed:61-78
• Gordis L. Text book of Epidemiology. 5th ed. Elsevier
• Roger Detels et al. Oxford Text Book of Public Health. 5th ed. New york(U.S.A): Oxford University
Press; 201
• JJ Deekset et al. Evaluating non-randomized intervention studies: Health Technology Assessment 2003;
Vol. 7: No. 27
• Friis RH, Sellers TA. Epidemiology for Public Health Practice. 4th ed. Sudbury, MA: Jones and Bartlett
Publishers; 2009.
• MacMahon B, Pugh TF. Epidemiology Principles and Methods. Boston, MA: Little, Brown; 1970.
• Merrill.M. Introduction to Epidemiology.2010;5th ed:83-153.
• Bonita R, Beaglehole R, Kjellstrom K. Basic Epidemiology.2006 Jan;2ND ed
• Bhalwar R. Text Book of Public Health and Community Medicine. 1st ed. Pune: Dept of Community
Medicine, AFMC. 2009. P. 144
• D’Agostino RB, Kwan H. Measuring effectiveness: what to expect without a randomized control group.
Med Care 1995;33:95–105.
• Grimshaw J, Campbell M, Eccles M, Steen N. Experimental and quasi-experimental designs for
evaluating guideline implementation strategies. Family practice. 2000 Feb 1;17(suppl_1):S11-6.