Evaluation

1. Evaluation of Evidence
10/3/2023

2. Objectives
 At the end of this session the student will be able to:
 Identify the factors that may result in false
associations
 Explain the mechanisms of avoiding or reducing the
role of chance, bias and confounding
 Analyze cause-effect relation ship
 Discuss the criteria of causation
10/3/2023

3. ANALYSIS OF CAUSE-EFFECT RELATIONSHIPS
The existence of statistical significant
association does not in itself constitute a proof
of causation.
The observed association could be real or false
(art factual)
10/3/2023

4. Alternative explanations for the observed association other
than cause and effect relationships
A) The association may be the result of chance
B) The association may be the result of bias
C) The association can be the result of a confounding effect.
D) An apparent cause can be an effect, rather than a cause
(reverse causation)
E) The cause can be both a cause and effect (reciprocal causation)
E.g. Vitamin A deficiency can cause diarrhea or diarrhea can cause
Vitamin A deficiency
10/3/2023

5. 10/3/2023

6. Common problems in studies that distort
observed findings
Inadequacy of the observed sample
Inappropriate selection of study subjects
Inappropriate data collection methods
10/3/2023

7. Validity of epidemiological studies
Accuracy = Validity + Precision
Validity
 the degree of closeness between a measured
value and the true value of what is being
measured
 Validity is often described as internal or
external.
10/3/2023

8. Internal validity
 Internal validity is threatened when the investigator does not
have sufficient data to control or rule out competing
explanations for the results
External validity(generalizability):
 can we make inferences beyond the subjects of the study?
 External validity is threatened, when the investigator
attempts to apply the findings of the study to a population,
which is not comparable to the population.
10/3/2023

9.  Judgment about causality should address 2
major area
– it is important to check the role of chance, bias
and confounding
– then assess other supportive evidences
“Bradford Hill’ framework criteria”
10/3/2023

10. Bias
 is a systematic error in the design,
implementation, or analysis of a study that
results in an estimate that differs from the
truth
 Any systematic error resulting in an incorrect
estimate of association between exposure
and risk of disease
 Direction of effect is unpredictable and
identifying sources of bias can be difficult
10/3/2023

11. Two Types of bias
1 . Selection bias: systematic error in the process of
identifying the study populations which may result
the sample to be unrepresentative of the study
population
2. Observation or information bias:
systematic error results from in away data on
exposure or outcome are obtained differently.
10/3/2023

12. 1. Selection bias
 Any error that arises in the process of identifying the study
populations
 selection bias affects the representativeness of the study
subjects, either as result of sample selection or as result of
non response or lost to follow up.
 Selection bias occur when non-comparable criteria are
used to enroll participants in the study.
 Selection bias is a major problem in case control studies
10/3/2023

13.  Selection bias include
1) Berkson’s bias- differential hospital
admission criteria, Hospitalization rates for case
& control differ)
 Case-control studies carried out exclusively in
hospital settings are subject to selection bias
attributable to the fact that risks of
hospitalization can combine in patients who
have more than one condition
10/3/2023

14. 2. Ascertainment (diagnostic) bias – occur when a disease is
more likely to be diagnosed in some one who has exposure
to a suspected risk factor.
Example: women who take oral contraceptives may be
screened more often for breast cancer than women who do
not take oral contraceptives b/c of the suspected link b/n
oral contraceptive and breast cancer.
3. Non-response bias- Rates of response to surveys and
questionnaires in many studies may also be related to
exposure status
10/3/2023

15. 4. Loss to follow-up-
 This is a major source of bias in cohort studies
 Persons lost to follow-up may differ from with respect
to both exposure and outcome, biasing any observed
association
5. Volunteer / Compliance bias– volunteer or comply
with medical treatment may differ from non
volunteers
10/3/2023

16. Ways of minimizing selection bias
 Give adequate explanation about the study for potential
participants
 Sensitize the community to the value of the research
 Appoint sensitive but persistent data collectors
 Use simple, concise instruments
 Population based studies are preferred
 One should avoid the inclusions of volunteer study subjects
 Select several different control groups in case control study
10/3/2023

17. 2. Observation or information bias
 includes any systematic error in the
measurement of information on exposure or
outcome
10/3/2023

18. Types of Information Bias
1. Investigator bias/ Interviewer bias/ Observer bias
– Occurs when investigators collect information differently in
different comparison groups
– an interviewer’s knowledge may influence the structure of
questions and the manner of presentation
2. Recall Bias – those with a particular outcome or exposure
may remember events more clearly or amplify their
recollections
10/3/2023

19. Information Bias (cont.)
3. Hawthorne effect( attention bias) – people act
differently if they know they are being watched
4. Surveillance bias – the group with the known
exposure or outcome may be followed more closely or
longer than the comparison group
10/3/2023

20. 5. Social desirability bias- Occurs because subjects are
systematically more likely to provide a socially acceptable
response.
6. Healthy worker bias - Refers to the bias in occupational
health studies which tends to underestimate the risk
associated with an occupation due to the fact that
employed people tend to be healthier than the general
population
10/3/2023

21. Ways of minimizing information bias
1. Blinding. There are three types of blinding
A. Single blind
– the study subjects doesn't know to which group they are assigned
B. Double blind
– neither the study subjects nor the data collector know the group to which
the subject has been assigned
C. Triple blind
– the study subjects, the data collector and the individual who is doing the
analysis are blinded
 Blinding is of greatest importance when the outcome is
subjectively determined. If the outcome is objectively
determined (e.g. death, stroke), blinding is less essential
10/3/2023

22. 2. In both comparison groups use the same
 procedures,
 instruments,
 questionnaires,
 interviewing techniques etc
3. Classification of study subjects according to their
outcome & exposure status should be based on the
most objective & accurate methods available
4. Choose "hard" (objective) rather than “soft” (subjective)
outcomes
5. Choose and stick to standardized questionnaire & instrument
– Use closed-ended questions whenever possible
10/3/2023

23. Confounding
 Confounding refers to the mixing of the effect of
an extraneous variable with the effects of the
exposure and disease of interest
 Confounding arises when some cause other than
the exposure under study is more or less,
prevalent in the exposed group than in the
unexposed group
10/3/2023

24. Characteristics of a Confounding Variable
1. the variable (confounder) must be associated with the exposure
and, independent of that exposure, be a risk factor for the disease
Exposure disease
confounder
2 . Associated with the study exposure but not as a consequence of
the exposure
10/3/2023

25. Effect of Confounding
 the effect could be :
 Totally or partially accounts for the apparent effect
 Mask an underlying true association
 Reverse the actual direction of the association
10/3/2023

26. Confounding
Age
Grey hair Death
Age (the confounder) is strongly and independently
associated both with the outcome (dying) and with the
exposure (grey hair)
If left uncontrolled, the confounder would have produced
a spurious association between exposure and disease
Example
10/3/2023

27. Control for Confounding Variables
 In the design stage confounding could be minimized by
1. Randomization
• ensures that all potential confounding factors are evenly distributed
among the comparison groups
2. Restriction
• Individuals who fall within a specific category or categories of the
confounder will be included in the study
• e.g if smoking is a potential confounding factor, either smokers only
or non-smokers only will be included in the study
3. Matching: matching can be done during the design & analysis stage
10/3/2023

28. Evaluation of confounding in the analysis by
1. Stratified analysis
 Used during the analysis stage
 Involves the evaluation of the association within
homogenous categories or strata of the confounding
variable
2. Multivariate analysis
control many confounding factors simultaneously
10/3/2023

29. The Role of chance
 One of the alternative explanation to the observed
association between an exposure and a disease is
chance
 Aim of epidemiological studies is to make
generalization about a larger group of individuals
on the basis of a sample population
 It is always important to evaluate the role of
chance or sampling variability
10/3/2023

30.  An association between exposure and outcome may
occur by chance alone
 The sample selected may by chance demonstrate an
association not present in the larger community
 Hence chance must always be considered as an
alternative explanation to observed findings,
i.e. we should quantify the power & role of chance
using tests of statistical significance: (p-value or CI)
10/3/2023

31. Assessment of evidence for causality
Judgments of causality must consider:
1. whether the observed association is valid
 Once we found that chance, bias and
confounding are all determined to be unlikely,
then we can conclude that a valid statistical
association exists
2. whether the accumulated evidence supports a
cause-effect relationship.
10/3/2023

32. Associations can be:
1.Artifactual (spurious) associations
Chance (i.e., type 1 error) and Bias
2.Noncausal (indirect) associations,
a. The associated factor is itself an effect, rather than a cause
(reverse causation), or both a cause and an effect (reciprocal
causation)
b. Confounding effect by a third variable
3. Causal associations, which can be established only when other
potential explanations of the association can be ruled out
10/3/2023

33.  To secure evidence experimental confirmation of a
cause-effect relationship is ideal, but often impossible.
 In the absence of experimental confirmation, the
‘Bradford Hill’ framework is often applied.
Assessment of evidence for causality
10/3/2023

34. The Bradford Hill Framework
1. Strength of association
2. Consistency of relationship
3. Specificity of association
4. Temporal relationship
5. Dose-response relationship: Biologic gradient:
6. Biological plausibility: explainable within the existing knowledge
7. Reversibility (experimental): supported by experiment
 It is the statement of epidemiological criteria of a causal
association formulated in 1965 by Austin Bradford Hill
(1897-1991)
10/3/2023

35. 1. Strength of the Association
• The stronger the association the more likely that it is a causal. As
RR/OR approaches 1, the association is weak. Strong --- the more
it is far from unity (<0.5 and >2).
• Relative risk Interpretation
• 1.1-1.3 : Weak
• 1.4-1.7: Modest
• 1.8-3.0: Moderate
• 3-8: Strong
• 8-16: Very strong
• 16-40: Dramatic
• 40+: Overwhelming
10/3/2023

36. 2. Consistency of Relationship
• The same association should be demonstrated by
other studies both with different methods, settings
and different investigators.
 That is why we compare our findings with other
studies.
• Special methods of combining of a number of well
designed studies is Meta Analysis.
10/3/2023

37. 3. Specificity of the association;
Single exposure Single disease
This works more to most living organisms as
causes.
Plasmodium Sp. Malaria
HIV AIDS
10/3/2023

38. 4. Temporal relationship
• It is crucial that the cause must precede the
outcome
• The study should show that exposure be observed
first and outcome next
• This is usually problematic in cross-sectional and
case-control designs.
exposure disease
10/3/2023

39. 5. Dose response relationship
The risk of disease increases with increasing
exposure intensity.
Ex-1: death due to ischemic heart disease among male
doctors above the age of 45 years followed in cohort
Ex-2 Death rate
per 100,000 pers-
yr
Risk
Ratio
Non-smokers 7 1.00
1-14 cigarett/ day 46 6.57
15-24 cigare/ day 61 9.71
25+ cigarettes/ day 104 14.86
10/3/2023

40. 6. Biological Plausibility
• Hypothesis should be coherent with what is
known about the disease; both biologically
and using laboratory.
• Knowledge about physiology, biology and
pathology should support the cause-effect
relationship
• If a finding doesn’t go with what is known,
then it is losing biological plausibility
10/3/2023

41. 7. Reversibility
• Removal of a possible cause results in a
reduced disease risk
 Ex: Cessation of cigarette smocking is
associated with reduction in risk of Lung
cancer relative to those who continue.
10/3/2023

42. Summary in Judging the evidence
There are no completely reliable criteria for
determining whether an association is causal or
not.
In judging the different aspects of causation,
• First look at the temporal relationship if it is
correct,
• Look at
–Plausibility,
–Consistency,
–dose-response relationship and
–Strength of the association
10/3/2023

43. Type of study Ability to give
evidence
Randomized controlled
trial
Most Stronger
Cohort study Stronger
Case-control study Moderate
Cross-sectional studies weak
Ecological studies More weaker
Types of study design
10/3/2023