This document discusses evaluation of evidence and common problems in observational studies. It describes the role of chance and how statistical significance and p-values are used to evaluate chance. Bias such as selection bias and information bias are discussed. Selection bias can occur through inappropriate selection of study subjects. Information bias arises from mistakes made during data collection. Confounding occurs when a variable is associated with both an exposure and an outcome. Methods to control for confounding include randomization, restriction, matching, and adjustment in analysis. Criteria for evaluating causal relationships include strength of association, dose-response relationship, consistency, temporality, specificity, plausibility, and prevention.

1. EVALUATION OF EVIDENCE
Shikur MohammedShikur Mohammed
(BSc, MPH)
2/2/2015 1By Shikur Mohammed (BSc, MPH)

2. Evaluation of Evidence
Evaluation is a process of determining the
usefulness, reliability and validity of something
against explicit predetermined standards
2/2/2015 2By Shikur Mohammed (BSc, MPH)

3. Common Problems in observed findings
Inadequacy of the observed sample (The role of
chance)
Inappropriate selection of study subjects & unfair
data collection methods (Bias)data collection methods (Bias)
Comparing unequal (Confounding)
2/2/2015 3By Shikur Mohammed (BSc, MPH)

4. 1. The role of chance
The larger the sample on which the estimate is
based, the less variability and the more reliable the
inference
This is done by performing an appropriate test of
statistical significancestatistical significance
A measure that is often reported from all tests of
statistical significance is the P-value
2/2/2015 4By Shikur Mohammed (BSc, MPH)

5. The role of chance cont…
P < 0.05 - statistically significant
P > 0.05 – no statistically significant association (
chance can not be excluded as a likely explanation)chance can not be excluded as a likely explanation)
2/2/2015 5By Shikur Mohammed (BSc, MPH)

6. The role of chance cont…
It is always advisable to report the actual P-value
rather than merely that the results did or did not
achieve statistical significanceachieve statistical significance
confidence interval (CI) is far more informative
measure than P-value to evaluate the role of chance
2/2/2015 6By Shikur Mohammed (BSc, MPH)

7. Bias
It is any systematic error in the design, conduct or
analysis of a study
Types of biasTypes of bias
1. Selection bias,
2. Information (Observation) bias
2/2/2015 7By Shikur Mohammed (BSc, MPH)

8. 1. Selection bias
It is a bias introduced while selecting the study
participants
It is a particular problem in case control andIt is a particular problem in case control and
retrospective cohort studies. E.g. If the way in
which participants are selected into the study is
different for cases and controls
2/2/2015 8By Shikur Mohammed (BSc, MPH)

9. Examples of selection bias
a) Diagnostic bias: Diagnostic approach related to
knowing exposure status
For example
women who take oral contraceptives (OCs) may be
screened more often for breast cancer than women who
do not take OCs because of the suspected link between
oral contraceptive and breast cancer
2/2/2015 9By Shikur Mohammed (BSc, MPH)

10. selection bias…
b). Volunteer bias/ Compliance bias: People who
accept to participate in a study or people who refuse
to participate are often quite different from the
general population
c). Non-response bias: This is due to differences in thec). Non-response bias: This is due to differences in the
characteristics between the responders and non-
responders to the study
2/2/2015 10By Shikur Mohammed (BSc, MPH)

11. selection bias cont…
d). Loss to follow up: Difference in completeness of
follow-up between comparison groups
e). Berkson’s bias: Studies carried out exclusively in
hospital settings are subject to selection biashospital settings are subject to selection bias
attributable to the fact that risks of hospitalization
can combine in patients who have more than one
condition
2/2/2015 11By Shikur Mohammed (BSc, MPH)

12. Ways of minimizing selection bias
• Population-based studies are preferable
• Avoid the inclusions as study subjects of people
who have volunteered on their own
In case-control study, it is useful to select several• In case-control study, it is useful to select several
different control groups
• keep losses to follow-up to minimum
2/2/2015 12By Shikur Mohammed (BSc, MPH)

13. 2. Information/Observation bias
Refers to bias which arises during the data collection
process
It occur because of mistakes in categorizing studyIt occur because of mistakes in categorizing study
subjects with respect to their exposure or disease
status
2/2/2015 13By Shikur Mohammed (BSc, MPH)

14. Examples of information bias
a) Investigator bias/ Interviewer bias/ Observer bias:
Occurs when investigators collect information
differently in different comparison groups
b) Response bias/Recall bias: Occurs as a result of
difficulty to recall prior exposuresdifficulty to recall prior exposures
c) Social desirability bias: Occurs because subjects are
systematically more likely to provide a socially
acceptable response
2/2/2015 14By Shikur Mohammed (BSc, MPH)

15. Examples information bias….
d). Placebo effect: tendency for individuals to report favorable
response to any therapy regardless of the physiologic efficacy
of what they received
e). Hawthorn effect: Refers to the change in the dependent
variable which may be due to the process of measurement or
observation itself
2/2/2015 15By Shikur Mohammed (BSc, MPH)

16. Ways of minimizing information bias
Blinding: the study subjects doesn't know to which
group they are assigned
using placebo: Use of placebo minimizes the bias in the
ascertainment of both subjective disease outcomes and
side effects. It facilitates that both groups in the study
gain equal attention
Using standard procedures, instruments, questionnaires,
interviewing techniques
2/2/2015 16By Shikur Mohammed (BSc, MPH)

17. Confounding
Confounding variable is a variable that can cause or
prevent the outcome of interest, is not an
intermediate variable, and is associated with the
factor under investigation
2/2/2015 17By Shikur Mohammed (BSc, MPH)

18. Confounding …
Confounding variable must fulfill each of the
following criteria:
1. the variable must be associated with the exposure and,
independent of that exposure, be a risk factor for the
diseasedisease
2. The distribution (frequency) of the confounding variable
should vary between the groups that are compared
3. Confounder must not be an intermediate link in a causal
pathway between exposure and outcome
2/2/2015 18By Shikur Mohammed (BSc, MPH)

19. Confounding …….Example
An observed association between drinking alcohol
and increased risk of lung cancer (LC) could be due
to the effect of cigarette smoking, since alcohol
drinking is associated with smoking and,drinking is associated with smoking and,
independent of alcohol consumption, smoking is a
risk factor for LC
2/2/2015 19By Shikur Mohammed (BSc, MPH)

20. Control for Confounding VariablesControl for Confounding VariablesControl for Confounding VariablesControl for Confounding Variables
In the design:
• Randomization
• Restriction
• Matching
During analysis:During analysis:
• Standardization
• Stratification
• Multivariate analysis
2/2/2015 20By Shikur Mohammed (BSc, MPH)

21. Establishing causal relationships
The following formal criteria are widely used to evaluate the
likelihood that an association is causal
1. Strength of the association
2. Dose-response relationship
3. Consistency of the relationship3. Consistency of the relationship
4. Temporal relationship
5. Specificity of the association
6. Biological plausibility (coherence with existing
information)
7. Prevention
2/2/2015 21By Shikur Mohammed (BSc, MPH)

22. Conclusion about causation
The above criteria are the ones most frequently employed in
trying to establish causation
None provides in itself a perfect means of providing
causation, and each has its limitationscausation, and each has its limitations
However, when they are considered together, the weight of
the evidence may allow a tentative conclusion to be reached
2/2/2015 22By Shikur Mohammed (BSc, MPH)