This document discusses validity and reliability in research. It defines validity as the extent to which a test measures what it claims to measure. Reliability is defined as the extent to which a test shows consistent results on repeated trials. The document then discusses various types of validity including content, face, criterion-related, construct, and ecological validity. It also discusses types of reliability including equivalency, stability, internal consistency, inter-rater, and intra-rater reliability. Factors affecting validity and reliability are presented along with how validity and reliability are related concepts in research.

1. DR. MOUSUMI SARKAR; PGT COMMUNITY MEDICINE

2. VALIDITY
“Validity is the extent to which a test measures what it
claims to measure”.
RELIABILITY
 Reliability is the extent to which an experiment, test, or
any measuring procedure shows the same result on
repeated trials.

4. TYPES OF VALIDITY
 Content validity
 Face validity
 Curricular validity
 Criterion-related validity
 Predictive validity
 Concurrent validity
 Construct validity
 Convergent validity
 Discriminant validity

5. Content Validity:
It is the extent to which the measurement method covers the entire
range of relevant behaviors, thoughts, and feelings that define the
construct being measured.
Face Validity:
 It is the extent to which the measurement method appears “on its face”
to measure the construct of interest.
Curricular Validity is the extent to which the content of the test matches
the objectives of a specific curriculum .

6. Criterion validity
It is the extent to which people’s scores are correlated with other
variables or criteria that reflect the same construct.
TYPES OF CRITERION VALIDITY:
Predictive Validity:
When the criterion is something that will happen or be assessed in the
future, this is called predictive validity.
Concurrent Validity:
When the criterion is something that is happening or being assessed at
the same time as the construct of interest, it is called concurrent
validity.

7. Construct Validity
 Construct validity refers to the degree to which a test or other
measure assesses the underlying theoretical construct it is
supposed to measure.
Convergent validity
 consists of providing evidence that two tests that are believed to
measure closely related skills or types of knowledge correlate
strongly.
Discriminant validity
 by the same logic, consists of providing evidence that two tests that
do not measure closely related skills or types of knowledge do not
correlate strongly.

9. Ecological Validity:
 It refers to the extent to which the findings can be generalized beyond
the present situation.

10. External Validity:
 It is the extent to which the results of a research study
can be generalized to different situations, different
groups of people, different settings, different
conditions, etc.
Internal Validity:
 It is basically the extent to which a study is free from
flaws and that any differences in a measurement are
due to an independent variable and nothing else.

11. Steps for Assessing Validity of an Experimental Study
Step Assessment Process Decision
1. Validity of statistical
conclusion
Assess statistical
significance
(i.e., p value is
0.05 and statistical results
are valid).
Difference is real and is not
likely due to chance
variation; proceed to next
step./Difference is likely due
to chance variation; stop
here.
2. Internal validity Assess internal validity on
basis of research design and
operational procedures.
Difference is most likely due
to the treatment; proceed to
next step./Difference is
probably due to the effects of
confounding factors or bias;
stop here.
3. External validity Examine inclusion and
exclusion criteria and
characteristics of study
participants.
Study participants are similar
to patients the report reader
sees; the treatment should
be useful./Study participants
are very different from
patients the report reader
sees; the treatment may or
may not be useful.

13. Measurement of Validity
 Important points to evaluate the validity of a measurement method.
 1. First, this process requires empirical evidence. A measurement method
cannot be declared valid or invalid before it has ever been used and the
resulting scores have been thoroughly analyzed.
 2. Second, it is an ongoing process. The conclusion that a measurement
method is valid generally depends on the results of many studies done over
a period of years.
 3. Third, validity is not an all-or-none property of a measurement method. It
is possible for a measurement method to judged "somewhat valid" or for one
measure to be considered "more valid" than another.

14. Measures of validity
 Sensitivity
 Specificity
 AUC

15. ROC curve

16. Factors Affecting Validity
 History
 Maturation
 Testing
 Instrumentation
 Statistical Regression
 Experimental Mortality
 Compensatory Rivalry by Control Group
 Compensatory Equalization of Treatments
 Resentful Demoralization of Control Group

19. RELIABILITY
 Unable to satisfactorily draw conclusions, formulate theories,
or make claims about the generalizability of the research.
 Types of reliability are:
 “ Equivalency ”
 “ Stability ”
 “Internal consistency”
 “ Inter-rater ”
 “ Intra-rater”

20. Equivalency reliability :-
 The extent to which two items measure identical
concepts at an identical level of difficulty.
 Equivalency reliability is determined by relating two sets
of test scores to one another to highlight the degree of
relationship or association.
Stability reliability (test, re-test reliability ):-
 It is the agreement of measuring instruments over time.
 To determine stability, a measure or test is repeated on
the same subjects at a future date. Results are compared
and correlated with the initial test to give a measure of
stability.

21. INTERNAL CONSISTENCY
 Internal consistency is the extent to which tests or
procedures assess the same characteristic, skill or quality.
 It is a measure of the precision between the measuring
instruments used in a study.

22. Inter-rater reliability :- The
extent to which two or more
individuals (coders or
raters) agree.
Intra-rater reliability : is a
type of reliability
assessment in which the
same assessment is
completed by the same
rater on two or more
occasions.

27. Reliability Coefficient
 Pearson product moment correlation:-
The extent to which the relation between two variables can be described by
straight line.
 Bland-Altman analysis:-
A plot of difference between two observations against the means of the two
observations.
 Cohen’s Kappa:-
How much better is their level of agreement is than that which result just
from chance

29. Threats to Reliability
 Subject Reliability:
Factors due to research subject.
 Observer Reliability:
Factors due to observer/rater/interveiwer.
 Situational Reliability:
Conditons under which measurements are made(eg, busy day at
the clinic)
 Instrument Reliability:
The research instrument or measurement approach itself.(eg,
poorly worded questions, quirk in mechanical devices)
 Data processing Reliability:
Manner in which data are handled(eg; miscoding).

30. Relationship of validity & reliability
 Validity and reliability are closely related.
 A test cannot be considered valid unless the measurements resulting
from it are reliable.
 Likewise, results from a test can be reliable and not necessarily valid.
 The proponents of the exclusive trend claim that the terms ‘validity &
Reliability’ do not make sense in qualitative research, so they should be
replaced.
 Internal validity-Credibility
 External validity-Transferability
 Reliability- Dependability

32.  Validity
 Does it measure what it is supposed to measure?
 Reliability
 How representative is the measurement?