2. Validity
□
Validity
Validity refers to the degree to which evidence and theory
support the interpretations of test scores entailed by proposed
uses of tests.
*Internal validity is the extent to which research findings are
free from bias and effects.
*External validity is the extent to which the findings can be
generalized.
3. Types of Validity
3. Content Validity:
Content-related evidence of validity focuses on the content and format of an
instrument.
Is it appropriate? Comprehensive? Is it logical? How do the items or questions
represent the content? Is the format appropriate?
Consult other experts who rate the items.
Rate items, eliminating or changing those that do not meet the specified content.
Repeat until all raters agree on the questions and answers.
4. Types of Validity
4. Face Validity:
Face validity occurs where something appears to be valid. This depends very much
on the judgment of the observer.
5. Test Validity:
Criterion:
This refers to the relationship between the scores obtained using the instrument and
the scores obtained using one or more other instruments or measures. For example,
are students’ scores on teacher made tests consistent with their scores on
standardized tests in the same subject areas?
6. Construct Validity:
Construct validity is defined as “establishing correct operational measures for the
concepts being studied” (Yin, 1984).
For example, if one is looking at problem solving in leaders, how well does a
particular instrument explain the relationship between being able to problem solve
and effectiveness as a leader.
5. Reliability
Reliability
*The consistency of scores or answers from one
administration of an instrument to another, or from
one set of items to another.
*A reliable instrument yields similar results if given
to a similar population at different times.
*Reliability coefficient - a number that tells us how
likely one instrument is to be consistent over
repeated administrations.
7. Types of Reliability
1. Stability Reliability:
Test-retest:
➢
➢
➢
➢ Test-retest reliability is the degree to which scores are
consistent over time. It indicates score variation that occurs
from testing session to testing session as a result of errors of
measurement.
Same test- different Times
Only works if phenomenon is unchanging
Example: Administering the same questionnaire at 2 different
times.
8. Types of reliability
2. Equivalence Reliability:
a) Inter-item reliability: (internal consistency)
➢
➢ The association of answers to set of questions designed to
measure the same concept.
Cronbach’s alpha is a statistic commonly used to measure
inter-item reliability which is based on the average of all the
possible correlations of all the split 1/2 s of set of questions on
a questionnaire.
9. Types of Reliability
b) Parallel form of Reliability
Split-Half Reliability:
□
□
Especially appropriate when the test is very long. The most
commonly used method to split the test into two is using the
odd-even strategy.
Since longer tests tend to be more reliable, and since split-half
reliability represents the reliability of a test only half as long as
the actual test.
C) Inter observer Reliability:
Correspondence between measures made by different observers.
10. Quantitative vs. Qualitative
Traditional Criteria for
Judging Quantitative
Research
Alternative Criteria for
Judging Qualitative
Research
Internal validity
External validity
Reliability
Objectivity
Credibility
Transferability
Dependability
Confirmability
