Psychological Testing and Assessment - Validity

1. VALIDITY

2. WHAT IS VALIDITY?
• It refers to the judgment or estimate of how well a test
measures what it purports to measure in a particular
context.
• Determines the appropriateness of inferences drawn from
the test scores.
• Inference: logical result/deduction
• How useful it is for a particular purpose with a particular
population of people?
• Be valid with a particular population of testtakers at a
particular time
• Reasonable boundaries of contemplated usage
• NOTE: Validity is required before reliability can be
considered in a meaningful way

3. FACE VALIDITY
• What a test appears to measure to the person
• Relevance of test items
• Lack of face validity could contribute to the lack of
confidence in its perceived effectiveness
• It is a matter of public relations than psychometric
soundness.

4. CONTENT VALIDITY
• How adequately a test samples behavior representative
of the universe of behavior
• Determines whether the content of the test
• Test Blueprint – type of information to be covered by
the number of items; number of items tapping;
organization of items
• C. H. Lawshe quantified content validity wherein raters
would determine whether the question is essential,
useful but not essential, or not necessary.

5. CRITERION-RELATED
VALIDITY
• How adequately a test score can be used to infer an
individual’s probable standing on some measure of interest.
• It involves the correlation between test and criterion
variable taken as a representative of the construct
• Compares the test with other measures/outcomes already
held to become valid.
• Criterion: the standard on which a decision may be based.
• A criterion must be:
• Relevant
• Valid
• Uncontaminated

6. CRITERION-RELATED
VALIDITY
• Concurrent Validity: measures the relationship
between test scores and criterion measures; test scores
are obtained at the same time as the criterion
measures.
• May be used to indicatethe individual’s current standing
on a criterion
• Predictive Validity: measures the relationship
between test scores and criterion measures obtained at
a future time; how does scores predict criterion
measures

7. CRITERION-RELATED
VALIDITY
• Validity Coefficient: a correlation coefficient provides
a measure of relationship between test scores and
scores on the criterion measures.
• Depending on the type of data, sample size, and shape
of the distribution, other correlation coefficients may be
used.
• Cronbach and Gleser argued that validity coefficients
needs to be large to enable test user to make accurate
decisions within the unique context in which a test is
being used

8. CRITERION-RELATED
VALIDITY
• Incremental Validity: degree to which an additional
predictor explains something about criterion measure that
is not explained by predictors being used
• Expectancy Tables: illustrates the likelihood that a testtaker
will score some interval on a certain criterion measures. It
shows the percentage of people within specified test-score
intervals.
• Taylor-Russell Tables: estimates the extent to which an
inclusion of a particular test in selection will improve
selection (i. e., employee selection).
• Naylor-Shine Tables: obtains the difference between
means of selected and unselected groups to derive an index
of what the test is adding to the established procedures

9. CRITERION-RELATED
VALIDITY
• Base rate: the extent to which a particular trait, behavior,
characteristic, or attribute exists in the population
• Hit rate: proportion of people a test accurately identifies as
possessing or exhibiting a particular trait
• Miss rate: the proportion of people the test fails to identify
as having, or not having, a particular characteristic or
attribute
• False Positive: testtaker possessed a particulat
trait/attribute when the testtaker did not
• False Negative: test predicted the testtaker did not possess
a particular characteristic when the testtaker did

10. CONSTRUCT VALIDITY
• A judgment of the appropriateness of inferences drawn
from test scores regarding individual standing on a
construct
• Constructs: informed, scientific idea
developed/hypothesized to describe or explain
behavior.
• According to the American Educational Research
Association, it is a unifying concept for all validity
evidence

11. CONSTRUCT VALIDITY
• Evidence of Construct Validity:
• Homogeneity
• Correlations between subtest scores and total test score
• Changes with Age
• If a test score purports to measure a construct expected to
change over time, then progressive changes in scores are
expected
• Pretest-Posttest
• Changes in scores as a result of experience
• Distinct Groups
• Demonstrate that scores on test vary in a predictable way as a
function of membership in some group

12. CONSTRUCT VALIDITY
• Evidence of Construct Validity (cont’d):
• Convergent Validity
• A test measuring the same construct as others purports to
measure the same construct
• Discriminant Validity
• A validity coefficient showing little relationship between
scores/variables with which scores on the test being construct
validated should be theoretically correlated.
• Multitrait-Method Matrix: results from correlating
variables/traits within and between methods.
• Factor Analysis
• Mathematical procedure used to identify specific
factors/variables that are attributes, characteristics, or
dimensions on which people may differ
• Exploratory Factor Analysis: extraction of factors
• Confirmatory Factor Analysis: how hypothetical model fits
the data

13. VALIDITY, BIAS, AND
FAIRNESS
• Bias: factor inherent in a test that prevents the accurate, impartial
measurement
• Intercept bias: regression line intersects with the X-axis
• Slope bias: test yields significantly different validity coefficients
for members of different groups
• Rating error: intentional/unintentional misuse of a scale.
• Leniency error: tendency of rater to be lenient in scoring
• Severity error: tendency for the rater to give strong ratings
• Central tendency error: tendency to be giving rating at either
positive or negative extremities
• Halo effect: the tendency for some raters to see no wrong
among ratees

14. VALIDITY, BIAS, AND
FAIRNESS
• Fairness is a difficult construct to be defined in
psychometric means.
• Unfairness can be due to differences
• Differences found must be an artifact of an unfair or biased
test
• Can be difficult to refute

15. WRAP-UP
• We discovered the various types of validity
• We must always ensure that our test would include
questions or factors that would seek answers onto
what we are quantifying
• We must make measures on how we can eliminate bias
with our own test/s.