2. DIFFERENTIAL ITEM FUNCTIONING (DIF)
Also referred to as item bias
Tests should have fairness in when test
takers from two subgroups of interest such
as males or females, etc.
DIF occurs when test takers on two groups of
interest who are matched on their ability
have unequal chances of success. The
assessment material is not used and
attempts to fond explanations for the
difference in performances are made.
3. DIFFERENTIAL ITEM FUNCTIONING
An item exhibits DIF (or is potentially biased)
if students of equal ability but from different
sub groups (ex. males and females) have an
opportunity for correctly responding to that
item (or an equal expected score on the
assessment task, for polychotomous items)
(Humbleton et al., 1991).
4. DIFFERENTIAL ITEM FUNCTIONING
If discrepancy in item performance between
the subgroups of interest is equal across the
entire range of abilities, then DIF is said to be
nonuniform.
5. STATISTICAL METHODS USING IRT
Comparing ICC’s
Significance test of item parameter estimates
of two groups (Lord, 1980).
Logistic regression-The item bias variable is
identified and coded. This variable is used as
a predictor together with the items. If the
interaction term is significant item bias exist
for that item.
6. STATISTICAL METHODS USING IRT
Mantel-Haenszel method
Matches examiners in two groups of interest
using total test score (or on the adjusted test
score by removing potentially biased test items
that might be identified by a judgmental review or
an initial statistical analysis).
Majority and minority group members are
assigned to groups based on their total scores.
Examinees are matched on their ability levels an
then item performance on the two groups are
compared in each score group.
7. STATISTICAL METHODS USING IRT
Differences in item performance in each
group are weighted by the size of the sample
and then summed over the score groups.
It is possible to test the sum for statistical
significance using chi-square.
8. JUDGMENTAL REVIEWS
Panelists consisting of minority group members,
curriculum specialist, and measurement
specialists are often formed to review
assessment materials for bias (racial, ethnic,
gender, regional, age, etc.)
Panel detects items that are reflecting
stereotyping (males performing physical
activities and females working in the house).
Panel looks at aspects of the assessment
material (situations, vocabulary, assessment
format, reading level, time limits)
9. ITEM BIAS IN A CROSS CULTURAL PERSPECTIVE
Construct bias
Method bias
Item bias
10. CONSTRUCT BIAS
Construct bias-different meanings attached
to some concepts.
Ex. Intelligence: Social intelligence (roles on
the family, ability to deal with complicated
situations) is not covered in psychological
tests.
11. METHOD BIAS
When a cultural factor that is not relevant to
the construct studied affects most or all of the
items of a test in a different way across the
culture studied.
Ex. self-disclosure, social desirability,
stimulus content,
Raven’s progressive matrix-lessens cultural
bias.
12. ITEM BIAS/DIFFERENTIAL ITEM FUNCTIONING
Anomalies in the instrument that are specific
to individual items.
Types:
Uniform bias
Nonuniform bias
13. DETECTING ITEM BIAS
2x2 contingency table for each item.
Ex. Pass and fail on the item vs. Filipino and
Singaporeans
Ex. right vs. wrong on an item vs. males and
females.
Analysis of Variance of item scores on a
categorical variable as a source of item bias.
Ex.Main and interaction effects of cultural
groups and ability groups on each item score.
