The document discusses the application of Item Response Theory (IRT) using the Rasch model to construct cognitive measures. It provides an overview of psychometric theory, classical test theory, and IRT approaches like the Rasch model. The Rasch model assumes that the probability of a correct response depends only on the difference between a person's ability and the item difficulty. It provides sample-independent item calibrations and person measures. The document outlines the assumptions, uses, and procedures of the Rasch model for test analysis.

1. The Application of IRT using
the Rasch Model in
Constructing Cognitive
Measures
Carlo Magno, PhD
De La Salle University-Manila
1

2. Outline
• Psychometric Theory
• Classical test Theory (CTT)
• Item Response Theory (IRT)
• Approaches in IRT
• Issues in CTT
• Advantages of the Rasch Model
• Assumptions of the Rasch Model
• Uses of the Rasch Model
• Procedure in Rasch Model
• Workshop
2

3. Psychometrics
• Psychometrics concerns itself with the
science of measuring psychological
constructs such as ability, personality,
affect and skills.
• Research in psychology involves the
measurement of variables in order to
conduct further analysis.
3

4. Branches of Psychometric
Theory
• Classical Test Theory
• Item Response Theory
4

5. Classical Test Theory (CTT)
• Regarded as the “True Score Theory”
• Responses of examinees are due only to
variation in ability of interest
• All other potential sources of variation existing in
the testing materials such as external conditions
or internal conditions of examinees are assumed
either to be constant through rigorous
standardization or to have an effect that is
nonsystematic or random by nature
5

6. Classical Test Theory (CTT)
TO = T + E
• The implication of the classical test theory
for test takers is that test are fallible
imprecise tools
• Error = standard error of measurement
Sm = S 1 - r
• True score = M +- Sm = 68% of the
normal curve
6

7. Normal curve
One SE
Range of True
mean Scores
7

8. Focus of Analysis in CTT
• frequency of correct responses (to indicate
question difficulty);
• frequency of responses (to examine
distracters);
• reliability of the test and item-total correlation
(to evaluate discrimination at the item level)
8

9. Item Response Theory
• Synonymous with latent trait theory, strong true
score theory or modern mental test theory
• More applicable to for tests with right and wrong
(dichotomous) responses
• An approach to testing based on item analysis
considering the chance of getting particular
items right or wrong
• each item on a test has its own item
characteristic curve that describes the probability
of getting each particular item right or wrong
given the ability of the test takers (Kaplan &
Saccuzzo, 1997)
9

10. Logistic Item Characteristic
Curve
– A function of ability
() – latent trait
– Forms the boundary
between the
probability areas of
answering an item
incorrectly and
answering the item
correctly
10

11. Approaches of IRT
• One dimension (Ogive) One parameter
model = uses only the difficulty parameter
• Two dimension (Rasch Model)  Two
parameter Model = difficulty and ability
parameter
• Three dimension (Logistic Model) Three
Parameter Model = ability, correct
response, item discrimination
11

12. Issues in CTT
• A score is dependent on the performance of the group
tested (Norm referenced)
• The group on which the test has been scaled has
outlived has usefulness across time
– Changes in the defined population
– Changes in educational emphasis
• There is a need to rapidly make new norms to adopt to
the changing times
• If the characteristics of a person changes and does not
fit s specified norm then a norm for that person needs to
be created.
• Each collection of norms has an ability of its own =
rubber yardstick
12

13. Advantages of the Rasch Model
• The calibration of test item difficulty is
independent of the person used for the
calibration.
• The method of test calibration does not matter
whose responses to these items use for
comparison
• It gives the same results regardless on who
takes the test
• The scores a person obtain on the test can be
used to remove the influence of their abilities
from the estimation of their difficulty. The result
is a sample free item calibration.
13

14. Rasch Model
• Rasch’s (1960) main motivation for his
model was to eliminate references to
populations of examinees in analyses of
tests.
• According to him that test analysis would
only be worthwhile if it were individual
centered with separate parameters for the
items and the examinees (van der Linden
& Hambleton, 2004).
14

15. Rasch Model
• The Rasch model is a probabilistic
unidimensional model which asserts that:
(1) the easier the question the more
likely the student will respond correctly to
it, and
(2) the more able the student, the more
likely he/she will pass the question
compared to a less able student.
15

16. Rasch Model
• The model was enhanced to assume that the
probability that a student will correctly answer a
question is a logistic function of the difference
between the student's ability [θ] and the difficulty
of the question [β] (i.e. the ability required to
answer the question correctly), and only a
function of that difference giving way to the
Rasch model
• Thus, when data fit the model, the relative
difficulties of the questions are independent of
the relative abilities of the students, and vice
versa (Rasch, 1977).
16

17. Assumptions of the Rasch Model
According to Fisher (1974)
• (1) Unidimensionality. All items are functionally
dependent upon only one underlying continuum.
• (2) Monotonicity. All item characteristic functions
are strictly monotonic in the latent trait. The item
characteristic function describes the probability
of a predefined response as a function of the
latent trait.
• (3) Local stochastic independence. Every
person has a certain probability of giving a
predefined response to each item and this
probability is independent of the answers given
to the preceding items.
17

18. Assumptions of the Rasch Model
According to Fisher (1974)
• (4) Sufficiency of a simple sum statistic. The
number of predefined responses is a sufficient
statistic for the latent parameter.
• (5) Dichotomy of the items. For each item there
are only two different responses, for example
positive and negative. The Rasch model
requires that an additive structure underlies the
observed data. This additive structure applies to
the logit of Pij, where Pij is the probability that
subject i will give a predefined response to item
j, being the sum of a subject scale value ui and
an item scale value vj, i.e. In (Pij/1 - Pij) = ui + vj
18

19. Uses of the Rasch Model
• Identifies items that are acceptable –
items that are significantly different from
0 are good items
• Indicates whether an item is extremely
difficult or easy
19