Item Response TheoryAdvance Psychometric Theory          CPS723P      Dr. Carlo Magno
Importance of Test Theories• Estimate examinee ability and  how the contribution of error  might be minimized• Disattenuat...
Psychometric History• Lord (1952, 1953) and other  psychometricians were interested in  psychometric models with which to ...
Limitations of the CTT• Item difficulty and item discrimination  are group dependent.• The p and r values are dependent on...
Assumptions in IRT• Unidimensionality  – Examinee performance is a single    ability• Response → Dichotomous  – The relati...
• Monotonicity of item performance  and ability is typified in an item  characteristic curve (ICC).• Examinees with more a...
• Mathematical model                                   linking the observable                                   dichotomou...
• Two-parameter          model: c=0        • One-parametera         model: c=0, a=1    b
• Three items  showing  different item  difficulties (b)
• Different levels  of item  discrimination
Polychotomous IRT Models• Having more than 2 points in the  responses (ex. 4 point scale)• Partial credit model• Graded re...
Graded Response model for a 5-point scale
• In IRT measurement framework,  ability estimates of an examinee  obtained from a test that vary difficulty  will be the ...
Test Characteristic Curve (TCC)                • TCC: Sum of ICC that                  make up a test or                  ...
Item Information Function          • I(Ѳ), Contribution of            particular items to the            assessment of abi...
Item Information Function
1                                                                       2                                                 ...
their corresponding IFFTest Information Function•   The sum of item information functions in a test.•   Higher values of t...
21.5  1 0.5      0                                    0                                3                             Abili...
Item Parameter Invariance          • Item/test characteristic            functions and item/test            information fu...
Benefits of ItemResponse Models• Item statistics that are independent of the  groups from which they were estimated.• Scor...
Application of IRT onTest Development• Item Analysis  – Determining sample invariant item    parameters.  – Utilizing good...
Application of IRT onTest Development• Item Selection  – Assess the contribution of each    item the test information func...
– Using item information functions:  • Describe the shape of the desired test    information function vs. desired range   ...
• Item banking  – Test developers can build an    assessment to fit any desired test    information function with items   ...
Irt 1 pl, 2pl, 3pl.pdf
Upcoming SlideShare
Loading in...5
×

Irt 1 pl, 2pl, 3pl.pdf

2,004

Published on

3 Comments
1 Like
Statistics
Notes
No Downloads
Views
Total Views
2,004
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
55
Comments
3
Likes
1
Embeds 0
No embeds

No notes for slide

Irt 1 pl, 2pl, 3pl.pdf

  1. 1. Item Response TheoryAdvance Psychometric Theory CPS723P Dr. Carlo Magno
  2. 2. Importance of Test Theories• Estimate examinee ability and how the contribution of error might be minimized• Disattenuation of variables• Reporting true scores or ability scores and associated confidence
  3. 3. Psychometric History• Lord (1952, 1953) and other psychometricians were interested in psychometric models with which to assess examinees independently of the particular choice of items or assessment tasks that were used in the assessment.• Measurement practices would be enhanced if item and test statistics would be made sample independent.• Birnbaum (1957, 1958)• George Rasch (1960)• Wright (1968)
  4. 4. Limitations of the CTT• Item difficulty and item discrimination are group dependent.• The p and r values are dependent on the examinee sample from which they are taken.• Scores are entirely test dependent.• No basis to predict the performance of examinees on an item.
  5. 5. Assumptions in IRT• Unidimensionality – Examinee performance is a single ability• Response → Dichotomous – The relationship of examinee performance on each item and the ability measured by the test is described as monotonically increasing.
  6. 6. • Monotonicity of item performance and ability is typified in an item characteristic curve (ICC).• Examinees with more ability have higher probabilities for giving correct answers to items than lower ability students (Hambleton, 1989).
  7. 7. • Mathematical model linking the observable dichotomously scored data (item performance) b a to the unobservable data (ability)c • Pi(θ) gives the probability of a correct response to item i as a function if ability (θ) • b is the probability of a b=item difficulty correct answer (1+c)/2 a=item discrimination c=psuedoguessing parameter
  8. 8. • Two-parameter model: c=0 • One-parametera model: c=0, a=1 b
  9. 9. • Three items showing different item difficulties (b)
  10. 10. • Different levels of item discrimination
  11. 11. Polychotomous IRT Models• Having more than 2 points in the responses (ex. 4 point scale)• Partial credit model• Graded response model• Nominal model• Rating scale model
  12. 12. Graded Response model for a 5-point scale
  13. 13. • In IRT measurement framework, ability estimates of an examinee obtained from a test that vary difficulty will be the same.• Because of the unchanging ability, measurement errors are smaller• True score is determined each test.• Item parameters are independent on the particular examinee sample used.• Measurement error is estimated at each ability level.
  14. 14. Test Characteristic Curve (TCC) • TCC: Sum of ICC that make up a test or assessment and can be used to predict scores of examinees at given ability levels. TCC(Ѳ)=∑Pi(Ѳ) • Links the true score to the underlying ability measures by the test. • TCC shift to the right of the ability scale=difficult items
  15. 15. Item Information Function • I(Ѳ), Contribution of particular items to the assessment of ability. • Items with higher discriminating power contribute more to measurement precision than items with lower discriminating power. • Items tend to make their best contribution to measurement precision around their b value.
  16. 16. Item Information Function
  17. 17. 1 2 2 1 2 30.8 1.50.6 4 1 10.40.2 0.5 3 4 0 0 –3 –2 –1 0 1 2 3 –3 –2 –1 0 1 2 3 Ability (θ) Ability (θ) Four item characteristic curves Item information for four test items Figure 6: Item characteristics curves and corresponding item information functions
  18. 18. their corresponding IFFTest Information Function• The sum of item information functions in a test.• Higher values of the a parameter increase the amount of information an item provides.• The lower the c parameter, the more information an item provides.•• The more information provided by an assessment at a particular level, the smaller the errors associated with ability estimation.
  19. 19. 21.5 1 0.5 0 0 3 Ability (θ) Figure 7: Test information function for a four–item test
  20. 20. Item Parameter Invariance • Item/test characteristic functions and item/test information functions are integral features of IRT.
  21. 21. Benefits of ItemResponse Models• Item statistics that are independent of the groups from which they were estimated.• Scores describing examinee proficiency or ability that are not dependent on test difficulty.• Test models that provide a basis for matching items or assessment tasks to ability levels.• Models that do not require strict parallel tests or assessments for assessing reliability.
  22. 22. Application of IRT onTest Development• Item Analysis – Determining sample invariant item parameters. – Utilizing goodness-of-fit criteria to detect items that do not fit the specified response model (χ2, analysis of residuals).
  23. 23. Application of IRT onTest Development• Item Selection – Assess the contribution of each item the test information function independent of other items.
  24. 24. – Using item information functions: • Describe the shape of the desired test information function vs. desired range abilities. • Select items with information functions that will fill up the hard to fill areas under the target information function • Calculate the test information function for the selected assessment material. • Continue selecting materials until the test information function approximates the target information function to a satisfactory degree.
  25. 25. • Item banking – Test developers can build an assessment to fit any desired test information function with items having sufficient properties. – Comparisons of items can be made across dissimilar samples.
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×