2. What is item analysis?
• Judging the quality of
test items by
examining the
students’ responses
• Competent vs. less
competent students?
• Difficulty of items?
3. How is item analysis done?
Administer the
test
Check the
students’
responses to
separate items
Check the total
scores
4. Tasks of item analysis
1st task: Item discrimination
• Sort the students who
know the topic well from
those who do not
• Correlate performance on
a single test item with
total test performance
• (+) Correlation better
discrimination
2nd task: Item difficulty
5. Electronic item analysis
Student Item 1
Score
Total
Score / 30
A 1 25
B 1 19
C 1 18
D 0 16
E 1 12
F 0 10
• Average total score
(correct Item 1) = 18.5
• Average total score
(incorrect Item 1) = 13
• Computed correlation
coefficient = 0.53
• Item is to some extent
related to the total score
Correct = 1
Incorrect = 0
6. Electronic item analysis
r = correlation of an option with the total score
p = percentage of students who chose that option (n = 65)
• Correct options should show positive correlations; distractors
should show negative correlations
Item 1: r = 0.25 low correlation
Item 2: r = 0.49 fairly good correlation
Item 3: r = 0.34 modest correlation
• r ≤ 0.15 course content is not being assessed well
eliminate the item (OR revise)
Item A B C
1 r = -0.27
p = 13.89
r = 0.25
p = 50.00
r = -0.06
p = 36.11
2 r = -0.46
p = 5.56
r = 0.49
p = 88.86
r = -0.22
p = 5.56
3 r = -0.30
p = 16.67
r = -0.13
p = 27.78
r = 0.34
p = 55.56
7. Electronic item analysis
r = correlation of an option with the total score
p = percentage of students who chose that option (n = 65)
• Standard error (SE) = 1 / √ (number of students – 1)
= 0.12
• Any r > 2(SE) will be accepted as other than a chance
relationship between the item and the total score
Item 1: r = 0.25 > 0.24 [2(SE)] very marginal but acceptable
Item A B C
1 r = -0.27
p = 13.89
r = 0.25
p = 50.00
r = -0.06
p = 36.11
2 r = -0.46
p = 5.56
r = 0.49
p = 88.86
r = -0.22
p = 5.56
3 r = -0.30
p = 16.67
r = -0.13
p = 27.78
r = 0.34
p = 55.56
8. Item analysis by hand
• Step 1: Arrange the students’ papers according to their
test scores (highest to lowest).
• Step 2: Divide these into “high scorers” vs. “low scorers”.
• Step 3: Tabulate the number of students who chose each
option in both groups.
• Step 4: Compute for the discrimination index.
Item 1 A B C D Total
High scorers 2 4 0 16 22
Low scorers 12 7 0 4 23
9. Item analysis by hand
• Step 4: Compute for the discrimination index (DI).
DI = (NumHigh – NumLow)
Number of students in larger group
= (16 – 4) / 23 = 0.52
* Ranges from 0 – 1.00
* Can also be negative (for distractors)
Item 1 A B C D Total
High scorers 2 4 0 16 22
Low scorers 12 7 0 4 23
10. Item analysis by hand
Alternative: Straight difference method
• Steps 1 – 3: Same
• Step 4: Compute for NumHigh – NumLow.
• If ≥ 0.10(n) adequate (used across all items)
16 – 4 = 12
0.10(45) = 4.5
Item 1 A B C D Total
High scorers 2 4 0 16 22
Low scorers 12 7 0 4 23
11. Analysis of distractors
• Distractor C was not chosen by any student 3-option
item (0.33 instead of 0.25 chance level of guessing the
item correctly)
• Good item – each distractor will be chosen more often
by the low scorers
Item 1 A B C D
High scorers 4 13 0 3
Low scorers 7 9 0 4
p 27.5 55.0 0.0 18.5
12. Tasks of item analysis
2nd task: Item difficulty
• Difficulty index/facility
index = proportion of
students who get an item
correctly
• Step 1: Award a score to
each student.
• Step 2: Arrange the scored
tests from highest to lowest.
• Step 3: Identify the upper
and lower 27%.
• Step 4: Count the response
counts in each group.
Item 1 A B C D Total
High
scorers
2 4 0 16 22
Low
scorers
11 7 0 4 22
13. Tasks of item analysis
2nd task: Item difficulty
• Difficulty index/facility
index = proportion of
students who get an item
correctly
• Step 5: Calculate the
difficulty index.
= H + L
N
H = no. of students in the high group with
a correct answer
L = no. of students in the low group with a
correct answer
N = total no. of students
Item 1 A B C D Total
High
scorers
2 4 0 16 22
Low
scorers
11 7 0 4 22
14. Tasks of item analysis
2nd task: Item difficulty
• Difficulty index/facility
index = proportion of
students who get an item
correctly
• Ranges from 0 – 1.00
• Best ≈ 0.50 (0.30 – 0.70)
• Larger index easier
item; smaller index
more difficult item
DI = (16 + 4) / 80
= 0.25
• Criteria:
• ≥0.35 = excellent question
• 0.25 – 0.34 = good question
• 0.15 – 0.24 = marginal
question revise
• <0.15 = poor question
discard
Item 1 A B C D Total
High
scorers
2 4 0 16 22
Low
scorers
11 7 0 4 22
15. Item analysis for essay tests
• Step 1: Identify the upper and lower 25% of the students.
• Step 2: Compute for the following:
Disc. = (Sum of scores for highs – sum of score for lows)
N x (max. possible score on item)
Diff. = (Sum of scores for highs + sum of score for lows)
2N x (max. possible score on item)
N = 25% of the number tested
Item Score
High Group Low Group
No. of Students No. of Students
x Score
No. of Students No. of Students
x Score
10 9 90 1 10
8 6 48 0 0
6 2 12 4 24
4 3 12 7 28
2 0 0 8 16
Total 20 162 20 78
16. Item analysis for essay tests
• Step 2: Compute for the following:
Disc. = (Sum of scores for highs – sum of score for lows)
N x (max. possible score on item)
= (162 – 78) / [(0.25 x 80) x 10] = 0.42
Satisfactory discrimination
Item Score
High Group Low Group
No. of Students No. of Students
x Score
No. of Students No. of Students
x Score
10 9 90 1 10
8 6 48 0 0
6 2 12 4 24
4 3 12 7 28
2 0 0 8 16
Total 20 162 20 78
17. Item analysis for essay tests
• Step 2: Compute for the following:
Diff. = (Sum of scores for highs + sum of score for lows)
2N x (max. possible score on item)
= (162 + 78) / [(2 x 0.25 x 80) x 10] = 0.60
Satisfactory difficulty
Item Score
High Group Low Group
No. of Students No. of Students
x Score
No. of Students No. of Students
x Score
10 9 90 1 10
8 6 48 0 0
6 2 12 4 24
4 3 12 7 28
2 0 0 8 16
Total 20 162 20 78
18. Item response theory and item analysis
• Calculates the odds of getting an item right converts
this number to a natural logarithm
• Allows faculty to equate the item difficulty scale on one
test to the scale of another test and across different
student groups
• Useful system for building item banks
