Test item analysis

1. LANGUAGE TESTING
ABD. HALIM

2. WHAT IS A TEST?
 Test is a measuring device for assessing the
achievement of the objectives in a training
system.
 It is a set of questions for which there is an
accepted set of correct answers.
 The characteristics of test may be knowledge,
ability, attitude, intelligence, etc.,.

3. PURPOSES OF TEST
• To assess the progress towards the goal or objectives
or the training.
• To compare the performance of a trainee with the
standard group.
• To improve trainee’s self understanding and
motivation.
• To diagnose the weakness of the trainees in certain
area.
• To promote grade or rate of the trainees.
• To assess the standard of instruction or curriculum.

4. CHARACTERISTICS OF A GOOD TEST
•VALIDITY (SAHIH)
•RELIABILITY (HANDAL, KONSISTEN)
•FAIRNESS (ADIL)

5. STEPS IN TEST CONSTRUCTION
• DETERMINE THE AIM OF TEST
• DETERMINE INDICATORS OF ACHIEVEMENT
• DISTRIBUTE THE TEST ITEMS BASED ON COMPETENCE, MATERIAL AND
TYPES OF TESTS.
• DEVELOP THE GRILLWORK OF TEST
• WRITE THE TEST ITEMS BASED ON THE GRILLWORK AND RULES
• VALIDATE THE TEST ITEMS THROUGH QUANTITATIVE WAY
• DEVELOP TEST SCORING
• DO TRY-OUT
• ANALYZE THROUGH QUANTITATIVE WAY
• IMPROVE THE TEST ITEMS

6. THE GRILLWORK OF TEST
NO STANDARD
COMPETENCE
BASIC
COMPETENCE
SMTR MATERIAL INDICATORS TYPE OF TEST NUMBR.
ITEMS
(1) (2) (3) (4) (5) (6) (7) (8)

7. GOOD INDICATORS
•USING APPROPRIATE OPERATIONAL VERBS
•USING ONE OPERATIONAL VERB FOR OBJECTIVE
TEST, AND MORE THAN ONE FOR DESCRIPTIVE
AND PERFORMANCE TESTS
•PREPARING DISTRUCTORS FOR OBJECTIVE
TESTS

8. TEST ITEM ANALYSIS
•QUALITATIVE
•QUANTITATIVE

9. ADVANTAGES OF TEST ITEM ANALYSIS
• HELPING THE TEST USERS EVALUATE THE TEST TO BE
USED
• HELPING THE TEST USERS CONSTRUCT INFORMAL OR
LOCAL TESTS
• ENCOURANGING TO WRITE EFFECTIVE TEST ITEMS
• HELPING TEACHERS IMPROVE TESTS IN CLASS
• IMPROVING TEST VALIDITY AND RELIABILITY

10. TWO WAYS OF QUALITATIVE ANALYSIS
•MODERATOR TECHNIQUE (DISCUSSION)
•PANEL TECHNIQUE (INDIVIDUAL WORK)

11. TWO WAYS OF QUANTITATIVE ANALYSIS OF
TEST ITEMS
•CLASSICAL WAY
•MODERN WAY

12. ASPECTS IN QUANTITATIVE ANALYSIS
•DIFFICULTY LEVEL
•DISCRIMINATING POWER
•OPTION DISTRIBUTION

13. 1. MEASURING THE DIFFICULTY LEVEL OF TEST
ITEMS
DIFFICULTY LEVEL (DL) =
NUMBER OF STUDT WITH CORRECT RESPONSES
TOTAL NUMBER OF STUDENTS

14. MEASURING DIFFICULTY LEVEL OF DESCRIPTIVE
TEST ITEMS
• MEAN = STUDENTS’ TOTAL SCORE ON ONE TEST ITEM
NUMBER OF STUDENTS
DIFFICULTY LEVEL = MEAN
MAXIMUM SCORE DITERMINED

15. CLASSIFICATION OF DIFFICULTY LEVEL
INDEX CATEGORIES
0.00 - 0.30
0.31 - 0.70
0.71 - 1.00
DIFFICULT
FAIR
EASY

16. TEST CASE FOR OBJECTIVE TEST
Find out the difficulty level of the test based on the following data
and interpret the result.
STUDENT SCORE STUDENT SCORE STUDENT SCORE
A 1 L 1 W 1
B 1 M 1 Z 1
C 0 N 1 A1 1
D 0 O 0 B1 0
E 1 P 0 C1 0
F 0 Q 0 D1 0
G 1 R 1 E1 0
H 1 S 1 F1 1
I 0 T 1 G1 1
Y 1 U 0 H1 0

17. TEST CASE FOR DESCRIPTIVE TEST
(Find out mean score and difficulty level and interpret the result)
STUDENT SCORE STUDENT SCORE STUDENT SCORE
A 1 O 3 C1 0
B 1 P 3 D1 0
C 2 Q 4 E1 1
D 3 R 4 F1 1
E 3 S 2 G1 2
F 2 T 2 H1 2
G 2 U 2 I1 3
H 4 V 1 J1 3
I 4 W 0 K1 4
J 3 X 0 L1 4
K 3 Y 1 M1 1

18. 2. DISCRIMINATING POWER OF TEST ITEMS
• THE ABILITY OF A TEST ITEM TO DISCRIMINATE
BETWEEN ONE WHO ACHIEVES A CERTAIN
LANGUAGE COMPETENCE AND ONE WHO FAILS.
• INDEX OF DISCRIMINATING POWER RANGING FROM
-1.00 TO +1.00
• THE HIGHER THE INDEX THE BETTER OR STRONGER
THE ITEM.

19. MEASURING DISCRIMINATING POWER OF
MULTIPLE CHOICE TEST ITEM
DP = CU – CL
⅟2 N
OR DP = 2 (CU – CL)
N
DP = DISCRIMINATING POWER
CU = TOTAL CORRECT SCORES OF UPPER GROUP
CL = TOTAL CORRECT SCORES OF LOWER GROUP
N = NUMBER OF SAMPLES

20. MEASURING DISCRIMINATING POWER OF A
DESCRIPTIVE TEST ITEM
• DP = MEAN OF UPPER GROUP – MEAN OF LOWER GROUP
MAXIMUM SCORE

21. THE CLASSIFICATION INDEX OF
DISCRIMINATING POWER
INDEX CLASSIFICATION
0.40 – 1.00
0.30 - 0.39
0.20 - 0.29
0.00 – 0.19
ACCEPTED
NEEDS MINOR CHANGES
REPAIRED
OMMITTED

22. TEST CASE : FIND OUT THE VALUE OF
DISCRIMINATING POWER OF A DESCRIPTIVE TEST
• 1. 90
• 2. 85
• 3. 84
• 4. 84
• 5. 60
• 6. 50
• 7. 50
• 8. 40
• 9. 30
• 10. 25

23. LIST OF STUDENTS’ SCORES OF TEST ITEM 5
STUDENTS WITH
CORRECT ANSWERS
TOTAL SCORES STUDENTS WITH
WRONG ANSWERS
TOTAL SCORES
1
2
3
4
5
6
7
8
9
10
11
12
19
18
18
16
16
16
15
13
13
13
12
12
14
15
16
17
18
19
20
21
22
23
24
25
17
16
15
14
14
12
12
12
12
12
11
11

24. • NUMBER OF CORRECT STUDENTS = 13
• NUMBER OF WRONG STUDENTS = 17
• TOTAL NUMBER OF STUDENTS = 30
• AVERAGE OF CORRECT ANSWERS = 192 : 13 = 14.7692
• AVERAGE OF WRONG ANSWERS = 200 : 17 = 11.7647
• AVERAGE OF TOTAL SCORES = (192+200) : 30 = 13.0667
• SD OF TOTAL SCORES = 3.0954
• TOTAL SCORES = 392
• 𝐫𝐩𝐛𝐢𝐬 =
𝑿𝒃− 𝑿𝒔
𝑺𝑫
𝒑𝒒

25. • 𝐫𝐩𝐛𝐢𝐬 =
𝟏𝟒,𝟕𝟔𝟗𝟐−𝟏𝟏,𝟕𝟔𝟒𝟕
𝟑,𝟎𝟗𝟓𝟒
𝟏𝟑
𝟑𝟎
+
𝟏𝟕
𝟑𝟎
• 𝐫𝐩𝐛𝐢𝐬 =
𝟑,𝟎𝟎𝟒𝟓
𝟑,𝟎𝟗𝟓𝟒
𝟎, 𝟒𝟑𝟑𝟑𝟑𝟑 𝟎, 𝟓𝟔𝟔𝟔𝟔𝟔
•= (0,9706338) (0,4955355)
•= 0,4809835
•= 0,48
• 𝐫𝐩𝐛𝐢𝐬 =
𝑿𝒃− 𝑿𝒔
𝑺𝑫
𝒑𝒒

26. TEST CASE: FIND DISCRIMINATING POWER USING
CORRELATION POINT BISERIAL (rpbis)
STUDENT WITH CORRECT
ANSWER
TOTAL SCORE STUDENTS WITH
INCORRECT ANSWER
TOTAL SCORES
1
2
3
4
5
6
7
8
9
10
11
12
13
14
19
19
19
17
17
16
16
15
14
13
12
12
12
12
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
17
16
15
14
13
13
12
12
12
11
11
11
9
9
8
7

27. TEST RELIABILITY
•THIS IS TO KNOW THE LEVEL OF PRECISION OF
TEST SCORES
•CONCISTENCY OF TEST SCORES
•RELIABILITY INDEX RANGING FROM 0 TO 1
•THE HIGHER THE COEFFICIENCE THE HIGHER
THE RELIABILITY

28. KUDER RICHARDSON
𝑲𝑹 − 𝟐𝟎 =
𝒌
𝒌 − 𝟏
𝟏 −
𝒑(𝟏 − 𝒑)
(𝑺𝑫) 𝟐
k = JUMLAH BUTIR SOAL
(SD)2 = VARIAN

29. MEASURING USING kr-20
STUDENTS TEST ITEMS
𝑺𝑪𝑶𝑹𝑬
𝑋 𝑋 − 𝑋 𝑋 − 𝑋2
1 2 3 4
A
B
C
D
E
F
1
1
0
0
1
1
0
1
0
0
1
1
0
0
1
0
0
1
0
0
1
0
1
1
1
2
2
0
3
4
2
2
2
2
2
2
-1
0
0
-2
-1
-2
1
0
0
4
1
4

30. (1-p) = 0.33 0.50 0.67 0.50
P (1-p) = 0.22 0.25 0.22 0.25
Ʃp (1-p) = 0.22 0.25 0.22 0.25 = 0.944
Number of students = 6
Total scores = 12
Variance = 𝑝(𝑋 − 𝑋)
2
/N
= 10:6
= 1.67
Deviation Standard = √1.67
= 1.29

31. • 𝐾𝑅 − 20 =
𝑘
𝑘−1
1 −
𝑝(1−𝑝)
(𝑆𝐷)2
• 𝐾𝑅 − 20 =
4
4−1
1 −
0.944
1.67
•0.58

32. THROUGH SPEARMAN-BROWN
NAME OF
STUDENTS
TEST ITEMS
𝑺𝑪𝑶𝑹𝑬
1 2 3 4
BACO
BECCE
SITTI
BORA
SAMPE
DULLA
1
1
0
0
1
1
0
1
0
0
1
1
0
0
1
0
0
1
0
0
1
0
1
1
1
2
2
0
3
4
p 0.67 0.50 0.33 0.50 12

33. NAME OF
STUDENTS
ODD
ITEMS
(1+3)
EVEN
ITEMS
(2+4)
Z SCORES FOR Z odd X Z even
ODD
ITEMS
EVEN
ITEMS
BACO
BECCE
SITTI
BORA
SAMPE
DULLA
1
1
1
0
1
2
0
1
1
0
2
2
0
0
0
-1.72
0
+1.72
-1.22
0
0
-1.22
+1.22
+1.22
0
0
0
2.10
0
2.10
P 0.67 0.50 0.33 0.50 12

34. • N = 6
• Mean = 1.0 1.0
• SD = 0.58 0.82
r12=
Ʃ𝒛 𝒐𝒅𝒅 𝒙 𝒛 𝒆𝒗𝒆𝒏
𝒏
r12 =
𝟒.𝟐
𝟔
= 0.70

35. Spearman Brown Reliability
=
𝟐𝒓𝟏. 𝟐
𝟏 + 𝒓𝟏. 𝟐
=
𝟐. (𝟎. 𝟕𝟎)
𝟏 + 𝟎. 𝟕𝟎
= 0.82

36. 3. ANSWER DISTRIBUTION
•THIS IS TO KNOW WHETHER THE OPTIONS
WORK, IT MIGHT BE SEEN THROUGH:
•SELECTED AT LEAST BY 5% OF TEST TAKERS
•SELECTED BY MANY OF STUDENTS WHO HAVE
NOT UNDERSTOOD THE TOPIC.