Reliability in Psychological Testing refers to the ability of the psychological test to give a consistent result. The presentation discusses ways to test reliability
9. Scoring Systems
How the test supposed to
be scored?
Is it standardized?
Standardized Test Manuals
do have Guidelines
HOW TO SCORE/COMPUTE
the score for test taker
10. SCORING SYSTEM
1. Someone took the Psych Test
2. Accomplish the questionnaire
3. Let the two rater compute the
score (based on the manual)
SAME COMPUTATION?
11. METHOD 3: INTER-SCORER
RELIABILITY
General Steps:
1. Look for at least two raters.
2. Teach Test Scoring System
3. Administer the Test to subjects
4. Let the raters score the test items/drawings
5. 5.Correlate
13. 5. Correlate
Rater 1
?
?
?
?
Rater 2
?
?
?
?
Drawings
Drawing 1
Drawing 2
Drawing 3
Drawing 4
+0.95
-0.80
+0.15
-0.15
Direct High Rel.
Inversely High Rel.
Direct Low Rel.
No Pattern
14. Direct Relationship- manifested by the positive sign,
It means that as one goes up, the other goes up too.
Inverse Relationship is manifested by negative sign,
It means that as one goes up, the other goes down.
16. Method 4: Internal Consistency
Checks reliability of individual test items.
METHODS:
A. Split-Half
B. KR-20
C. Cronbach’s Alpha
17. A. Split-Half Reliability
-Dividing the test items into 2 groups
Steps:
1. Randomly group test items into 2
(E.g. Fishbowl Method)
Why randomized?
To Equalize difficulty
18. Step 2: Administer each half to single
subject
First Half:
#s 1, 3, 5, 8
Second Half:
#s 2, 4, 6, 7
19. A. Split-Half Reliability
Step 3: Total Each Half
Data Entry
Subject # First Half
(1, 3, 5, 8)
Second Half
(2, 4, 6, 7)
1 High High
2 Low Low
3 High High
4 Low Low
23. B. KUDER RICHARDSON METHOD
(KR 20)
Use for OBJECTIVE TEST
It cannot be used with Personality Test
Main Question:
How consistently people get an item right or wrong
24. In a Nutshell
100 Subjects
Item 1= 1:1
Item 2=20:1 (The higher the test ability to discriminate
between high and low performer the More consistent
and reliable the test item is)
RATIO: RIGHT to WRONG answers per item
25. STATISTICA/SPSS INPUT
Students Test Item 1 Test Item 2 Test Item 3
1 Wrong Correct Wrong
2 Correct Correct Correct
3 Correct Wrong Wrong
KR Value (Right-
Wrong Ratio)
KR Value (Right-
Wrong Ratio)
KR Value (Right-
Wrong Ratio)
26. DISADVANTAGES: KUDER
RICHARDSON 20 METHOD (KR 20)
1. N/A when items has unequal difficulty
Assumption=all items SAME Difficulty
SOLUTION: KR21 (unequal Difficulty)
2. KR 20 does not work for non-objective test (i.e.
personality test)
27. C. Cronbach’s Alpha
Lee Cronbach’s was a student of Lewis Terman.
Cronbach’s Alpha can be used for both objective and
non-objective test (Likert type)
Average of all possible split half.
28. Cronbach’s Alpha: Use ALL possible HALF
TEST COMBINATIONS + Spearman Brown
First Half
#1
#3
#6
First Half
#4
#5
#6
Second Half
#1
#2
#8
Second Half
#2
#4
#5
x2
x2
29. Cronbach’s Alpha Item
Deleted
The Higher the CA the
better
Q1 .767 Most Reliable
Q2 .799
Q3 .802
Q4 .770
Q5 .823 Least Reliable
Q7 .774
30. QUESTION:
What will make an item unreliable?
Test TOO long (Why?)
Item is vaguely/unclearly written
31. Example of Vague Items (Psych
Achievement Test)
True or False
The UNCONSCIOUS always contains negative images
which represent a person’s past negative experience.
Answer: NEITHER
True: if according to Freud
False: if according to Jung
Therefore, you can modify or delete the item
32. 3 TEST OF RELIABILITY
TEST ADMINISTRATION
Test-Retest Method
Alternate Form
SCORING SYSTEM
Inter-rater reliability
TEST ITEMS
Split Half
Kuder Richardson
Cronbach’s