Reliability in Psychological Testing refers to the ability of the psychological test to give a consistent result. The presentation discusses ways to test reliability

1. RELIABILITY
IN PSYCHOLOGICAL TEST
Presented by: SANDRA ARENILLO
MP-CP

2. Psychometric Property:
RELIABILITY
First Try: 350lbs
Second Try: 125lbs.
Third Try: 150 lbs.
IS THE RESULT
RELIABLE?

3. Psychometric Property:
RELIABILITY
Personality Test
First try: High Neuroticism
Second Try: Low Neuroticism
Third Try: Moderate Neuroticism
Is the Test results reliable?

4. What is a reliable Test?
Reliability is the ability of
the test to give consistent result.

5. Checking Reliability
Testing Procedure
Scoring System
Test Item

6. Considerations in Testing
Procedure
1. Environment
2. Delivery of Instructions
3. Behavior of the Examiner

7. Methods in Testing Reliability of
The Testing Procedure
Method 1: Test-Retest Method
Method 2: Alternate Form Method

8. Checking Reliability
Testing Procedure
Scoring System
Test Item

9. Scoring Systems
How the test supposed to
be scored?
Is it standardized?
Standardized Test Manuals
do have Guidelines
HOW TO SCORE/COMPUTE
the score for test taker

10. SCORING SYSTEM
1. Someone took the Psych Test
2. Accomplish the questionnaire
3. Let the two rater compute the
score (based on the manual)
SAME COMPUTATION?

11. METHOD 3: INTER-SCORER
RELIABILITY
General Steps:
1. Look for at least two raters.
2. Teach Test Scoring System
3. Administer the Test to subjects
4. Let the raters score the test items/drawings
5. 5.Correlate

12. +
Direction of
the
relationship
0.95 Degree of
Correlation

13. 5. Correlate
Rater 1
?
?
?
?
Rater 2
?
?
?
?
Drawings
Drawing 1
Drawing 2
Drawing 3
Drawing 4
+0.95
-0.80
+0.15
-0.15
Direct High Rel.
Inversely High Rel.
Direct Low Rel.
No Pattern

14. Direct Relationship- manifested by the positive sign,
It means that as one goes up, the other goes up too.
Inverse Relationship is manifested by negative sign,
It means that as one goes up, the other goes down.

15. Checking Reliability
Testing Procedure
Scoring System
Test Item

16. Method 4: Internal Consistency
Checks reliability of individual test items.
METHODS:
A. Split-Half
B. KR-20
C. Cronbach’s Alpha

17. A. Split-Half Reliability
-Dividing the test items into 2 groups
Steps:
1. Randomly group test items into 2
(E.g. Fishbowl Method)
Why randomized?
To Equalize difficulty

18. Step 2: Administer each half to single
subject
First Half:
#s 1, 3, 5, 8
Second Half:
#s 2, 4, 6, 7

19. A. Split-Half Reliability
Step 3: Total Each Half
Data Entry
Subject # First Half
(1, 3, 5, 8)
Second Half
(2, 4, 6, 7)
1 High High
2 Low Low
3 High High
4 Low Low

20. Advantage of Split Half:
Time Efficient! (One Sitting)

21. Disadvantage of Split Half
1. Not Applicable in heterogeneous test
2. Reliability only bases on 50% of the test

22. Spearman Brown
Estimates reliability of half test if it becomes a whole
First Half:
#s 1, 3, 5, 8
Second Half:
#s 2, 4, 6, 7
X 2

23. B. KUDER RICHARDSON METHOD
(KR 20)
Use for OBJECTIVE TEST
It cannot be used with Personality Test
Main Question:
How consistently people get an item right or wrong

24. In a Nutshell
100 Subjects
Item 1= 1:1
Item 2=20:1 (The higher the test ability to discriminate
between high and low performer the More consistent
and reliable the test item is)
RATIO: RIGHT to WRONG answers per item

25. STATISTICA/SPSS INPUT
Students Test Item 1 Test Item 2 Test Item 3
1 Wrong Correct Wrong
2 Correct Correct Correct
3 Correct Wrong Wrong
KR Value (Right-
Wrong Ratio)
KR Value (Right-
Wrong Ratio)
KR Value (Right-
Wrong Ratio)

26. DISADVANTAGES: KUDER
RICHARDSON 20 METHOD (KR 20)
1. N/A when items has unequal difficulty
Assumption=all items SAME Difficulty
SOLUTION: KR21 (unequal Difficulty)
2. KR 20 does not work for non-objective test (i.e.
personality test)

27. C. Cronbach’s Alpha
Lee Cronbach’s was a student of Lewis Terman.
Cronbach’s Alpha can be used for both objective and
non-objective test (Likert type)
Average of all possible split half.

28. Cronbach’s Alpha: Use ALL possible HALF
TEST COMBINATIONS + Spearman Brown
First Half
#1
#3
#6
First Half
#4
#5
#6
Second Half
#1
#2
#8
Second Half
#2
#4
#5
x2
x2

29. Cronbach’s Alpha Item
Deleted
The Higher the CA the
better
Q1 .767 Most Reliable
Q2 .799
Q3 .802
Q4 .770
Q5 .823 Least Reliable
Q7 .774

30. QUESTION:
What will make an item unreliable?
 Test TOO long (Why?)
 Item is vaguely/unclearly written

31. Example of Vague Items (Psych
Achievement Test)
True or False
The UNCONSCIOUS always contains negative images
which represent a person’s past negative experience.
Answer: NEITHER
True: if according to Freud
False: if according to Jung
Therefore, you can modify or delete the item

32. 3 TEST OF RELIABILITY
TEST ADMINISTRATION
Test-Retest Method
Alternate Form
SCORING SYSTEM
Inter-rater reliability
TEST ITEMS
Split Half
Kuder Richardson
Cronbach’s