Understanding Reliability and Validity.pptx

1. Understanding Reliability
and Validity
•Cyprus International University
•Student Name: Mohammad Faisal
•Student No.: 21814715
•Course Code: ELT506
•Course Name: Language Testing
•Year: 2018 - 2019

2. What does ‘Reliability’ mean?
• Reliability is the degree to which an assessment tool produces stable and
consistent results.

3. Types of Reliability
• Test-Retest Reliability
• Parallel-Forms Reliability
• Internal Consistency Reliability
A. Split-Half Reliability
B. Average Inter-Item Correlation

4. • Test-Retest Reliability
It is a measure of reliability obtained by administering the same test twice over a period of
time to a group of individuals, The scores from Time 1 and Time 2 can then be correlated in
order to evaluate the test for stability over time.
Test-Retest Correlation Between Two Sets of Scores of Several College Students, Given Two Times a Week Apart

5. • Parallel-Forms Reliability
It is a measure of reliability obtained by administering different versions of an
assessment tool (both versions must contain items that probe the same
construct, skill, knowledge base, etc.) to the same group of individuals. The
scores from the two versions can then be correlated in order to evaluate the
consistency of results across alternate versions.
Example: If you wanted to evaluate the reliability of a critical thinking
assessment, you might create a large set of items that all pertain to critical
thinking and then randomly split the questions up into two sets, which would
represent the parallel forms.

6. • Internal Consistency Reliability
It is a measure of reliability used to evaluate the degree to which different test items that probe
the same construct produce similar results.
A measure of reliability obtained by administering different versions of an assessment tool
(both versions must contain items that probe the same construct, skill, knowledge base, etc.) to
the same group of individuals. The scores from the two versions can then be correlated in
order to evaluate the consistency of results across alternate versions.
Split-Half Correlation Between Several College Students’ Scores on the Even-Numbered Items and
Their Scores on the Odd-Numbered Items

7. Split-half reliability
It is subtype of internal consistency reliability.
Sometimes referred to as internal
consistency
Indicates that subjects’ scores on some
trials consistently match their scores
on other trials

8. B. Average inter-item correlation
It is another subtype of internal consistency reliability.
It is obtained by taking all of the items on a test that probe the same construct
(e.g., reading comprehension), determining the correlation coefficient for
each pair of items, and finally taking the average of all of these correlation
coefficients. This final step gets the average

9. To analyze the factors which affect the reliability based on scores,
let us see the factors which can affect the scores of test papers.
Lyle Bachman (1997) considers that the scores of test papers are
determined by the following three factors: the language ability
of candidates, testing methods and personal features.

10. The language ability of candidates
The scores will be decided by the language ability of the
candidates, which is a basic factor. If candidates take the same
language test over and over again and their performance is not
affected, we may estimate their language ability through their
scores.

11. Testing methods
Testing methods mainly refer to testing environment, testing
arrangement, or time division, which can also influence the testing
scores.
For example, if we use direct testing, we can get the marks on the spot,
but the candidates may be affected psychologically by the
circumstantial factors and can not show their real ability.
If we use indirect testing, the candidates may get higher scores because
of the relaxation.

12. Personal features
Personal features refer to some personal characteristics which
are irrelevant to the language ability of candidates.
like the teacher’s point of view or the students’ attitude. Even
though the teacher is objective, the candidates themselves will
be influenced by their own factors psychologically and
physically.

13. How to increase the reliability of test papers?
. Do not allow the candidates too much freedom.
. Do not write ambiguous items in test papers.
. The scorers should get some training.
. Identify candidates by number instead of names.

14. What does ‘Validity’ mean?
• Validity is the extent to which a test and other assessments measure what
they were intended to measure.

15. Test validity
Validity is the most important issue in selecting a test.
Validity refers to what characteristic the test measures and
how well the test measures that characteristic.
Validity tells you if the characteristic being measured by a test
is related to job qualifications and requirements.

16. Types of Validity
• Face Validity
• Content Validity
• Criterion-Related Validity
• Construct Validity

17. Understanding the differences between reliability and validity.
Validity will tell you how good a test is for a particular situation.
Reliability will tell you how trustworthy a score on that test will be.
You cannot draw valid conclusions from a test score unless you are sure that the
test is reliable. Even when a test is reliable, it may not be valid. You should be
careful that any test you select is both reliable and valid for your situation.

19. The End