This document discusses reliability and validity in testing. It defines reliability as the consistency of test measures and discusses various methods to assess reliability including test-retest, equivalent forms, internal consistency using split-half and alpha coefficient methods. The document also defines validity as the appropriateness of test inferences and discusses three types of validity evidence: content, criterion, and construct validity. It further explains threats to internal validity such as subject characteristics, location effects, and data collector bias that can influence test outcomes.

1. RELIABILITY
AAINAA AMIRA BINTI AMIR
NURUL ATIQAH BINTI RUSLI
2018671044
2018483472

2. RELIABILITY
Reliability is the degree to which a test consistently
measures whatever it measures.
1. Stability of measures
✔ Test-Retest Method
✔ Equivalent – Forms Method
2. Internal-Consistency Method
✔ Split-half Procedure
✔ Kuder-Richardson Approaches
✔ Alpha Coefficient

3. TEST-RETEST METHOD
► Test-retest reliability is the degree
to which scores are consistent over
time. It indicates score variation that
occurs from testing session to
testing session as a result of errors
of measurement.

4. EQUIVALENT – FORMS
METHOD
► When the equivalent-forms method
is used, two different but equivalent
(also called alternate or parallel )
forms of an instrument are
administered to the same group of
individuals during the same time
period.

5. SPLIT-HALF PROCEDURE
► Especially appropriate when the test is
very long. The most commonly used
method to split the test into two is using
the odd-even strategy.

6. KUDER-RICHARDSON
APPROACHES
► K = number of items on the test
► M = mean of the set of test scores
► SD = standard deviation of the set of
test scores

7. ALPHA COEFFICIENT

8. VALIDITY
(2019728445)
NORFARHANAH BINT AHMAD
NUR FADZLIN BT ABDUL AZIZ
(2019791145)

9. Valid
instrument
The Importance of Validity
Valid
information
Valid
research

10. Validity
► validity has been defined as referring
to the appropriateness, correctness,
meaningfulness, and usefulness of the
specific inferences researchers make based
on the data they collect.
► Validation is the process of collecting and
analyzing evidence to support such
inferences.

11. ► Content-related evidence of validity
► Criterion-related evidence of validity
► Construct-related evidence of
validity
Types on evidences

12. Content-related evidence of
validity
► Refers to the content and format of the
instrument.
► How appropriate is the content?
► How comprehensive?
► Does it logically get at the intended
variable?
► Is the format appropriate? (size, font,
etc)
► The content and format must be
consistent with the definition of the
variable and the sample of subjects to
be measured.

13. ► refers to the relationship between scores
obtained using the instrument and scores
obtained using one or more other instruments or
measures (often called a criterion).
► Criterion is a second test or other assessment
procedure presumed to measure the same
variable.
Criterion-related evidence of
validity

14. ► There are two forms of criterion-related validity—
predictive and concurrent.
► To obtain evidence of predictive validity,
researchers allow a time interval to elapse
between administration of the instrument and
obtaining the criterion scores.
► when instrument data and criterion data are
gathered at nearly the same time, and the
results are compared, this is an attempt by
researchers to obtain evidence of concurrent
validity.
► A key index in both forms of criterion-related
validity is the correlation coefficient (the degree
of relationship that exists between the scores
individuals obtain on two instruments .

15. ► When a correlation coefficient is used to
describe the relationship between a set of
scores obtained by the same group of
individuals on a particular instrument and
their scores on some criterion measure, it is
called a validity coefficient.
► An expectancy table is nothing more than a
two-way chart, with the predictor categories
listed down the left-hand side of the chart
and the criterion categories listed
horizontally along the top of the chart.

16. ► Construct-related evidence of validity refers
to the nature of the psychological construct
or characteristic being measured by the
instrument.
► How well does a measure of the construct
explain differences in the behavior of
individuals or their performance on certain
tasks?
Construct-related evidence of
validity

17. 3 steps obtaining
construct-related evidence
of validity:
1. the variable being measured is clearly
defined
2. hypotheses, based on a theory underlying
the variable, are formed about how people
who possess a lot versus a little of the
variable will behave in a particular situation
3. The hypotheses are tested both logically
and empirically

18. Internal validity
► When a study lacks internal validity, one or
more alternative hypotheses exist to explain
the outcomes.
► These alternative hypotheses are referred to
by researchers as threats to internal
validity.
► When a study has internal validity, it means
that any relationship observed between two
or more variables is unambiguous, rather
than being due to something else.

19. THREATS TO INTERNAL
VALIDITY

20. Subject Characteristics
► The selection of people for a study may result
in the individuals or groups differing (i.e., the
characteristics of the subjects may differ)
from one another in unintended ways that are
related to the variables to be studied.

21. LOSS OF SUBJECTS
(MORTALITY)
► No matter how carefully the
subjects of a student (the sample)
are selected, it is common to lose
some of them as the study
progresses.
► This is known as mortality. Such a
loss of subjects may affect the
outcomes of a study.

22. Location
► The particular locations in which data are
collected, or in which an intervention is carried
out, may create alternative explanations for any
results that are obtained.

23. Instruments
► The way in which instruments are used
may also constitute a threat to the
internal validity of a study.
Possible instrumentation threats
include changes in the instrument,
characteristics of the data collector(s)
and/or bias on the part of the data collectors.

24. Data Collector Characteristics.
► The use of a pretest in intervention
studies sometimes may create a
“practice effect” that can affect the
results of a study.
► A pretest can also sometimes affect
the way subjects respond to an
intervention.
► Gender, age, ethnicity, language
patterns, or other characteristics of
the individuals who collect the data in
a study may affect the nature of the
data they obtain.
► If these characteristics are related to
the variables being investigated, they
may offer an alternative explanation
for whatever findings appear.

25. ► There is also the possibility that the data collector(s)
and/or scorer(s) may unconsciously distort the data in
such a way as to make
certain outcomes (such as support for the hypothesis)
more likely.
► The two principal techniques for handling data collector
bias:
► to standardize all procedures, it requires some sort
of training of the data collectors
► to ensure that the data collectors lack the
information they would need to distort results —also
known as “planned ignorance”.
Data Collector Characteristics
– data collector bias

26. Testing threat
► data are collected over a period
of time, it is common to test
subjects at the beginning of the
intervention(s).
► If substantial improvement is
found in posttest (compared to
pretest)
scores, the researcher may
conclude that this improvement is
due to the intervention.
► It involves pre-test & post test
data collection.

27. History threat
► On occasion, one or more unanticipated and
unplanned for events may occur during the
course of a study that can affect the responses
of subjects. This is known as a history threat.

28. Maturation threat
► Sometimes change during an intervention
study may be due more to factors associated
with the passing of time than to the
intervention itself. This is known as a
maturation threat.

29. Subject attitude threat
► The attitude of subjects toward
a study (and their participation
in it) can create a threat to
internal validity.
► When subjects are given
increased attention and
recognition because they are
participating in a study, their
responses may be affected.
This is known as the Hawthorne
effect.

30. Regression
► Whenever a group is selected
because of unusually high or low
performance on a pretest, it will,
on average, score closer to the
mean on subsequent testing,
regardless of what transpires in
the meantime.

31. CONTROLLING THREATS TO
INTERNAL VALIDITY
► Whenever an experimental group is treated in
ways that are unintended and not a necessary
part of the method being studied, an
implementation threat can occur.
► 4 alternatives that can control internal validity:
1. standardizing the conditions under which the
study occurs
2. obtaining and using more information on the
subjects of the study
3. obtaining and using more information on the
details of the study
4. choosing an appropriate design.