The document explains the concepts of reliability and validity in research measurement, detailing types of reliability such as test-retest, inter-rater, parallel forms, and internal consistency. It emphasizes how reliability measures consistency over time, across different observers, and among test items, while validity assesses the accuracy of what is being measured. Additionally, it discusses the differences between research-made, standardized, and modified-standardized instruments in the context of their validation and reliability.

1. CHAPTER 1 - LESSON 1
Reliability vs. Validity

2. Learning Objectives
Students should be able
to identify the concept of
reliability and validity in
the context of research
and measurement.
Students should be able
to differentiate between
various types of
reliability, such as test-
retest reliability, inter-
rater reliability, parallel
forms reliability, and
internal consistency.
Students should be
able to apply the
concepts of reliability
and validity to different
fields, such as
psychology, education,
medicine, and
business.
1 2 3

3. RELIABILITY
VALIDITY
&

4. If you use the same
research instrument
under the same
conditions, do you get
consistent results?
Reliability

5. It refers to the repeatability or consistency of
your measurement.
WHAT IS RELIABILITY?
Reliability
It also refers to whether research methods can
reproduce the same results multiple times. If
your research methods can produce consistent
results, then the methods are likely reliable and
not influenced by external factors.

6. EXAMPLES: Classroom Test
Reliability
Reliable Test
• If a student takes the test today and scores 85%,
and then takes the same test next week under
similar conditions and also scores around 85%,
the test is reliable. The scores are consistent over
time.
Unreliable Test
• If the same student scores 85% today but scores
50% next week without any change in their
knowledge or preparation, the test might be
unreliable. The inconsistency in scores suggests
that the test may not accurately reflect the
student's understanding.

7. How can we measure reliability?
Over Time:
Test-Retest Reliability
Let’s have a think about the different ways in which we might measure reliability:
Across different researchers :
Inter-Rater Reliability
Across different tool:
Parallel Forms Reliability
Across different items:
Internal Consistency Reliability

8. Do you get the
same results
when you repeat
the
measurement?
When doing an experimental,
Test-Retest
Reliability
(Over Time)

9. It involves assessing the consistency of
measurements over time. It’s like taking
the same measurement or test twice –
once and then again after a certain period.
If the results align closely, it indicates that
the measurement is reliable over time.
Think of it as capturing the essence of
stability.
WHAT IS TEST-RETEST RELIABILITY?
Test-Retest
Reliability
(Over Time)

10. Do you get the same
results when
different people
conduct the same
measurement?
When you conduct your experimental study,
Inter-Rater
Reliability
(Across different
researchers)

11. This type of reliability assesses the level of
agreement between different observers
when evaluating the same phenomenon.
This relates to consistency across people. If
someone else repeats the measurement
(e.g., someone else rates my intelligence)
will they produce the same answer?
WHAT IS INTER-RATER RELIABILITY?
Inter-Rater
Reliability
(Across different
researchers)

12. It measures the correlation between two
equivalent versions of a test. You use it when you
have two different assessment tools or sets of
questions designed to measure the same thing.
WHAT IS PARALLEL FORMS
RELIABILITY?
Parallel Forms
Reliability
(Across different tool)
Example:
If the same students take two different versions
of a reading comprehension test, they should
get similar results in both tests.

13. When you conduct your study,
Internal Consistency
Reliability
(Across Items)
Do you get the same
results from different
parts of a test that are
designed to measure the
same thing?

14. It assesses the correlation between
multiple items in a test that are intended to
measure the same construct or topic.
WHAT IS INTERNAL CONSISTENCY
RELIABILITY?
Internal Consistency
Reliability
(Across items)
Example:
If you are designing a test on geometry,
then all questions on the test should be
about geometry.

15. Cronbach’s Alpha
when designing and testing a
new survey or assessment
instrument
helps them evaluate the quality of
the tool during the design phase
before deploying it fully.
before including various questions
in a scale, they must be sure that all
items reliably measure the same
construct
to ensure that items agree, but they
need to use it with other analyses that
evaluate whether the items measure
the correct characteristic.

16. Does the research
instrument measure
what it’s supposed to
measure?
Validity

17. Validity is an evaluation of how accurate the
study is. It describes the extent to which the
study actually measures what it intends to
measure.
WHAT IS VALIDITY?
Validity
Example:
A valid driving test should include a practical
driving component and not just a theoretical test
of the rules of driving.

18. Some Example of Validity Testing
Content Validity: Used when you want to know whether a sample of items truly
reflects an entire universe of possible items in a certain topic.
Example: “I want to design a test to measure my students’ ability in statistics”
Q1: Calculate the mean of the sample.
Q2: Calculate the mean of the other sample.
Q3: Calculate the mean of the other sample.
Q4: Calculate the mean of the other sample.
Q5: Calculate the mean of the other sample.
Q1: Calculate the mean.
Q2: Calculate the standard deviation.
Q3: Calculate the correlation.
Q4: Calculate a t-test.
Q5: Calculate an ANOVA.

19. Some Example of Validity Testing
Criterion Validity: Used when you want to assess
whether a test reflects a set of abilities in a
current or future setting.
a.Concurrent Validity: Checks how well a test
correlates with an existing measure at the same
time.
b.Predictive Validity: Checks how well a test predicts
future performance or behavior.

20. Some Example of Validity Testing
Construct Validity: Does the
instrument measure the
correct attribute?

21. Reliability vs. Validity

22. Research
Instruments
• Research-Made
• Standardized
• Modified Standardized

23. Research-Made Instrument:
How research-made instruments
related to Validation and Reliability
Research-Made
A researcher-made instrument refers to a
measurement tool that is created by the researcher
specifically for their study. These instruments are
tailored to the unique characteristics and objectives
of the research. They are designed to gather data
relevant to the specific research questions, variables,
or phenomena under investigation.

24. Research-Made Instrument:
How research-made instruments
related to Validation and Reliability
Research-Made
Research-made instruments offer flexibility, allowing
researchers to customize questions and response
formats based on the study’s focus. However, they
need careful development to ensure validity and
reliability.
VALIDATION: YES RELIABILITY: YES

25. Research-Made Instrument:
How standardized instruments related
to Validation and Reliability
STANDARDIZED
INSTRUMENT The standardized instruments is a pre-existing
measurement tool that has been developed and
validated by researchers, often through rigorous
testing and standardized procedures. These
instruments have established reliability and validity
and are designed to be administered consistency
across different settings and populations

26. Research-Made Instrument:
How standardized instruments related
to Validation and Reliability
STANDARDIZED
INSTRUMENT Standardized instruments enable researchers to
make comparisons across studies and populations,
enhancing the generalizability of findings. Examples
include established psychological assessments,
educational Test, or health surveys.
VALIDATION: NO RELIABILITY: NO

27. Research-Made Instrument:
How modified-standardized instruments
related to Validation and Reliability
MODIFIED-
STANDARDIZED
INSTRUMENT A modified standardized instruments is a pre-
existing measurement tool that has been altered or
adapted better suit the specific context or
population of a research study. Researchers may
make modifications to ensure cultural relevance,
linguistics appropriateness, or to address specific
research objectives.

28. Research-Made Instrument:
How modified-standardized instruments
related to Validation and Reliability
MODIFIED-
STANDARDIZED
INSTRUMENT While modification can enhance the instrument’s
applicability to the study, researchers must be
cautious to maintain the validity and reliability of the
original instruments after any alterations.
VALIDATION: YES RELIABILITY: YES

29. Remember
A valid instrument is a reliable
instrument, while a reliable
instrument is not always a
valid instrument.

30. Thank you for
listening!