VALIDITY, RELIABILTIY AND ALIGNMENT TO DETERMINE THE EFFECTIVENESS OF ASSESSMENT

1. VALIDITY, RELIABILTIY
AND ALIGNMENT TO
DETERMINE THE
EFFECTIVENESS OF
ASSESSMENT

2. ASSESSMENT
VALIDITY RELIABILITY
STRATEGIES STANDARDS
ALIGNMENT

3. ASSESSMENT
Assessment is the process of gathering
and discussing information from multiple
and diverse sources in order to develop a
deep understanding of what students
know, understand, and can do with their
knowledge as a result of their educational
experiences. (Huba and Freed,2000)

4. VALIDITY
- refers to what the assessment is actually
testing.
RELIABILITY
- refers to the consistency of the
assessment.
ALIGNMENT
- refers to the connection between learning
objectives, learning activities and
assessment

5. A. VALIDITY
Definitions:
Validity - refers to what the assessment is
actually testing
- refers to how accurately a conclusion,
measurement or concept corresponds to what is
being tested
- defined as the extent to which an
assessment accurately measures what it is
intended to measure.

6. Factors Affecting Validity:
1. Student’s reading ability - Educators should
ensure that an assessment is at the correct
reading level of the students.
2. Student self-efficacy - If students have low
self-efficacy or beliefs about their abilities in
the particular area they are being tested in,
they will typically perform lower.
3. Student test anxiety level - Students with high
test anxiety will underperform due to
emotional and physiological factors

7.  Types of Validity:
1. Face Validity - ascertains that the measure appears to be
assessing the intended construct under study.
2. Construct Validity - is used to ensure that the measure is
actually measure what it is intended to measure and not
other variables
3. Criterion-Related Validity - is used to predict future or
current performance - it correlates test results with another
criterion of interest.
4. Formative Validity - when applied to outcomes assessment it
is used to assess how well a measure is able to provide
information to help improve the program under study.
5. Sampling Validity - ensures that the measure covers the
broad range of areas within the concept under study

8. Kinds of Validity Claim:
1. Predictive evidence - form of construct validation
that examines whether performance on an
assessment is strongly related to real-world success
in the domain that the assessment is meant to
reflect.
2. Consequential evidence - refers to the kinds of
consequences an assessment and its uses have for
learners and for instruction.

9. Why is validity necessary?
While reliability is necessary, it alone is not sufficient. For
a test to be reliable, it also needs to be valid
Ways on how to improve validity:
1. Make sure your goals and objectives are clearly defined
and operationalized.
2.Match your assessment measure to your goals and
objectives.
3.Get students involved; have the students look over the
assessment for troublesome wording, or other
difficulties.
4. If possible, compare your measure with other
measures, or data that may be available.

10. B. RELIABILITY
Definitions:
Reliability - refers to the consistency of the
assessment
- the degree to which an assessment
tool produces stable and consistent results

11. Factors Affecting Reliability:
1. The length of the assessment
2. The suitability of the questions or tasks for
the students being assessed.
3. The phrasing and terminology of the
questions.
4. The consistency in test administration
5. The design of the marking schedule and
moderation of marking procedures.
6. The readiness of students for the assessment

12. Types of Reliability:
1. Test-retest reliability - is a measure of reliability
obtained by administering the same test twice over
a period of time to a group of individuals.
2. Parallel forms reliability - is a measure of reliability
obtained by administering different versions of an
assessment tool to the same group of individuals
3. Inter-rater reliability - is a measure of reliability
used to assess the degree to which different judges
or raters agree in their assessment decisions

13. 4. Internal consistency reliability - is a measure of
reliability used to evaluate the degree to which different
test items that probe the same construct produce similar
results.
a.) Average inter-item correlation - It is obtained by
taking all of the items on a test that probe the same
construct determining the correlation coefficient for each
pair of items, and finally taking the average of all of these
correlation coefficients
b.) Split-half reliability – The process of obtaining
this begun by “splitting in half” all items of a test that are
intended to probe the same area of knowledge in order
to form two “sets” of items.

14. C. ALIGNMENT
Definition: It is the connection between
learning objectives, learning activities and
assessment
OBJECTIVES ACTIVITIES ASSESSMENT

15.  3 Components of Alignment:
1. Objectives - What do I want students to know
how to do when they leave this course?
2. Activities - What kinds of activities in and out
of class will reinforce my learning objectives
and prepare students for assessments?
3. Assessment - What kinds of tasks will reveal
whether students have achieved the learning
objectives I have identified?

16. Why is alignment important?
Proper alignment keeps you going in the
right direction. If assessments are misaligned
with objectives or strategies, it can undermine
both student motivation and learning.

17. HOW CAN WE DEVELOP AN
EFECTIVE ASSESSMENT?
Strategies
Standards
Alignment

18. STRATEGIES FOR DEVELOPING
AN EFFECTIVE ASSESSMENT
1. Remember the effect of assessment on
student learning behaviors and outcomes.
2. Align the course assessment with the
learning outcomes and curriculum.
3. Prepare students for assessment by
providing formative tasks and explaining
the structure of the assessment for their
course.

19. STRATEGIES…
4. Design quality assessment tasks and
items.
5. Review Assessment Data.
6. Understand how to set standards and
grade cut-offs.
7. Give feedback to students.

20. STANDARDS
1. Assessment of Higher-Order Cognitive
Skills
2. High-Fidelity Assessment of Critical
Abilities
3. Assessments that Are Internationally
Benchmarked
4. Use of Items that Are Instructionally
Sensitive and Educationally Valuable

21. 1. Assessment of Higher-Order
Cognitive Skills
One widely used approach in
conceptualizing knowledge and skills represented in
curriculum, teaching, and assessment Webb’s Depth of
Knowledge (DOK) taxonomy.
Using the DOK framework as a guide, if
assessments are to reflect and encourage transferable
abilities, a substantial majority of the items and tasks
(at least two-thirds) should tap conceptual knowledge
and abilities.

22. Webb’s Depth of Knowledge (DOK)
taxonomy

23. 2. High-Fidelity Assessment of
Critical Abilities
This standard identifies a number of areas of knowledge
and skills that are clearly so critical for college and career
readiness that they should be targeted for inclusion in new
assessment systems. As described in the standard, these
includes:
 Research
 Analysis and Synthesis of Information
 Experimentation and Evaluation
 Communication in Oral, Written, Graphic, and Multi-Media
Forms
 Collaboration and Interpersonal Interaction
 Modeling, Design, and Complex Problem Solving

24. 3. Assessments that Are
Internationally Benchmarked
The assessments should be as rigorous as those of
the leading education countries, in terms of the kind of
content and tasks they present, as well as the level of
performance they expect.
Such assessments sought were, in order of
importance:
1) teamwork
2) problem solving
3) interpersonal skills
4) oral communications
5) listening skills

25. 4. Use of Items that Are
Instructionally Sensitive and
Educationally Valuable
Assessment tasks should also be
instructionally sensitive and educationally
useful. That is, they should 1) represent the
curriculum content in ways that respond to
instruction, and 2) have value for guiding and
informing teaching.