The document discusses test appraisal systems and item analysis. It defines test appraisal as the process used by educational institutions to evaluate how effectively they have conducted and evaluated students. Item analysis is a statistical technique used to select and reject test items based on their difficulty and ability to discriminate between stronger and weaker students. Item analysis provides benefits such as improving test construction, identifying weaknesses, and enhancing teaching methods.

1. Test Appraisal System
Concepts of Appraisal & Procedures

2. Topics of Discussion
• What is Test Appraisal System ?
• What is the concept of Appraisal System ?
• Assembling the Classroom Test
• Important Test Characteristics
• Item Analysis and its benefits
• Item Analysis for Norm Referenced Test
• Item Analysis for Criterion Referenced Test
• Establishing Test Validity and Reliability
• Preparing Test Item Bank

3. According to Newstrom: “It is the process of
evaluating the performance of Students,
sharing that information with them and
searching for ways to improve their
performance’’.
Definitions

4. • Test appraisal is the step where the
Institutions management finds out how
effective it has been at conduction and
evaluation of students.
• A “TEST Performance appraisal” is a process
of evaluating a Student’s performance of a test
in terms of its placement and promotion.
Meaning

5. Objectives of TEST Appraisal
Measuring the
efficiency
Maintaining Control.
Concrete and tangible
particulars about their
work
Assessment of
performance
Mutual goals of the
students, teachers & the
organization.
Growth & development
Increase harmony &
enhance effectiveness
 Personal development
 Self Satisfaction
 Involvement in the
Work.
STUDENTS INSTITUTIONAL
According to:
Aims at:

6. Assembling the TEST
• Record items on index cards
– File them in an item bank
• Double-check all individual test items
– Review the checklist for each item type
• Double-check the items as a set
– Still follows the table of specifications?
– Enough items for desired interpretations?
– Difficulty level appropriate?
– Items non-overlapping so don’t give clues?
9/21/2018 UNIVERISTY OF EDUCATION, LAHORE

7. Assembling the TEST
• Arrange items appropriately,
which usually means:
– Keep all items of one type together
– Put lowest-level item types first (T/F, matching, short-
answer, MC, interpretive, RR essay, and then ER essay)
– Within item types, put easiest learning outcomes first
(knowledge, comprehension, application, etc.)
– Administer time-consuming extended-response essays
and performance-based tasks separately
– Why put items of a type together? Clearer and more
efficient.
– Why put easiest items first? Motivational.
9/21/2018 UNIVERISTY OF EDUCATION, LAHORE

8. Assembling the TEST
• Prepare directions
– How to allot time
– How to respond (pick best alternative, etc.)
– How and where to record answers (circle, etc.;
same vs. separate page)
– How guessing will be treated (or whether to
answer all questions)
– How extended essays will be evaluated (accuracy,
organization, etc.)

9. Assembling the TEST
• Reproduce the test
– Leave ample white space on every page
– List multiple choice options vertically
– Keep all parts of an item on the same page
– The introduction to an interpretive item may be on
a facing page
– When not using a separate answer sheet, provide spaces
for answering down one side of the page (preferable the
left)
– When using a separate answer sheet
– Number items consecutively
– Proofread

10. Desirable characteristics for tests:
Apart from assessment and evaluation,
we must consider the following points
while testing or going to develop a test
appraisal system:
• Validity
• Reliability
• Utility
• Practicality
• Discrimination

11. VALIDITY
• Validity refers to the degree in which our test or other
measuring device is truly measuring what we intended
it to measure. The test question “1 + 1 = _____” is
certainly a valid basic addition question because it is
truly measuring a student’s ability to perform basic
addition.
• The subjective testing inevitably associated with testing
the productive skills reduces reliability but they are
more valid because they test integrative knowledge of
the language so they give the teacher the opportunity
to see how students really use the language.
• Teacher can have a better view of their students’
competence of language.
• So the higher reliability, the less valid or all the
opposite.

12. RELIABILITY
• Reliability refers to the consistency of the
scoring of the test, both between different
raters, and between the same rater on
different occasions.
• Objective testing should give perfect
reliability.
• However, faulty tests (ambiguous multiple
choice, wrong answers on an answer sheet,
etc) can reduce the reliability of even an
objective test.

13. UTILITY, PRACTICALITY and DISCRIMINATION
• Utility: A test which provides a lot of feedback
to assist in the planning of the rest of a course
or future courses.
• Practicality: the efficiency of the test in
physical terms. (Does it require a lot of
equipment? Does it take a lot of time to set,
administer or mark?)
• Discrimination: the ability of a test to
discriminate between stronger and weaker
students.

14. Issues in appraisal system
Appraisal
Design
Whose performance?
Who are the raters?
What problems?
How to solve?
What to evaluate?
When to evaluate?
What methods?

15. Advantages
Provide a record of
performance over a
period of time.
Provide an
opportunity for a
manager to meet &
discuss performance
Provide the employee
with feedback about
their performance
Provide an
opportunity for an
employee to discuss
issues and to clarify
expectations
Can be motivational
with the support of a
good reward and
compensation

16. If not done appropriately, can
be a negative experience.
Subject to rater errors &
biases.
If not done right can be a
complete waste of time.
Can be stressful for all
involved
Disadvantages

17. What is Item Analysis ?
• Item analysis is a statistical technique which
is used for selecting and rejecting the items
of the test on the basis of their difficulty
value and discriminated power.
• The Procedure used to Judge the quality of
an item.
• It is the Process of looking at item responses
of test.

18. PURPOSE OF ITEM ANALYSIS
• Select best question
• Identify structural content defect in question
• Detect learning difficulties
• Identify weakness
• Interpretation of student’s performance
• Control the quality of test
• Evaluate the students
• Understand the behavior of testing item

19. BENEFITS OF ITEM ANALYSIS
• Judge the quality or worth of test
• Increase skills of quality test construction
• Planning future activities of performance
check
• Basis for discussing test results
• Improving the teaching techniques and
methods