Reliability ITEM ANALYSIS GRADING AND REPORTING Relative Vs Absolute Scoring Record Keeping and Grading Software

1. EDUCATIONAL ASSESSMENT AND EVALUATION BY HINA JALAL (PHD SCHOLAR GCUF)
Reliability
Reliability is the degree to which an assessment tool produces stable and consistent results. The reliability of an
assessment tool is the extent to which it measures learning consistently.
Test-retest reliability is a measure of reliability obtained by administering the same test twice over a period of
time to a group of individuals. The scores from Time 1 and Time 2 can then be correlated in order to evaluate
the test for stability over time.
Inter-rater reliability is a measure of reliability used to assess the degree to which different judges or raters agree
in their assessment decisions. Inter-rater reliability is useful because human observers will not necessarily
interpret answers the same way; raters may disagree as to how well certain responses or material demonstrate
knowledge of the construct or skill being assessed.
Split-half reliability is another subtype of internal consistency reliability. The process of obtaining split-half
reliability is begun by “splitting in half” all items of a test that are intended to probe the same area of knowledge
(e.g., World War II) in order to form two “sets” of items. The entire test is administered to a group of
individuals, the total score for each “set” is computed, and finally the split-half reliability is obtained by
determining the correlation between the two total “set” scores.
Factors which can affect reliability:
The length of the assessment – a longer assessment generally produces more reliable results.
The suitability of the questions or tasks for the students being assessed.
The phrasing and terminology of the questions.
The consistency in test administration – for example, the length of time given for the assessment, instructions
given to students before the test.
The design of the marking schedule and moderation of marking procedures.
The readiness of students for the assessment – for example, a hot afternoon or straight after physical activity
might not be the best time for students to be assessed.
Validity is a bit more complex because it is more difficult to assess than reliability. There are various ways to
assess and demonstrate that an assessment is valid, but in simple terms, validity refers to how well a test
measures what it is supposed to measure. Validity refers to how well a test measures what it is purported to
measure. The validity of an assessment tool is the extent by which it measures what it was designed to measure.
Criterion-Related Validity is used to predict future or current performance - it correlates test results with another
criterion of interest.

2. EDUCATIONAL ASSESSMENT AND EVALUATION BY HINA JALAL (PHD SCHOLAR GCUF)
ITEM ANALYSIS
After creating test items, the next step would be to check whether the items are appropriate, neither too
difficult nor too easy. An item analysis is a valuable, yet easy procedure which educational
professionals can use to analyze each item on a test to determine the proportions of students selecting
each answer, besides evaluating student's strengths and weaknesses. Guildford has mentioned that
"the major goals of item analysis are the improvement of total score reliability or total score validity
(or both) and the achievement of better item sequences and types of score distribution.
PURPOSES OF ITEM ANALYSIS
Item analysis serves the following purposes
• Improving the test.
• Identifying items which may have a bias.
• Serving as a basis for class discussion.
• Diagnosing the students' strengths and weaknesses.
• Increasing the skill of item construction.
Care must also be taken that the items assess not only the lowest level of knowledge, but also higher-level
cognitive skills. Item analysis facilitates in editing wording of the items, reducing ambiguity besides
helping in identifying concepts which are not clearly understood by the students. Items have to be
periodically assessed using the following methods:
• Difficulty is found by examining the number of test takers who answered the item correctly.
• Discrimination is the degree to which answering an item correctly relates to doing well on the
whole test.
• The item characteristic curve is used for prediction of the probability of getting the answer right
with the level of success on the test.
STEPS USED IN ITEM ANALYSIS
Step l Rank the answer sheets in order trout the highest to the lowest.
Step 2 Create criterion-groups from the extremes of scores. With large sample generally the top 279E
of Upper Group (//) and the bottom 27% of Lower Group (t) of the sample is taken.
Step 3 Count the number of students in the Upper Group (II) who have selected the correct answer.
Step 4 Count the number of students in the Lower Group (L) who have selected the correct answer.
Step 5 Enter the values in the test item card.
Step 6 Compute the difficulty level using the formula.
Step 7 Compute the discriminating power of each item.
Step 8 Find the effectiveness of distracters in each item.
Step 9 Interpret the item analysis report.

3. EDUCATIONAL ASSESSMENT AND EVALUATION BY HINA JALAL (PHD SCHOLAR GCUF)
GRADING AND REPORTING
Grading and reporting student progress are among one of the more frustrating aspects of teaching-there are so
many factors to consider, and so many decisions to be made.
FUNCTIONS OF GRADING AND REPORTING
School grading and reporting systems are designed to serve a variety of functions in the school. These include
instructional uses, reports to parents, and administrative and Instructional Uses.
A. Instructional Uses
The focus of the grading and reporting system should be the improvement learning and development. This is
most likely to occur when the report
(1) Clarifies the instructional objectives,
(2) Indicates the student's strengths and weaknesses in learning,
(3) Provides information concerning the student's personal-social development,
(4) Contributes to the student's motivation.
These functions require much more comprehensive report than the single letter grade. The improvement of
student learning is probably best achieved by the day by day assessments of learning and the feedback from
tests and other assessment procedures. A portfolio of work designed to display progress can show the strides
that have been made during the year with concrete examples.
Improve students’ learning by:
• clarifying instructional objectives for them
• showing students’ strengths & weaknesses
• providing information on personal-social development
• enhancing students’ motivation (e.g., short-term goals)
• indicating where teaching might be modified
Best achieved by:
• Day-to-day tests and feedback
• Plus, periodic integrated summaries
B. Reports to parents/guardians
Informing parents (or guardians) of their children's school progress is a basic function of a grading and reporting
system. These reports should help parents understand the objectives of the school and how well their children
are achieving the intend learning outcomes of their particular program. This information is important for several
reasons.
• Communicates objectives to parents, so they can help promote learning
• Communicates how well objectives being met, so parents can better plan
Grades especially conveyed through report cards provide a critical feedback to parents about their children’s
progress in school. Aside from grades in the report cards however, feedbacks can also be obtained from

4. EDUCATIONAL ASSESSMENT AND EVALUATION BY HINA JALAL (PHD SCHOLAR GCUF)
standardized tests, teachers’ comments. Grades also help parents to identify the strengths and weaknesses of
their child. Depending on the format of report cards, parents may also receive feedbacks about their children’s
behavior, conduct, social skills and other variables that might be included in the report card. On a general point
of view, grades basically tell parents whether their child was able to perform satisfactorily. However, parents
are not fully aware about the several and separate assessments which students have taken that comprised their
grades. Some of these assessments can be seen by parents but not all. Therefore, grades of students,
communicated formally to parents can somehow let parents have an assurance that they are seeing the overall
summary of their children’s performance in school.
C. Report to Students
Grades are one way of providing feedbacks to students such that it is through grades that students can recognize
their strengths and weaknesses. Upon knowing these strengths and weaknesses, students can be able to further
develop their competencies and improve their deficiencies. Grades also help students to keep track of their
progress and identify changes in their performance. This feedback is directly proportional with the age and year
level with the students such that grades are given more importance and meaning by a high school student as
compared to a grade one student (see Raven, 2010); however, Motivation plays a role in grades. Such that grade
one students (young ones) are motivated to get high grades because of external rewards and high school students
(older ones) are also motivated internally to improve one’s competencies and performance.
D. Administrative and guidance uses
• Grades can serve as one factor in determining if a student will be promoted to the next level or not.
• Grades are factors to be considered in placing students according to their competencies and deficiencies.
• Provide input for realistic educational, vocational, and personal counseling
• Through the grades of students taking a certain program, program effectiveness can be somehow evaluated.
Grades of students can be a factor used in determining whether the program was effective or not.
• External organizations from the school also use grades as reference for admission. When students transfer
from one school to another, their grades play crucial role for their admission.
• Exceptionalities, disorders and other malfunctions can also be determined through the use of grades.
• It is through the grades of students that teachers can somehow seek the assistance of a counselor.
(Magno, Lynn, Lee, & Ko, 2009).

5. EDUCATIONAL ASSESSMENT AND EVALUATION BY HINA JALAL (PHD SCHOLAR GCUF)
Relative Vs Absolute Scoring
Grading and scoring are not the same
• Scoring (using a rubric) involves assigning an objective description to a student’s performance
• Grading involves a value judgment; the same score can be assigned different grades based on a number
of factors
• Two different teachers might assign different grades to the same scores in different classrooms
• One teacher might assign a score one grade at the beginning of a term when the students are just learning,
and a lower grade at the end of the term when students are expected to know much more.
Absolute Scoring (criterion-referenced)
In this system, each point value is assigned a letter grade. This is the system that most students in primary school
are under. For example, the most common absolute scoring system in the United States is the one that assigns
an A for points 90 to 100, a B for points 80 to 89, a C for points 70 to 79, a D for points 60 to 69, and an F for
points 0 to 59. All of these points are from a test worth 100 points.
In this system, it is possible for all students to pass and even for all of them to get As. If all students score a 90
or above on the test you have just given, then all of your students will get an A on this test.
Absolute Criterion-/task-referencing → based on a defined set of standards when evaluating a student’s
performance
Advantages Disadvantages
No reference to the
performance of others
Performance standards are difficult to specify and justify, as they may vary
unintentionally due to variations in test difficulty, student ability, and
instructional effectiveness
All students can obtain
high grades
May be subject to rater’s subjectivity
Relative scoring (Norm referenced test)
In this system, scores are given based on the student's score compared to the others in the class. This system is
used in some universities and colleges and even in some advanced high school classes. In this system, a few
students are guaranteed to receive an A and a few students are guaranteed to receive an F.
Point values in this system don't translate directly into letter grades. For example, if most of your students scored
between 80 and 90 out of 100, then this would mean that scores around 85 points are assigned a C. Scores
around the 90 marks are assigned a B, and the highest scores are assigned an A or A+. If the highest score is a
92, then 92 will equate to an A+. The lowest score will equate to an F. If the lowest score is a 75, then that 75
is an F.

6. EDUCATIONAL ASSESSMENT AND EVALUATION BY HINA JALAL (PHD SCHOLAR GCUF)
Relative scoring allows for the teacher to interpret the results of an assessment and determine grades based on
student performance. One example of this is scoring “on the curve.” In this approach, the grades of an
assessment are forced to fit a “bell curve” no matter what the distribution is. A hard grade to the curve would
look as follows.
A = Top 10% of students
B = Next 25% of students
C = Middle 30% of students
D = Next 25% of students
F = Bottom 10% of students
As such, if the entire class had a score on an exam between 90-100% using relative scoring would still create a
distribution that is balanced. Whether this is fair or not is another discussion.
Some teachers will divide the class grades by quartiles with a spread from A-D. Others will use the highest
grade achieved by an individual student as the A grade and mark other students based on the performance of
the best student.
There are times when institutions would set the policy for relative grading. For example, in a graduate school,
you may see the following grading scale.
A = top 60%
B = next 30%
C = next 10%
D, F = Should never happen
the philosophy behind this is that in graduate school all the students are excellent, so the grades should be better.
Earning a “C” is the same as earning an “F.” Earning a “D” or “F” often leads to removal from the program.
What each student scores, is only good if it is good in comparison to the grades of the rest of the class. And as
more and more universities are transitioning to relative scoring system, the question of scoring systems is
coming to the fore more frequently.
Relative Scoring/Grading -Norm-/group-referencing: based on how a student’s performance compared to others
in a group/class
Advantages Disadvantages
Easy to interpret as it describes a rank
in a group
Provides inconsistent interpretation as the meaning of a grade varies
with the ability of the student group
Can discriminate among levels of
student performance
Can be assigned without using a clear reference to specific student
performance

7. EDUCATIONAL ASSESSMENT AND EVALUATION BY HINA JALAL (PHD SCHOLAR GCUF)
Relative and absolute criteria tests
These two expressions are also referred to in the literature as norm-referenced and criterion-referenced tests.
It is very important to distinguish between tests based on reference to the “norm” (i.e. in accordance with the
curve for the results of all the students who have taken the same test, and that is why this criterion is
termed relative), and tests based on reference to a “criterion” (i.e., in relation to the description of an acceptable
performance, that is to say, the specific educational objective fixed in advance).
An absolute criteria test is one deliberately designed to give results than can be directly interpreted in terms of
the acceptable level of performance of the person tested. It enables a person's performance to be evaluated in
relation to a previously specified level of performance. The aim, therefore, is to determine whether a person has
or has not mastered a particular task, and not to compare one person's performance with that of another or of a
group of persons. A relative criteria test, on the other hand, aims at enabling a valid discrimination to be made
between persons on the basis of different types of performances; it is thus a competitive test.
Relative criteria tests are the ones most frequently employed for examination purposes. Unfortunately, their
disadvantages greatly outnumber their advantages, for if a group of students is particularly brilliant the
utilization of relative criteria tests will lead to some of them being failed although their level of performance
may be satisfactory from the absolute viewpoint. On the other hand, if a given group of students has on the
whole a low performance level (because the appropriate instruction has not been given, or has been poorly
given, or for some other reason) the relative criteria system may allow “poor” students to pass if they are above
the average of their group. The consequences may be extremely harmful for the health of the population.
If, on the other hand, a situation arises in which a certain number of persons have to be selected for admission
to a given course of study, it becomes necessary to compare their performances. In that case a relative criteria
test is appropriate.
If it is felt unanimously by an evaluation committee, for example, that all the students should be able to master
an emergency procedure, then this can be ascertained only by an absolute criteria test. These tests are, indeed,
the only ones that justify the certifying of any health worker as having demonstrated an acceptable level of
performance.
It is thus theoretically possible, and even desirable, that all the students taking an absolute criteria test should
“pass”. That would demonstrate the high degree of effectiveness of the training program. It would also, of
course, be theoretically possible for all the students to fail.
On the other hand, a relative criteria test is, by its nature, one which will always divide the students taking it
into at least two categories, those who succeed and those who do not, without any guarantee that the former is
also competent.

8. EDUCATIONAL ASSESSMENT AND EVALUATION BY HINA JALAL (PHD SCHOLAR GCUF)
Record Keeping and Grading Software
Grading software has been around for several years. The early programs were uncomplicated databases, with a
student's name, a place for grades, and simple calculators to figure test scores and averages. Today, software
has become much easier to use and more sophisticated. The software links below will get you started on your
own computerized revolution!
Homeschool Tracker by TGHomeSoft
One of the most popular software programs, homeschool tracker allows you track everything from lesson plans,
to test scores to attendance records. Print transcripts, report cards, reading lists and more.
Transcript Pro
Parents of high school students will appreciate the professional looking results of
[www.edplus.com/transpro.asp Transcript Pro]. This program even allows you to print professional looking
transcripts with or without GPA and many other options.
Edu-Track
A complete record keeping and lesson planning software that will help you keep your homeschool organized.
Online Records
Another option is to keep your homeschool records on a subscription-based website. This frees up space on
your hard drive, since you likely won't have to download a bulky software program. Also, if your computer
crashes, you won't lose valuable records you may not have backed up or printed off.
Homeschool Skedtrack
This site offers the ability to plan lessons, keep records and keep a running homeschooling schedule for a busy
family. Available for K through 12. The system will also track the number of days completed and even hours
logged, if you'd like.
Homeschool Incorporated
Another free web-based homeschool tracker system is located on the Homeschool Incorporated website. You'll
find some additional helps on this site, including legal forms, community forums and a curriculum store.
WINDOWS SOFTWARE
If you're using a personal computer with Windows, the following software has been developed for your
computer system.
• 1st Class Grade Book This program was developed "by teachers for teachers." You can download a free
trial copy to try out before spending money on the program. The program even offers a version in Chinese!

9. EDUCATIONAL ASSESSMENT AND EVALUATION BY HINA JALAL (PHD SCHOLAR GCUF)
• AutoGrade This easy-to-learn program stores information for up to ten classes with 50 students each. You
can send e-mail progress reports directly from the program. Download a free trial from the site, and view a
sample program screen.
• Class Action Gradebook You can use a "wizard" to help set up this grade book! The software calculates
grading curves and drops low scores. You can print reports in English or Spanish!
• ClassBuilder This free program is a grade book, a lesson planner, and a lot more. Teachers can create tests,
assignments, reports, and a calendar with this program. Interested teachers can view sample screens at the
Web site too.
• Class Mate Grading Tools This software received a five-star rating from ZDNet. Organize seating charts
and choose from many grading options -- including dropping a grade, assignment point weighting, and
weighting grades. Teachers can view sample reports and download the program online.
• ClassRoom Portals Teachers can download a free version of this program or buy the "pro" version. Two
teachers who wanted an online grade book that looked like a regular grade book developed the program.
They developed a "wizard" to lead users through the process of creating class records. The Web site provides
samples of all the screens, so teachers can see what the screens will look like on their computers.
• Excel-lent Gradebook For teachers who already keep grades in Microsoft Excel but would like more
features, this program offers a good option. It runs inside Excel, so those files are compatible. Teachers can
e-mail grades to students and parents.
• E-Z Grader E-Z Grader offers several different software packages for different grading options. One
software option lets teachers keep grades and create lesson plans with the same package. A Spanish version
is also available.
• Grade Genie! If you are running Windows 98 or Windows NT, Grade Genie! is designed specifically for
you. Using this program, you can cut and paste grades from a database or a spreadsheet. If you keep grades
in a spreadsheet now, this software can save a lot of re-entering.
• MyGradebook.com This free grading software allows parents to check their children's grades through a
secure online site. Teachers can also e-mail parents about their children's progress.
• Online School Reports.com Instead of keeping grades on their computers, teachers can register with this
service and keep them online. Students and parents have limited access to grades at any time.
• VAR Grade VARed Software offers two versions of grading software, one for Windows 3.1 and the other
for Windows 95, 98, 2000, or NT. The number of records teachers can keep is just about unlimited. The
program recognizes database records and is Internet-enabled.
SOFTWARE FOR MACS
• Gradekeeper This program is available for Windows, Mac OS X, Palm OS, and Pocket PC. You can track
a student's absences, missing assignments, and day-to-day grades with this program. The inexpensive
program is easy to use and set up. Its grade sheet format will seem familiar to most teachers.
• Grade Machine Teachers who are interested in posting grades online can do so with Grade Machine. They
can also scan grades or other information into the program. Users can subscribe to a newsletter to keep
abreast of the latest versions and to receive valuable tips and tricks for using the program most effectively.
Grade Machine is available for Macs and PCs.

10. EDUCATIONAL ASSESSMENT AND EVALUATION BY HINA JALAL (PHD SCHOLAR GCUF)
• The Pretty Good Grading Program (PGGP) Developed by a teacher, this is a simple program developed
especially for elementary school teachers. It can be used for PCs or for Macs.