Essays

This presentation was developed for the
exclusive use of students enrolled in:

Educational Testing & Grading,
Professor Gregory E. Stone.

© 2004 Gregory E. Stone. All rights reserved. This presentation may not be
reproduced in any form, in part or as a whole, without the express written permission
of the author.

Essays
& other non-objective items

We’ll include:

Essays, Short answers and
other items that can fit
within the general parameters
Mathematical problem solving
Other partial credit items

Allow for free expression
without artificial restriction
and without prompting.

Two Fundamental Types:

Restricted Response
Extended Response

Restricted Response

Limit the ways in which you will
permit the students to answer.

Restricted Response

Limit the ways in which you will
permit the students to answer.
� There ARE correct answers.
� We allow students to express
the answer in their own words.

Restricted Response
“Write a brief essay comparing and
contrasting the term analysis and
synthesis as they relate to
constructing (a) objective items
and (b) essay items.”
am ple
ex

Restricted Response

“What is the poet’s attitude
toward literature as it is
apparent in lines 1 to 8?
what words in these lines
make this apparent.

am ple
ex

Short Answer
Restricted Response
“A car traveling 50 mph leaves
Chicago at 9am. A train traveling at
70 mph leaves Milwaukee at 10 am.
Who will arrive in Toledo (250 miles
away) first? Show your work.”

am ple
ex

Restricted Response
Predicated on the notion that
students supply the answers rather
than selecting the answer from a
group options.

Extended Response

Allow students to express their own
ideas and interrelationships among
ideas and use their own strategy for
organization.
� No “correct” answers but
reasonable/un logic.

Extended Response

Sometimes people write just for the fun of it. This is a
chance for you to have some fun writing. Pretend that
you are a pair of tennis shoes. You’ve done all kinds of
things with your owner in all kinds of weather. Now you
are being picked up again by your owner. Tell what
you, as the tennis shoes, think about what’s going to
happen to you. Tell how you feel about your owner.

ple
Space it provided below and on the next page.

ex am

Extended Response

Devise a plan to determine whether the democrats or
republicans are evenly distributed throughout the city, or
whether the supporters of each party are concentrated in
certain wards.

Design an experiment to calculate the height of a
redwood tree.

am ple
ex
Create an original story.

Extended Response

Because the focus is on logical
argument and reasoned answering
the teacher must be open and
accepting of uncomfortable
responses.

Extended Response

Based on the creative process of the
student - often including reasoning
and factual presentations.

Essays have a unique ability to assess
multiple taxonomic levels at once - while
maintaining a good understanding of
strengths and weaknesses.

Look at the family photo attached. Describe the
mood or feeling in the photo as well as the body
language of the people. Use metaphors or
similes to make these descriptions.

Analysis Knowledge
Comprehension

Is your essay a good essay?
 Checklist
1. Does the item test an important
aspect of the learning objective?
Essays should ideally be targeted
at no less than Application!

 Checklist
2. Does the item match the
learning objective in terms of
performance, emphasis and
“number of points” (on the
exam)?

 Checklist
3. Does the item require the
students to apply their
knowledge and skill to solve a
new and novel problem?
Essays should ideally be targeted
at no less than Application!

 Checklist
4. When viewed in relation to the
other items on the test, does this
item measure new information
covering the range of content and
behavior specified in the learning
objectives?

 Checklist
5. Is the item focused? Does it define
a task with specific directions,
rather than leave the assignment
so broad that almost any answer
would be acceptable?

 Checklist
6. Is the task defined by the item
within a level of complexity that
is appropriate for the intellectual
maturity of the students?

 Checklist
7. To get a good grade on the item, is
the student required to
demonstrate more than facts,
ideas, lists, definitions,
generalizations, etc.

 Checklist
8. Is the item worded in a way
that leads all students to
interpret the assignment in the
way you intended.

 Checklist
9. Does the wording of the item
make clear to the students:
Length, purpose, time needed
and the basis for evaluation?

 Checklist
10. If the essay asks students to state and
support their opinions on controversial
matters, does the wording of the item
make it clear that students’ assessments
will be based on the logic and evidence
supporting their arguments, rather
than on the actual position or opinion?

Creating scoring rubrics

Essential for ensuring fair,
reliable, valid and meaningful
assessments.

One of the hardest problems!

Rubrics come in 2 varieties:

Holistic (logic of argument)

Analytic (correct answers
provided in the student’s own
words)

Holistic (logic of argument)

Best used for Extended Response
Essays

Grading essays on how well the
argument/essay is written in
general.

Analytic (correct answers)

Best used for Restricted
Response Essays

Grading essays on whether or
not the student has provided the
correct answer.

Analytic rubrics require the
student to provide specific
information in their answer
in order to receive credit.

No “bluffing” allowed!

To construct a rubric:

(3)Ask - what is your objective?
(4)Ask - what is the taxonomy?
(5)Ask - what pieces of the puzzle
do I expect the student to
provide?

Example: Comprehend Vertical Leap

Concepts: Potential/Kinetic energy

Describe the concepts of Potential and
Kinetic energy. Give an example of
each in your description.

Describe the concepts of Potential and Kinetic
energy. Give an example of each in your
description.
1. Student correctly defined terms
2 points: Answer was clear and fully correct.
1 point: One or more answers were nearly
correct, but student missed key concept.
0 points: Student failed to provide correct answer.
2. Examples were clear and correct
2 points: Both examples were appropriate.
1 point: One example was appropriate.
0 points: Neither example was appropriate.

Creating rubrics is essential.

Using the rubrics in a
standard, systematic fashion is vital!

The loss of reliability and validity
are the leading causes of essay
derailment.

May be caused by poorly written
essays; or
May be caused by poor use of the
created rubrics.

Inconsistent Standards
Raters of essays may vary
greatly. Individuals may grade
essays differently on one day
versus the next.

(Reliability loss)

Rater Drift
Raters tend to either fail to pay
attention to criteria or change
them - midstream - over time.

(Reliability loss)

Changes in Topic and Prompt
Raters often grade essays of
different topics differently.
(e.g. I like this topic more so
I’m easier)

(Validity loss)

Reader Bias
Halo Effect

Grading based on non-graded topics.
(e.g If the student uses good grammar,
grade the essay higher, even
if the student doesn’t fully
answer the item)

Reader Bias
Carryover Effect

If question 1 was answered poorly, the
rater already has a bad feeling about
the student when they
begin grading question 2.

Or vice versa!

Scoring Suggestions!
You’ve created a rubric, so USE it!

Score one item at a time.

Score essays “anonymously”

Score subject-matter correctness separately.

Provide your students with feedback!

Essays

More Related Content

What's hot

Viewers also liked

Similar to Essays

More from shoffma5

Recently uploaded

Essays