This presentation provides an overview of current developments in the onscreen marking of essays. This includes examples of voice grading, annotation software and digital rubric use. It describes benefits such as efficiency gains in marking essay of large student groups, storage and re-use of feedback and cost savings.

1. Online Marking of Essays
Trends and Developments
Patris van Boxel
Vrije Universiteit Amsterdam
30 November 2012

2. Content
• Role of (written) feedback in learning
• Challenges in marking of essays
• Trends and developments
– Growth in marking and feedback tools
– Feedback types (annotations, rubrics, voice)
– Learning analytics
• Pilots and evaluation in Dutch SCALA project
• Conclusions
2

3. The role of feedback
• Powerful impact on study behaviour &
performance
• Characteristics of good feedback
– Timely, personalised, information-rich
– ‘Feedforward’ function: bridging the gap
between current and desired (end) result
(The Power of Feedback, Hattie & Timperley, 2007)
3

4. Giving feedback on essays
Typical teacher challenges:
• Logistical workflow
– Collection, printing, distribution, storage
• Content
– May involve a lot of repetitive commentary
– Inter-marker inconsistency
• Contextual demands
– Large cohorts
– Often need for fast turnaround
4

5. Receiving feedback on essays
• Is feedback Illegible demand for legibility
OUR item expressing suspicion that the UK Royal Mail
– legible? is deliberately smudging the printing on its "write the
postcode clearly" messages reminded Howard
Greenwood that he once received a piece of
– understandable? homework marked by his physics teacher bearing an
unreadable scrawl in the usual red ink.
– timely?
Howard took his homework book up to the teacher
– actionable? and asked him what his comment said. The teacher
looked at the page and exploded, sending Howard
– accessible? back to his desk with grave threats of what would
happen if he were to be so impertinent in future.
Howard was completely bewildered until, after
much deciphering, he and his friends worked out
that the scrawl said, "You must write more
legibly".
SOURCE: NEW SCIENTIST, March 2012
5

6. Tools for essay marking
• Generic software solutions
• Dedicated marking tools
– Annotation tools (e.g. Markin)
– Automated feedback (e.g. ETS)
– Electronic assignment management (e.g.
GradeMark/Turnitin)
• Integration with learning management systems
(e.g. Blackboard-GradeMark or Moodle-Lightwork)
6

7. Marking with GradeMark in Bb at VU
• Turnitin software suite
• Advantages of
integration with LMS
• Lowers barrier for
teachers to use
• Less system admin
overhead
• Integrates with pre- and
post-marking workflows
• Shared marking
7

8. GradeMark in Blackboard

9. Feedback types
• Annotations
• Rubrics
• Voice grading
9

10. Annotations
• Contextual
• Can be stored
• Reused
• Categorised
• Shared amongst
teachers
• Basis for learning
analytics
10

11. Rubrics
• Matrix with multiple assessment criteria
• Described in terms of achievement levels
• May also be used for grading
• Advantages
– Provides insight in current and required
performance
– Offers growth perspective
– Higher-level feedback over lifespan of course and /
or curriculum
11

12. Rubrics
• Growing number of tools for (digital) rubric development (e.g.
Waypoint Outcomes)
• Basic rubric functions emerging in LMS environments
• Self-made solutions e.g. in Google docs

13. Voice grading
• Personalised feel
• Localised – inline - feedback
• Emphasis, clarification, repetition
Examples
• Voice comments in GradeMark
• Audio feedback via iAnnotate (demo)
13

14. Grading analytics
• Marking tools’ ability
to provide data
– Feedback type and
quantity
– Individual / group
level
• Diagnostic evidence
of recurring problems
• More information:
EBEAM project (JISC,
UK)
C. Ellis (2011), Univ of Huddersfield
14

15. ‘Green grading’
Imagine an instructor has 15 students each turning in two-page journal
entries printed single-sided. Let’s say the instructor will spend about
eight minutes grading each journal entry with corrections and
comments regardless if it is digital or on paper.
→ Traditional grading: It would take 1,800 watt-hours (1.8 kWh) of
energy to produce 30 sheets of paper to print the assignments.*
→ Paperless grading: It would take 316 watt-hours (0.316 kWh) to run an
average computer continuously for 2 hours to grade electronically.**
That’s a savings of 1,484 watt-hours (1.484 kWh) by grading on a
computer as opposed to using paper, the equivalent of saving 1.4 lbs
(.635 kg) of coal and reducing emissions by 3.9 lbs (1.769 kg) of carbon
dioxide.
15

16. SCALA
Investigating the potential of digital marking
• ‘Scaffolding Assessment for Learning’
(2012-2014)
• National funding (SURF) to use (digital) methods for
efficient provision of written feedback
• Project aims
– Reduction in feedback workload
• Storage and re-use of feedback
• Beter assignment management procedures
– Improvement in feedback quality and timeliness

17. SCALA project plan
• 30 pilots across four institutions
• Variation of marking tools & feedback types
• Variation in disciplines, study years, cohort sizes
• Deliverables
– Good practices
– Teacher guidelines on digital marking
– Generic feedback components e.g. rubrics for
assessment of writing skills
17

18. Conclusions
Marking tools
• make anytime, anywhere marking increasing
possible
• offer potential for efficiency and quality gains
• stimulate more consistent approaches
towards marking and use of assessment
criteria
• are able to provide timely, rich, and personal
feedback
18

19. p.van.boxel@vu.nl
19