Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Trends and developments in online marking of essays


Published on

This presentation provides an overview of current developments in the onscreen marking of essays. This includes examples of voice grading, annotation software and digital rubric use. It describes benefits such as efficiency gains in marking essay of large student groups, storage and re-use of feedback and cost savings.

Published in: Education
  • Be the first to comment

  • Be the first to like this

Trends and developments in online marking of essays

  1. 1. Online Marking of Essays Trends and Developments Patris van Boxel Vrije Universiteit Amsterdam 30 November 2012
  2. 2. Content• Role of (written) feedback in learning• Challenges in marking of essays• Trends and developments – Growth in marking and feedback tools – Feedback types (annotations, rubrics, voice) – Learning analytics• Pilots and evaluation in Dutch SCALA project• Conclusions 2
  3. 3. The role of feedback• Powerful impact on study behaviour & performance• Characteristics of good feedback – Timely, personalised, information-rich – ‘Feedforward’ function: bridging the gap between current and desired (end) result (The Power of Feedback, Hattie & Timperley, 2007) 3
  4. 4. Giving feedback on essaysTypical teacher challenges:• Logistical workflow – Collection, printing, distribution, storage• Content – May involve a lot of repetitive commentary – Inter-marker inconsistency• Contextual demands – Large cohorts – Often need for fast turnaround 4
  5. 5. Receiving feedback on essays• Is feedback Illegible demand for legibility OUR item expressing suspicion that the UK Royal Mail – legible? is deliberately smudging the printing on its "write the postcode clearly" messages reminded Howard Greenwood that he once received a piece of – understandable? homework marked by his physics teacher bearing an unreadable scrawl in the usual red ink. – timely? Howard took his homework book up to the teacher – actionable? and asked him what his comment said. The teacher looked at the page and exploded, sending Howard – accessible? back to his desk with grave threats of what would happen if he were to be so impertinent in future. Howard was completely bewildered until, after much deciphering, he and his friends worked out that the scrawl said, "You must write more legibly". SOURCE: NEW SCIENTIST, March 2012 5
  6. 6. Tools for essay marking• Generic software solutions• Dedicated marking tools – Annotation tools (e.g. Markin) – Automated feedback (e.g. ETS) – Electronic assignment management (e.g. GradeMark/Turnitin)• Integration with learning management systems (e.g. Blackboard-GradeMark or Moodle-Lightwork) 6
  7. 7. Marking with GradeMark in Bb at VU• Turnitin software suite• Advantages of integration with LMS • Lowers barrier for teachers to use • Less system admin overhead • Integrates with pre- and post-marking workflows • Shared marking 7
  8. 8. GradeMark in Blackboard
  9. 9. Feedback types• Annotations• Rubrics• Voice grading 9
  10. 10. Annotations• Contextual• Can be stored• Reused• Categorised• Shared amongst teachers• Basis for learning analytics 10
  11. 11. Rubrics• Matrix with multiple assessment criteria• Described in terms of achievement levels• May also be used for grading• Advantages – Provides insight in current and required performance – Offers growth perspective – Higher-level feedback over lifespan of course and / or curriculum 11
  12. 12. Rubrics• Growing number of tools for (digital) rubric development (e.g. Waypoint Outcomes)• Basic rubric functions emerging in LMS environments• Self-made solutions e.g. in Google docs
  13. 13. Voice grading• Personalised feel• Localised – inline - feedback• Emphasis, clarification, repetitionExamples• Voice comments in GradeMark• Audio feedback via iAnnotate (demo) 13
  14. 14. Grading analytics• Marking tools’ ability to provide data – Feedback type and quantity – Individual / group level• Diagnostic evidence of recurring problems• More information: EBEAM project (JISC, UK) C. Ellis (2011), Univ of Huddersfield 14
  15. 15. ‘Green grading’Imagine an instructor has 15 students each turning in two-page journalentries printed single-sided. Let’s say the instructor will spend abouteight minutes grading each journal entry with corrections andcomments regardless if it is digital or on paper. → Traditional grading: It would take 1,800 watt-hours (1.8 kWh) of energy to produce 30 sheets of paper to print the assignments.* → Paperless grading: It would take 316 watt-hours (0.316 kWh) to run an average computer continuously for 2 hours to grade electronically.**That’s a savings of 1,484 watt-hours (1.484 kWh) by grading on acomputer as opposed to using paper, the equivalent of saving 1.4 lbs(.635 kg) of coal and reducing emissions by 3.9 lbs (1.769 kg) of carbondioxide. 15
  16. 16. SCALA Investigating the potential of digital marking• ‘Scaffolding Assessment for Learning’ (2012-2014)• National funding (SURF) to use (digital) methods for efficient provision of written feedback• Project aims – Reduction in feedback workload • Storage and re-use of feedback • Beter assignment management procedures – Improvement in feedback quality and timeliness
  17. 17. SCALA project plan• 30 pilots across four institutions• Variation of marking tools & feedback types• Variation in disciplines, study years, cohort sizes• Deliverables – Good practices – Teacher guidelines on digital marking – Generic feedback components e.g. rubrics for assessment of writing skills 17
  18. 18. ConclusionsMarking tools• make anytime, anywhere marking increasing possible• offer potential for efficiency and quality gains• stimulate more consistent approaches towards marking and use of assessment criteria• are able to provide timely, rich, and personal feedback 18
  19. 19. 19