This document discusses techniques for automatically assessing student summaries, as grading summaries manually is time-consuming for teachers. While existing techniques like BLEU and ROUGE work well for long texts, they are not satisfactory for evaluating short summaries. The paper proposes a new method that integrates semantic relations between words and their syntactic composition, which works better than current techniques for assessing short summaries. Experiments show the proposed method improves performance and is preferred over existing techniques.

1. ABSTRACT Summary writing is a process for creating a short version of a source
text. It can be used as a measure of understanding. As grading students’
summaries is a very time-consuming task, computer-assisted assessment can help
teachers perform the grading more effectively. Several techniques, such as BLEU,
ROUGE, N-gram co-occurrence, Latent Semantic Analysis (LSA), LSA_Ngram and
LSA_ERB, have been proposed to support the automatic assessment of students’
summaries. Since these techniques are more suitable for long texts, their
performance is not satisfactory for the evaluation of short summaries. This
paper proposes a specialized method that works well in assessing short
summaries. Our proposed method integrates the semantic relations between words,
and their syntactic composition. As a result, the proposed method is able to
obtain high accuracy and improve the performance compared with the current
techniques. Experiments have displayed that it is to be preferred over the
existing techniques. A summary evaluation system based on the proposed method
has also been developed.
Automatic summarization assessment through a combination of semantic and
syntactic information for intelligent educational systems - ResearchGate.
Available from:
http://www.researchgate.net/publication/273398082_Automatic_summarization_assess
ment_through_a_combination_of_semantic_and_syntactic_information_for_intelligent
_educational_systems [accessed May 14, 2015].