Download as PDF, PPTX
Uploaded byKeisuke Sakaguchi
BEA12_sakaguchi
This document describes NAIST's system for the HOO 2012 Shared Task on grammatical error correction. It discusses the system's configuration for correcting spelling errors, preposition errors, and determiner errors. For spelling correction, it uses a spelling checker and language model ranking. For prepositions, it trains a maximum entropy model on two corpora to detect and correct errors. For determiners, it checks noun phrases and trains two parser models to correct errors. It analyzes the results of different system runs and configurations. The system achieved preliminary F-scores of 52.4%, 72.2%, and 60.7% for spelling correction, and aimed to improve correction of existing words, use richer verb knowledge, and add more determin
More Related Content
BEA12_sakaguchi
- 1. NAIST at the HOO 2012 Shared Task Keisuke Sakaguchi, Yuta Hayashibe, Shuhei Kondo, Lis Kanashiro, Tomoya Mizumoto, Mamoru Komachi, Yuji Matsumoto ComputaFonal LinguisFcs Lab. Graduate School of InformaFon Science, Nara InsFtute of Science and Technology (NAIST), Japan Configurations of the system! Input → → Output !Spelling correction! Determiner correction! Preposition correction! System Architecture for Spelling Error Correction! System Architecture for Preposition Error Correction! System Architecture for Determiner Error Correction! Experiment and Result! Spelling correction for unknown words ! Open source spelling checker: GNU Aspell! Ranked by Google Web 1T 5-gram language model! Preliminary experiment: 52.4, 72.2, and 60.7% (Precision, Recall, F-score) 12 target prepositions (Chodorow et al., 2010): ! of, in, for, to, by, at, on, from, as, about (covering 91%) ! Replacement and insertion errors: a single model for detection and correction! Deletion errors: focus whether direct objects of verbs need prepositions ! Syntactic & semantic features described in (Tetreault et al., 2010)! Classifier: Maximum Entropy modeling! Trained on 2 types of corpus (FCE): ! “Gold” (Corrected except for Prep errors)! “Original” (FCE plain texts)! 3 target determiners: a, an, the (“an” was normalized to “a” in training & test)! check determiners of the left boundary of a noun phrase! 2 parser models: “Normal” (trained on the normal treebank) vs. ! “mixed” (trained on the treebank & its modified version in which articles at the left boundary of NPs were removed.)! Feature vector representation for each NP using syntax-based feature templates inspired by (De Felice, 2008)! Classifier: Passive aggressive algorithm! Training corpus: the CLC FCE dataset and the BNC data with feature augmentation approach of (Daumé III, 2007)! References! Summary! Future Work! Correcting spelling errors of existing words (e.g. *the then)! Getting rich knowledge about verbs from VerbNet and FrameNet! Adding target determiners (this, my, etc.)! Martin Chodorow, Michael Gamon, and Joel Tetreault. 2010. The Utility of Article and Preposition Error Correction Systems for English Language Learners: Feedback and Assessment. Language Testing, 27(3):419– 436.! Joel Tetreault, Jennifer Foster, and Martin Chodorow. 2010. Using Parse Features for Preposition Selection and Error Detection. In Proceedings of the 47th Annual Meeting of the Association for Computational Linguistics Short Papers, pages 353–358, Uppsala, Sweden.! Rachele De Felice. 2008. Automatic Error Detection in Non-native English. Ph.D. thesis University of Oxford. ! Hal Daumé III. 2007. Frustratingly Easy Domain Adaptation. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 256–263, Prague, Czech Republic. ! http://cl.naist.jp/en/ {keisuke-sa, yuta-h, shuhei-k, lis-k, tomoya-m, komachi, matsu}@is.naist.jp Combined after revision! Spelling: Spelling correction improved preposition error correction! Prepositions: Performed better when trained with the “original” set! Determiners: The "mixed" model improved performance! 8 different configurations (Runs) Preposition! Determiner! *We re-‐evaluated the Run2 because we submiTed the Run2 with the same condiFon as Run0.