京大黒橋・河原研で開催されたEMNLP読み会の資料。
紹介した論文は、
Distinguishing Past, On-going, and Future Events: The EventStatus Corpus
Ruihong Huang; Ignacio Cases; Dan Jurafsky; Cleo Condoravdi; Ellen Riloff
Variational Template Machine for Data-to-Text Generationharmonylab
公開URL:https://openreview.net/forum?id=HkejNgBtPB
出典:Rong Ye, Wenxian Shi, Hao Zhou, Zhongyu Wei, Lei Li : Variational Template Machine for Data-to-Text Generation, 8th International Conference on Learning Representations(ICLR2020), Addis Ababa, Ethiopia (2020)
概要:Table形式の構造化データから文章を生成するタスク(Data-to-Text)において、Variational Auto Encoder(VAE)ベースの手法Variational Template Machine(VTM)を提案する論文です。Encoder-Decoderモデルを用いた既存のアプローチでは、生成文の多様性に欠けるという課題があります。本論文では多様な文章を生成するためにはテンプレートが重要であるという主張に基づき、テンプレートを学習可能なVAEベースの手法を提案します。提案手法では潜在変数の空間をテンプレート空間とコンテンツ空間に明示的に分離することによって、正確で多様な文生成が可能となります。また、table-textのペアデータだけではなくtableデータのないraw textデータを利用した半教師あり学習を行います。
京大黒橋・河原研で開催されたEMNLP読み会の資料。
紹介した論文は、
Distinguishing Past, On-going, and Future Events: The EventStatus Corpus
Ruihong Huang; Ignacio Cases; Dan Jurafsky; Cleo Condoravdi; Ellen Riloff
Variational Template Machine for Data-to-Text Generationharmonylab
公開URL:https://openreview.net/forum?id=HkejNgBtPB
出典:Rong Ye, Wenxian Shi, Hao Zhou, Zhongyu Wei, Lei Li : Variational Template Machine for Data-to-Text Generation, 8th International Conference on Learning Representations(ICLR2020), Addis Ababa, Ethiopia (2020)
概要:Table形式の構造化データから文章を生成するタスク(Data-to-Text)において、Variational Auto Encoder(VAE)ベースの手法Variational Template Machine(VTM)を提案する論文です。Encoder-Decoderモデルを用いた既存のアプローチでは、生成文の多様性に欠けるという課題があります。本論文では多様な文章を生成するためにはテンプレートが重要であるという主張に基づき、テンプレートを学習可能なVAEベースの手法を提案します。提案手法では潜在変数の空間をテンプレート空間とコンテンツ空間に明示的に分離することによって、正確で多様な文生成が可能となります。また、table-textのペアデータだけではなくtableデータのないraw textデータを利用した半教師あり学習を行います。
This paper contributes a noun phrase-annotated SMS corpus and proposes a weak semi-Markov CRF model for noun phrase chunking in informal text. The weak semi-CRF model improves training speed over linear-CRF and semi-CRF models while maintaining similar accuracy. Experiments on the SMS corpus show the weak semi-CRF achieves F1 scores comparable to other models but trains faster, especially with larger training data sizes.
This document presents a new method for automatically detecting false friends between Spanish and Portuguese using word embeddings. The method builds word vector spaces for each language using word2vec, finds a linear transformation between the spaces, and measures vector distances to classify word pairs as cognates or false friends. In experiments on a dataset of 710 word pairs, the method achieved state-of-the-art accuracy of 77.28% and high coverage of 97.91%, outperforming previous work. Future work will explore using different word embeddings and fine-grained classifications of partial false friends.
This document describes a Spanish language corpus for humor analysis that was created through crowd-sourcing annotations. Over 27,000 tweets were collected from humorous accounts and annotated through a web interface. The corpus contains over 100,000 annotations of the tweets' humor and funniness. Inter-annotator agreement was higher for this corpus than a previous Spanish humor corpus. The dataset will help analyze subjectivity in humor and was used in a shared task on humor classification and funniness prediction.
This document discusses position bias in instructor interventions in MOOC discussion forums. It finds that instructors are more likely to intervene in threads that appear higher on the discussion forum user interface due to their recent activity. To address this, it proposes a debiased classifier that weights examples based on their propensity for intervention. It finds this approach identifies intervention opportunities that were overlooked due to position bias. The debiased classifier outperforms a standard classifier on several metrics, demonstrating it can better predict unbiased intervention needs.
The document summarizes the history and current state of the ACL Anthology, a repository of publications from ACL-sponsored conferences. It discusses how the Anthology was established in 2001 and is now maintained by volunteers, containing over 45,000 papers. The presentation calls for community involvement to help future-proof the Anthology through efforts like migrating its infrastructure and improving documentation. It also proposes hosting the Anthology on the main ACL website and recruiting a new editor.
The document presents SAMSA, a new automatic evaluation measure for structural text simplification. SAMSA uses semantic parsing to measure the preservation of semantic structures and relations between an original text and its simplified version. It correlates significantly better with human judgments of meaning preservation and structural simplicity than prior reference-based metrics. SAMSA is the first evaluation method designed specifically for structural simplification operations like sentence splitting.
(1) Sequicity is a framework that simplifies task-oriented dialogue systems using single sequence-to-sequence architectures.
(2) It formalizes dialogues as sequences of belief spans and responses and decodes them in two stages: generating a belief span followed by a response.
(3) An experiment on two datasets found that a two-stage CopyNet instantiation of Sequicity outperformed several baselines in effectiveness, efficiency and handling out-of-vocabulary requests.
The document summarizes a study that explored how people's strategies for giving commands to a robot change over time during a collaborative navigation task. Ten participants each directed a robot for one hour via dialogue. Initially, participants predominantly used metric units like distances in their commands, but over time their commands increasingly referred to environmental landmarks. The study collected audio, text, and robot data to analyze parameters in commands. Future work aims to automate dialogue response generation based on this data.
The document describes a system for estimating emotion intensity in tweets. It takes a lexicon-based and word vector-based approach to create sentence embeddings for tweets. Various regression models are trained and an ensemble is used to predict emotion intensity scores between 0-1 for anger, sadness, joy and fear. The system achieved third place in predicting emotion intensity and second place for intensities over 0.5. Future work involves using contextual sentence embeddings to improve predictions.
This document describes Toshiba's machine translation system submitted to the WAT2015 workshop. It discusses using statistical post-editing (SPE) to improve rule-based machine translation (RBMT) output, as well as combining SPE and SMT systems using reranking with recurrent neural network language models. Experimental results show that the combined system achieved the best BLEU and RIBES scores compared to the individual SPE and SMT systems on several language pairs, including Japanese-English and Chinese-Japanese. However, human evaluation correlations were not entirely clear.
The document describes improvements made to the KyotoEBMT machine translation system. It discusses using forest parsing of input sentences to handle parsing errors and syntactic divergences. It also describes using the Nile alignment tool along with constituent parsing to improve word alignments from the training corpus. New features were added and the reranking was improved by incorporating a neural machine translation-based bilingual language model.
El documento describe el sistema de traducción basado en ejemplos KyotoEBMT. El sistema utiliza análisis de dependencia tanto del idioma origen como del idioma destino y puede manejar ambigüedades en las hipótesis de traducción mediante el uso de reglas de rejilla. Los resultados oficiales del WAT2015 muestran mejoras en las métricas BLEU y RIBES con la reranqueación de traducciones, aunque la reranqueación empeora la evaluación humana para la dirección de traducción japonés-chino. El sistema Ky
This document evaluates several neural machine translation models for English to Japanese translation. It finds that simple neural models outperform statistical machine translation baselines. Soft attention models with LSTM units performed best. However, training these models on pre-reordered data hurt performance. The neural models tended to produce grammatically correct but incomplete translations by omitting information. Replacing unknown words helped some models but more sophisticated solutions are needed for models trained on natural order data.
This document evaluates various neural machine translation models for English to Japanese translation. It compares different network architectures, recurrent units, and training data configurations. Results show that soft-attention models outperformed multi-layer encoder-decoder models, and training on pre-reordered data hurt performance. Neural machine translation models tended to generate grammatically correct but incomplete translations.
This document describes NAVER's machine translation systems for the WAT 2015 evaluation. For English-to-Japanese translation, the best system combined tree-to-string syntax-based machine translation with neural machine translation re-ranking, achieving a BLEU score of 34.60. For Korean-to-Japanese translation, the top system used phrase-based machine translation and neural machine translation re-ranking, obtaining a BLEU score of 71.38. The document also analyzes the effectiveness of character-level tokenization and other techniques for neural machine translation.
Toshiba presented their machine translation system for the WAT2015 workshop. Their system uses statistical post-editing (SPE) to correct rule-based machine translation (RBMT) output. It also combines SPE and phrase-based statistical machine translation (SMT) results by reranking the merged n-best lists using a recurrent neural network language model. Evaluation showed the combined system achieved the best results on most language pairs compared to SPE and SMT individually. Analysis of system selections by the combination found it primarily chose translations from SPE.
The document summarizes research conducted by NICT at the WAT 2015 workshop. They tested simple translation techniques like reverse pre-reordering for Japanese-to-English and character-based translation for Korean-to-Japanese. The techniques were found to work effectively and the researchers encourage wider use of these techniques if confirmed through human evaluation at the workshop.
Neural reranking of machine translation output improves both automatic metrics and subjective human evaluations of translation quality. The document analyzes reranking results from a statistical machine translation system using an attentional neural machine translation model. Reranking corrected errors related to reordering, insertion, deletion, substitution and conjugation. Specifically, it improved phrasal reordering, auxiliary verb insertion/deletion, and coordinate structures. The gains were mainly in grammatical aspects rather than lexical selection. While reranking is shown to be effective, questions remain about comparing it to pure neural machine translation and neural language models.
More from Association for Computational Linguistics (20)
Graham Neubig - 2015 - Neural Reranking Improves Subjective Quality of Machin...
Jinan Xu - 2015 - Integrating Case Frame into Japanese to Chinese Hierarchical Phrase-based Translation Model
1. Integrating Case Frame into Japanese to Chinese
Hierarchical Phrase-based Translation Model
Jin’an Xu, Jiangming Liu, YuFeng Chen, Yujie Zhang, Fang Ming,
Shaotong Li
Natural Language Processing group, Beijing Jiaotong University
5. Verb Case Frame
Subject
• Deep verb case frame
• Specific to Japanese explicit case frame
case auxiliary
送って
行き
ます
香港 特別 行政区 から の 学生 を
空港
まで
車
で
ToolObject Goal Verb
Object Time Location Tool……
Verb
attribute
6. Verb Case Frame
香港 特別 行政区
空港
まで
車
で
送って
行き
ます
から の 学生 を
私 達 は
今日
今天 我们 开车 送
VerbTool Object GoalTime Agent
• Deep verb case frame between paralleled sentences in two
languages
送って
行き
ます:
[+ ATOGT]
Agent Time Object Goal Tool Verb
送:
[+ ATOGT]
来自 香港 特别
行政区的学生
去 飞机场
7. Verb Case Frame
• Deep case frame to shallow case frame for Japanese
送って
行き
ます
香港 特別 行政区
から の 学生 を
Tool
空港
まで
車
で
Object Goal Verb
私 達 は
今日
Agent Time
ガ(は) 時間(NULL) ヲ(を) マデ(まで) デ(で)
9. Method
• Case Frame Rule extraction
• Obtain case frame rules from paralleled sentences with word
alignments
• Transforming to SCFG
• Transform case frame rules into hiero rules.
11. Method
• Transforming to SCFG
• examples o1:
r6: X X _
X _
o6: (x1: (
x2
r1: X _
(a) the example of phrase rule transformation
(b) the example of reordering rule transformation
13. Experiment
• Experiment data
• CWMT 2011 Japanese-Chinese Corpus (sentence pairs)
• Training data: 280 thousand
• Development data: 500
• Testing data: 900
• ASPEC-JC Corpus (sentence pairs)
• Training data: 680 thousand
• Development data: 2090
• Testing data: 1800
14. Experiment
• System
• exp1: Strong hierarchical phrase-based system (baseline)
• exp2: exp1 with case frame rules
• exp3: exp1 with manually case frame rules
• Variables in rule are without distinction during decoding
15. Experiment
• Experiment result
system system CWMT ASPEC
# of Rules BLEU(%) # of Rules BLEU(%)
exp1 baseline 14.7M 28.05 215.67M 27.17
exp2 exp1+cfr 14.7M+0.71M 28.36 215.67+7.21M 27.48
exp3 exp1+cfr / 28.65 / 27.63
19. Conclusion
! This paper presented an approach to improve HPB model
systems by augmenting the SCFG with Japanese CFRs.
! The CF are used to introduce new linguistically sensible
hypotheses into the translation search space while
maintaining the Hiero robustness qualities and avoiding
computational explosions.
! We obtain significant improvements over a strong HPB
baseline in the Japanese-to-Chinese task.
20. Future work
• Soft/hard constraints on case frame rule matching
• Challenge to resolve the problem of tense and aspect etc.