Presented at Baltic HLT 2018 https://www.hlt2018.ut.ee/

1. The Impact of Corpora Quality
on Neural Machine Translation
Matīss Rikters
Tilde
{firstname.lastname}@tilde.lv
September 28, 2018

2. The Story
• WMT 2017
• Weak results
• Why?

4. Tilde MT
https://tilde.com/products-and-services/machine-translation

6. The Story
• WMT 2017
• Weak results
• Why?
• WMT 2018!

7. Problems in Corpora

8. Problems in Corpora

9. Corpora Filters
• Unique parallel sentence filter – removes duplicate source-target sentence pairs
• Equal source-target filter – removes sentences that are identical in the source side and the target
side of the corpus
• Multiple sources - one target and multiple targets - one source filters – remove repeating
sentence pairs where the same source sentence is aligned to multiple different target sentences
and multiple source sentences aligned to the same target sentence
• Non-alphabetical filters – remove sentences that contain over 50% non-alphabetical symbols on
either the source side or the target and sentence pairs that have significantly more (at least 1:3)
non-alphabetical symbols in the source side than in the target side (or vice versa).
• Repeating token filter – removes sentences that exhibit repeated words or repeated phrases that
may occur in back-translated data
• Correct language filter – uses language identification software to estimate the language of each
sentence and removes any sentence that has a different identified language from the one
specified
• Moses Scripts and Subword NMT – calls Moses scripts for tokenising, cleaning, truecasing, and
Subword NMT for splitting into subword units

10. Experiments - Filtering

11. Experiments – Training NMT
Transformer architecture models trained with Sockeye
• 6 encoder and decoder layers
• 8 transformer attention heads per layer
• word embeddings and hidden layers of size 512
• dropout of 0.2
• shared subword unit vocabulary of 50 000 tokens
• maximum sentence length of 128 symbols
• a batch size of 3072 words

12. Results

13. WMT 2018
http://matrix.statmt.org/matrix/systems_list/1884

14. WMT 2018
http://matrix.statmt.org/matrix/systems_list/1885

15. WMT 2018
http://matrix.statmt.org/matrix/systems_list/1894

16. WMT 2018
http://matrix.statmt.org/matrix/systems_list/1895

17. WMT 2018
https://groups.google.com/forum/#!topic/wmt-tasks/YeJgziR8Tso

18. Conclusion
Clean your data before training models!
Use this - https://github.com/M4t1ss/parallel-corpora-tools

19. ”Neural Network Modelling for Inflected Natural Languages”
No. 1.1.1.1/16/A/215.