The document discusses machine translation quality estimation, which aims to predict the Human-targeted Translation Error Rate (HTER) score for machine translated text without human evaluation. It presents two approaches - a word vectors based approach and an RNN based approach. The RNN based approach achieved better results, with Pearson's scores of 0.63 for German-English and 0.55 for English-German. Possible improvements include using neural networks for regression and training the RNN model on more parallel data to generate more accurate quality vectors.

1. Machine
Translation Quality
Estimation
V Harsha Vardhan
Neeraj Battan
Kartavya Gupta
Professor: Vasudeva Verma
Mentor: Nisarg Jhaveri

2. Introduction

3. Shared Task: Quality Estimation
The aim of our project is to determine the quality of a machine
translated text by predicting the HTER (Human-targeted
Translation Error Rate) scores.
Data provided by the shared task of Quality Estimation
contains english to spanish and spanish to english machine
translated sentences with respective HTER scores
17 Baseline features are provided

4. Applications
- Decide whether a given translation is good enough of
publishing as it is
- Inform readers of the target language only whether or not
they can rely on a translation
- Filter out the sentences which are not good enough for
posting and need post-editing by human
- Select the best translation among options from multiple
Machine Translation and/or translation memory systems
- Highlight the words that need post-editing task

5. Approaches

6. Word Vectors Based Approach

7. RNN Based Approach

8. Results (Pearson’s score)
Word Vectors based approach
- For German-English: 0.45
- For English-German: 0.43
RNN based approach
- For German-English: 0.63
- For English-German: 0.55

9. Possible Improvements
Word Vectors based approach
- Using neural networks based
regression to improve the results
RNN based approach
- Train the RNN model with a large
parallel corpus giving more
accurate quality estimation vectors
which will give us a huge boost
- Incorporating more features along
with the generated QEV

10. Thanks
Link for website
https://talent404.github.io/IRE-MTQE/
Link for code
https://github.com/talent404/IRE-MTQE