Post-Editing of Machine Translation is becoming a standard process in technical communication and we expect there will be soon a brisk demand for PEMT professionals on labour market. To be able to respond to this trend and to offer graduates of our university a wider range of application on labour market, University of Constantin the Philosopher has incorporated Machine Translation and Post-Editing of MT into its curriculum since 2013. We believe this will open not only new horizons to students but also allow researchers to observe and study PEMT processes and analyse them. Being able to measure the productivity of PE, their behaviour and the quality of their PEMT output, we can find answers to the question such as: What are the differences between competencies of translators and post-editors? What skills and competencies could be enhanced by PE training and what is a suitable methodology? What is necessary amount of workload during training which can significantly improve a PE performance? We will be happy to share and discuss our findings from our pilot study.

1. MT and Post-Editing in Master's Level
Translation Education
Constantine the Philosopher University (UKF) Nitra, Slovakia
Daša Munková, Jakub Absolon
User conference MT@Work 2015 (4 December — Brussels)

2. MT a PEMT in Current System of
Translation Studies
Goals:
• Make know principals of
MT
• Basic of Post-Editing of
Machine Translation
• Errors typology, most
frequent errors for EN-SK
made by MT engines
• PEMT practice is a part of
internship program
Courses with content
connected with PEMT

3. • Identify differences in job
description of Translator vs. Post-
Editor
• Trainability of PE skills and
competences
• Necessary workload for PEMT
training
• Recommended Methodology for
PEMT (e.g. split techniques)
Post-Editor
Interpreter
Translator
LiteraryTranslator
Challenges and Questions

4. Pilot Study Goals
• Find out the attitude of future translators to
PEMT process (before and after pilot study)
• Find out increase/decrease of productivity
comparing PEMT with HT
• Find out if the productivity increase will be
the same for fast and slower translators
(correlation between HT and PEMT)
• Find out if there will be significant increase
of productivity after short exercises.

5. MT, format and text
characteristics
• We use Google Translate API integrated into Memsource CAT
• We use exported tables for revision in word format
• We use three types of texts:
HT and PEMT A
MT type: statistical MT
MT Specification: general
MT: Google Translate
Document Type: IT, SW user manual
Readability Index (Flesch-Kincaid):
55.7
Average Grade Level: 10.6
EVALUATION
MT type: statistical MT
MT Specification: general
MT: Google Translate
Document Type: IT, SW user manual
Readability Index (Flesch-Kincaid):
12.7
Average Grade Level: 20.3
PEMT B
MT type: statistical MT
MT Specification: general
MT: Google Translate
Document Type: IT, SW user manual
Readability Index (Flesch-Kincaid):
54.5
Average Grade Level: 11.3

6. Activities in Pilot Study
PEMT basics
MT Quality and usability
evaluation (IT text)
HT
(IT text)
MT Quality and usability
evaluation (adhesives leaflet)
PEMT
(IT text)
HT
(IT text)
PEMT
(human rights text)
PEMT
(IT text)
MT Quality and usability
evaluation (HR text)
PEMT
(human rights text)

7. Who were our respondents?
39 first year MA university
students of Translation Studies
in UKF (language pairs EN-SK)

8. Do you have any experience with
commercial translation?
• No I do not translate (only at
school) - 79.4%
• I am translating at least 5 pages per
week - 20.6 %

9. Have you ever met
with demand for PEMT?
• I've never heard of PEMT 17.6%
• Personally, no, but I heard about
PEMT 52.9%
• Yes, but I have no practical
experience 26.5%
• I have practical experience with
PEMT 2.9%

10. Which machine
translation systems
you've used?
NO
OTHER
100 %
32,4 %
8,8 %

11. How DoYou Use MT?
• I translate unknown words 55.9%
• I translate a whole sentence and then
edit them 32.4%
• I only use MT for unknown languages
11.8%
• I do not use it at all 0%

12. What DoYouThink About
the Quality of MT?
• It is a high quality translation 0%
• MT quality varies, but is a great help
55.9%
• MT helps, but it must not be
automatically copied to target 35.3%
• MT slows down the process of
translation 8.8%
• MT is absolutely unacceptable 0%

13. Translation Studies students attitude
toward PEMT before and after pilot study
• There was significant
improvement in students
attitude to PEMT although
initial level was very high in
comparison to average
translator
before after
Switch on
MT
Switch on
MT
Switch-off
MT
Switch-off
MT

14. Find out increase/decrease
of productivity comparing
PEMT with HT
0
0.2
0.4
0.6
0.8
1
1.2
1.4
0.0% 50.0% 100.0% 150.0% 200.0% 250.0% 300.0%
PEMT vs. HT acceleration
Possible reason:
• Using MT is very natural for young people
• Well selected text for MT
• Students were not experienced translators

15. Find out if the productivity
increase will be the same for
fast and slower translators
(correlation between HT and
PEMT)
We found no linear dependence between
• speed of HT speed and speed of PEMT
• HT quality and speed of PEMT
Possible reasons:
• There are other KPIs
(further research needed)
higher quality lower quality

16. Find out if there will be significant increase
of productivity after short exercises
We found no significant
difference
Possible reasons:
• Not enough amount of
exercises
(further research needed)

17. Added value of Study
for students
• Familiarizing students with the theory and current practice in the
PEMT
• We gave them the opportunity of direct comparison and HT PEMT
• We familiarize them with the system of human assessment PEMT
• We taught them to identify major MT errors, to be able to react more
quickly on them.
• We gave them possibility to measure their potential in PEMT, to be
able to decide if the PEMT will be one of their future professional
activity.

18. Future plans
• Continue in PEMT activities
• Enhance research focused on identification of
KPI for PE, creating specific competency
based model of PE, which can be used for PE
recruitment and selection
• Identification of trainable skills and
competencies
• Creation of methodology for PEMT training
• Creation of error typology specific for MT in
EN SK language combination
Our Goal:
Carefully selected
and well trained
Post-Editor
Arnold Turned His Biggest
Weakness His Calves Into A
Showcase Muscle

19. Pilot Study Results
Based on our study we find out:
• Rýchlosť prekladu (predpokladaná šikovnosť) rastie priamo úmerne
s rýchlosťou PEMT, nepotvrdil sa teda náš predpoklad, že najväčšie
zrýchlenie dosiahnu tí najpomalší v preklade
• Všetci testovaní dosiahli v PEMT vyššiu rýchlosť oproti HT
• Priemerné zvýšenie rýchlosti oproti HT bolo 2,5 násobok čo predstavu
vyššiu hodnotu než sme predpokladali na základe výskumu (xxx)
• Zvýšilo sa pozitívne vnímanie PEMT (z 91% na 99%, už na začiatku sa
prejavil mladý vek testovanej vzorky v pozitívnom postoji k MT)

20. Skóre:
1 – 100 percente akceptovateľný preklad
2 – akceptovateľný preklad (vyžaduje si menšie opravy , ale
oplatí sa mať vetu predpreloženú)
3 – čiastočne akceptovateľný preklad (vyžaduje si opravy, a
neoplatí sa mať vetu predpreloženú)
4 – neakceptovateľný preklad
5 – neviem posúdiť
Typológia chýb podľa DGT:
slovosled - W
gramatické chyby - G
terminologické chyby- T
chyby vo význame - M
ostatné – O