'Optimizing for User Benefit' - EXPERT Business Showcase - Rome 2016

1. Optimizing for User Benefit
Designing User-Centric MachineTranslation Systems
Chris Hokamp
Dublin City University
chokamp@computing.dcu.ie
@Mystical_Wiz

2. What is translation?
Source text:
“My idea
is…”
Target text:
“Mijn idee
is…”

3. Source
Language
Text
Tunable MT
System
Target
Language
Text
Human
Translation
Quality
Metric
Learning to Translate Automatically

4.  Segment-level: BLEU, METEOR
 Edit Distance: TER/HTER, (Fuzzy)
Levenshtein
 Post-editing: Post-editing time, Post-editing
Effort
Measuring Translation Quality

5.  Post-editing time, Post-editing Effort
Interactive Prediction Quality
 How much will our predictions help?
 How can we measure how much they help?
Measuring Translation Utility

6.  Post-process Machine Translation
 Learn from translator feedback using Translation
Memories or Online/ Incremental Retraining
 Use Quality Estimation to guide translators to
“problem spots”
State of the Art

7. Task-Based Optimization
 Formalize the purpose of
Machine Translation for the task
 Take advantage of the user’s
context to improve predictions
 Explicitly train Machine
Translation systems to maximize
user benefit

8. Source
Language
Text
Tunable MT
System
Target
Language
Prediction
Human
Prediction
Quality
Metric
Redefining the Objective
User
Context

9. Optimizing Interactive Prediction
 Neural Machine Translation
for Interactive Translation
tasks
 Neural Machine Translation
for Post-editing tasks

10.  Task based optimization is not limited to machine
translation systems
 This model can be applied to any scenario where a
user interacts with an agent to achieve a shared
goal
The Vision