This presentation is a part of the MosesCore project that encourages the development and usage of open source machine translation tools, notably the Moses statistical MT toolkit. MosesCore is supported by the European Commission Grant Number 288487 under the 7th Framework Programme. For the latest updates, follow us on Twitter - #MosesCore

1. TAUS
MACHINE
TRANSLATION
SHOWCASE
TAUS
Dynamic
Quality
Framework
Tools
11:30
–
11:50
Wednesday,
12
June
2013
Rahzeb
Choudhury
and
Maxim
Khalilov
TAUS

2. 2
Clients
MLV’s
In
country
offices/partners
Distributed
translators/authors
4
to
30
vendors
10
to
40
languages
100’
to
1000’s
translators/authors
Vendor
Management
Project
Management
Quality
Assurance
TranslaOon
Memory
Account
Management
Resources
Management
Quality
Assurance
Project
Management
TranslaOon
Memory
Resources
Management
Quality
Assurance
Project
Management
TranslaOon
Memory
Quality
Assurance
TranslaOon
Memory
Cascaded
Supply
Chain
This slide may not be used or copied without permission from TAUS

3. Dynamic
Quality
Framework
• Need
to
go
up
and
down
in
quality
and
the
translaPon
industry’s
use
of
quality
evaluaOon
models
and
metrics
needed
to
reflect
this,
• Need
industry
benchmarking
to
strengthen
and
improve
the
credibility
of
quality
assurance
processes,
• Need
to
lower
cost
of
quality
assurance
at
buyers
and
providers.
"
We
aim
to
achieve
this
through
an
industry
shared
knowledge
base,
guidance
on
QE
model
selecOon
and
shared
tools
for
benchmarking.
This slide may not be used or copied without permission from TAUS

4. Project
OrganizaPon
• Ongoing
iniPaPve
since
January
2011
• Bimonthly
project
management
calls,
primarily
with
TAUS
members
• TAUS
DQF
team
–
Nora
Aranberri,
Maxim
Khalilov,
Sharon
O’Brien,
Jaap
van
der
Meer,
Rahzeb
Choudhury
• Development
outsourced
This slide may not be used or copied without permission from TAUS

5. DYNAMIC
QUALITY
FRAMEWORK
THEORY
This slide may not be used or copied without permission from TAUS

6. This slide may not be used or copied without permission from TAUS

7. This slide may not be used or copied without permission from TAUS
DYNAMIC
QUALITY
FRAMEWORK
IN
PRACTICE

8. DQF
Content
Profiling
–
Since
July
2012
This slide may not be used or copied without permission from TAUS

9. DQF
Content
Profiling
• DelineaPng
quality
requirements
• Shared
approach
to
idenPfy
best
matching
QE
model(s)
for
content
type
and
operaPng
model
• Greater
credibility
to
decisions
• Based
on
feedback
during
QE
Summit
Dublin
(March
2013),
working
on
version
2
• Available
for
scruPny
QE
Summit
San
Jose,
October
9
This slide may not be used or copied without permission from TAUS

10. DQF
Knowledgebase
–
Since
July
2012
This slide may not be used or copied without permission from TAUS

11. DQF
Knowledgebase
This slide may not be used or copied without permission from TAUS

12. DQF
Knowledgebase
This slide may not be used or copied without permission from TAUS

13. DQF
Knowledgebase
This slide may not be used or copied without permission from TAUS

14. DQF
Knowledgebase
This slide may not be used or copied without permission from TAUS

15. Acer
DQF
KB
• Leverage
industry
best
pracPces
to
create/refine
quality
programs
• Central
resource
that
can
be
systemaPcally
reviewed
and
improved
to
benefit
all
• Work
together
to
standardize
approaches,
definiPons
and
metrics
• Following
QE
Summit
Dublin
held
consultaPons
and
published
publicly
available
best
pracPces
guidelines
This slide may not be used or copied without permission from TAUS

16. DQF
Best
PracPce
Guidelines
–
Since
May
2013
This slide may not be used or copied without permission from TAUS

17. DQF
Best
PracPce
Guidelines
This slide may not be used or copied without permission from TAUS

18. DQF
Tools
This slide may not be used or copied without permission from TAUS

19. Metrics:
Repor-ng
ü See
how
you
compare
against
peers
ü Drill
down
to
see
the
impact
of
individuals
ü Assess
correlaPon
between
automated
metrics
and
human
evaluaPon
ü Shared
learning
to
feed
fundamental
research
Apply
objec-ve
criteria
and
raise
the
credibility
of
transla-on
quality
assurance
func-on
across
the
industry
This slide may not be used or copied without permission from TAUS

20. Quality
evaluaPon
ProducPvity
tesPng
System
comparison
-­‐ Adequacy
-­‐ Fluency
-­‐ Error
typology
Post-­‐edited
MT
/
translaPon
from
scratch
-­‐ Ranking
-­‐ Quick
comparison
DQF
tools
This slide may not be used or copied without permission from TAUS

21. Metrics:
Tools
This slide may not be used or copied without permission from TAUS

22. MT
EvaluaPon
Tool:
new
project
This slide may not be used or copied without permission from TAUS

23. MT
EvaluaPon
Tool:
new
project
This slide may not be used or copied without permission from TAUS

24. MT
EvaluaPon
Tool:
new
project
This slide may not be used or copied without permission from TAUS

25. MT
EvaluaPon
Tool:
new
project
This slide may not be used or copied without permission from TAUS

26. MT
EvaluaPon
Tool:
new
project
This slide may not be used or copied without permission from TAUS

27. MT
EvaluaPon
Tool:
quality
evaluaPon
project
This slide may not be used or copied without permission from TAUS

28. MT
EvaluaPon
Tool:
quality
evaluaPon
project
This slide may not be used or copied without permission from TAUS

29. MT
EvaluaPon
Tool:
quality
evaluaPon
project
This slide may not be used or copied without permission from TAUS

30. Metrics:
Repor-ng
This slide may not be used or copied without permission from TAUS

31. Metrics:
Repor-ng
This slide may not be used or copied without permission from TAUS
Post-­‐editor
1

32. Metrics:
Repor-ng
This slide may not be used or copied without permission from TAUS

33. Upcoming
DQF
• ConsutlaPons
(pricing
MT
post-­‐ediPng)
• TranslaPon
Tools
Adopt
DQF
Models
• DQF
Tool
v2
-­‐
Go
live
August
• LocalizaPon
InsPtute
Course
translaPon
quaity
evaluaPon;
will
be
based
largely
on
the
DQF,
starts
September
2013
• QE
Summit,
San
Jose,
9
October
2013,
hosted
by
Adobe
This slide may not be used or copied without permission from TAUS

34. QE
Summit
San
Jose
This slide may not be used or copied without permission from TAUS