In all of our translation production activities we are producing data, lots of data. We are not talking now about the actual translations that are stored as translation memory data. These translation memory data have proven to be very valuable over the years and recently again as training data for Machine Translation engines. But in this session we are talking about the other data: data about the translation process. How much time was spent on different tasks, for different languages, content types, per project? What was the quality score for the translator, for the vendor? What was the user feedback on this machine translated support article? How is our MT engine performing? And has it improved since last year, since we have added 13 million more words in the training set? Some of the buyers and providers of translation are further ahead with the use of all these translation management data than others. The TAUS Dynamic Quality Framework (DQF) tracks translation management data through plug-ins that are already available for various translation tools and platforms. The vision is becoming very clear: the translation industry can have its own “Big Data”. In the past couple of months TAUS enterprise members have contributed their wishes and requirements for an industry benchmarking platform for translation quality and productivity. In this session several TAUS members will share and discuss their plans for using DQF and the Quality Dashboard. What data would you like to track? Session host: Daniel Goldschmidt (Microsoft) Presenters and panelists are: Annya Sedakova-Bertram (EMC), Fred Tuinstra (Lionbridge), Achim Ruopp (TAUS)

1. TAUS Industry Leaders Forum – Dublin Jun 6, 7
Productivity Metrics – the proverbial minefield…
What data do we want to capture, and do we even care?

2. Translation Productivity | What metrics do we need?
• Hardly anybody buys a car without
looking at the labels
• Plenty of confusion –
– mpg, km/l, l/100km
– fuel types & carbon emissions
– Urban, City, Highway
– But mostly agreed units and
measurement practices,
although…
• Are Translation Productivity Metrics
like Fuel Consumption Data?

3. Productivity Metrics | Areas where they provide value
• Cost / Effort / Value – continuous
validation of pricing models
– Source Complexity
– Fuzzy grids
– Edit Distance Data
– Ratios between per hour / per word
pricing, both for Selling and Buying
• Scheduling & Planning – accurate
prediction of throughputs
• Resourcing
– “Improvement Potential” of new
resources
– Optimized Task assignments

4. Productivity Metrics | More questions than answers
• Bare Productivity (throughput) numbers
don’t mean much without reference to things
like
– Complexity of the input
– Quality of the output
– The actual activity (T/E/P)
– Identity (experience level of a resource)
– Location
– MT engine in use
– Etc., etc.
• Large data sets will take care of outliers but
we need to be able to further dissect
• Case Study done on a distinct program –
– Fuzzy grid in use for a particular program
was inadequate
– however, net new words throughput was
much higher at nearly 400 words / hour

5. Productivity Metrics | Challenges for an LSP
• Variability in ‘work types’, customer
quality standards, resource
qualifications, instructions, toolsets,
MT engines, and more
• “Linguistic” productivities are only
part of the total TAT calculation
• Most translation tasks are paid ‘per
unit’
– Productivity (speed) is rarely a goal
in itself
– We don’t instruct translators that
they will be ‘measured’ for
productivity
Agreed definitions and metadata on
how to represent productivity metrics