Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Top Ten Best Practices About Translation Quality Measurement

964 views

Published on

Top Ten Best Practices About Translation Quality Measurement

Published in: Technology
  • Be the first to comment

Top Ten Best Practices About Translation Quality Measurement

  1. 1. Copyright © 2015 SDL PLC www.sdl.com Top Ten Best Practices about Translation Quality Measurement Estimated Reading Time: 5 minutes (@ 200 words/minute) 1. Preferential feedback needs a relief valve If you don’t give reviewers a way to suggest improvements or personal preferences, then you are forcing them to classify their feedback as errors. Even if you instruct them not to make preferential changes, they will. Studies have shown, on average, that 30% of feedback is preferential (sometimes, even 100%). So, instead of fighting their personal preferences, allow their opinions. Why prevent them from suggesting an improvement, even if it’s personal? Just don’t call it an error. Don’t waste time debating if something is right or wrong: When in doubt, classify the reviewer’s feedback as a personal preference, and move on. 2. KPIs: Use “errors per 1,000 words” instead of “Pass/Fail” We all love Pass/Fail criteria: They’re simple, and one doesn’t have to know the target language to understand that the translations are substandard. Dr. W. Edwards Deming, quality guru, loathed them. Why? Because they stifle continuous process improvement. Once a supplier reaches the magical “pass zone”, what’s their incentive to keep improving? Metrics based on “errors per 1,000 words” make for a better Key Performance Indicator (KPI) of translators’ performance. In case you’re wondering “why 1,000 words?”: 1,000 words indicate the average throughput of a professional translator in half a business day’s work (4 hours)—a sufficiently long time to assess performance. (For those who are Six Sigma savvy, you can use DPMO, Defects Per Million Opportunities.) 3. Don’t manage by numbers alone It’s very tempting for managers (especially when we don’t understand the target language), to simply look at the numbers of the translation quality scores. Remember that numbers are a convenient way to tell a story, but numbers are not the only source of information. It’s important to gather data from all sources. Supplement the story with the expert opinion of the reviewers. For example, a translation might get a good score, but your in-country reviewer doesn’t like it. Why is that? You need to find out. In Six Sigma quality management, there’s a concept called gemba, or “go and see.” Talk to your reviewers to better understand what’s going on. You might learn something that the numbers alone cannot tell you. 4. Need for error arbitration / reconciliation There are two sides to a story. Your reviewer might complain about the quality of the translations. But what is the translator’s feedback? Did the translator receive everything necessary to do a good job? Sometimes, errors are caused for reasons outside of the translator’s control, or the reviewer was missing some key piece of information. The so-called arbitration or reconciliation cycle is a fundamental step before conclusions can be reached. Yes, it can lengthen the turnaround time. But, with so much depending on these scores, it behooves us to ensure that they are accurate.
  2. 2. Copyright © 2015 SDL PLC www.sdl.com 5. Distinguish error types from error severities Don’t pre-assign a severity weight to error categories. Severities indicate the consequence of an error. Classifying errors types into categories helps managers to understand what areas require improvement. These are two different things, with different objectives. For example, not all typos are made equally. A typo will have a different consequence if found in the title on the front cover of a manual, as opposed to inside the manual in a footnote. The former will likely cause a reprint, while the latter will not. Same error type, but different consequence, so different severity. 6. Reproducibility & repeatability For measurements to be useful, they have to be both accurate and precise. They also need to be reproducible and repeatable. Reproducible means that different reviewers will look at the same sample and reach the same (or very similar) conclusions. Repeatable means that the same reviewer will evaluate the same sample multiple times and reach the same conclusion. Measurement System Analysis (MSA) and Gage R&R are some Six Sigma techniques used to make sure that the differences in the data are due to actual differences in what is being measured and not due to variation in measurement methods or with the reviewers themselves. 7. Ensure that reviewers are impartial Not all lawyers make good judges. Ergo, a reviewer who is very opinionated might be a good translator, but won’t make for a good reviewer. You might not always have a choice in the reviewer, but it’s something to look out for. A biased reviewer will provide biased measurements. Also, good reviewers will provide impartial feedback in a constructive and professional manner. They help the translators to become better at what they do, as opposed to “beating them down with a stick.” 8. Refer to authoritative standards Any measurement method is useless unless it relies upon standards of reference. Glossaries of approved terminology, language style guides, project instructions and compliance guidelines all serve this purpose. Generally agreed upon language conventions (think The Chicago Manual of Style and the Merriam-Webster dictionary for American English) can also serve as authoritative references. If something is wrong, one should be able to cite a reference. 9. Avoid measuring samples that are too small You wouldn’t use a telescope to observe something up close. When samples are too small, the error tolerance disappears, making it impossible to measure variance. Lean Six Sigma teaches that inspection is a form of waste because it doesn’t add value. Instead of wasting time measuring small samples (e.g. less than 1,000 words), spend that time for an extra round of proofreading. 10. Focus on what’s important A good measurement system will look at Critical-to-Quality (CTQ) parameters. Often, translation quality methods place too much emphasis on imperfections that the end users hardly notice. Is measuring punctuation errors and double spaces (and then debating them) a good use of everyone’s time? If you have any questions about translation quality best practices, feel free to contact the SDL Business Consulting team: Derek Patrick (Vice President, dpatrick@sdl.com) and Franco Zearo (Business Consultant, fzearo@sdl.com). Both are based at SDL USA in Superior, Colorado.

×