Proz  Virtual Conference Post-editing MT overview
Upcoming SlideShare
Loading in...5
×
 

Proz Virtual Conference Post-editing MT overview

on

  • 2,382 views

This slide deck has an associated audio stream also available on the Proz website at http://www.proz.com/virtual-conferences/361/program/7208

This slide deck has an associated audio stream also available on the Proz website at http://www.proz.com/virtual-conferences/361/program/7208

Statistics

Views

Total Views
2,382
Views on SlideShare
1,294
Embed Views
1,088

Actions

Likes
0
Downloads
27
Comments
0

44 Embeds 1,088

http://kv-emptypages.blogspot.com 391
http://kv-emptypages.blogspot.it 81
http://kv-emptypages.blogspot.com.es 79
http://kv-emptypages.blogspot.com.ar 68
http://kv-emptypages.blogspot.gr 58
http://feeds.feedburner.com 51
http://kv-emptypages.blogspot.co.uk 38
http://kv-emptypages.blogspot.nl 38
http://kv-emptypages.blogspot.pt 34
http://kv-emptypages.blogspot.ca 31
http://kv-emptypages.blogspot.fr 28
http://kv-emptypages.blogspot.de 25
http://kv-emptypages.blogspot.co.nz 19
http://kv-emptypages.blogspot.ie 18
http://kv-emptypages.blogspot.ru 17
http://www.proz.com 14
http://kv-emptypages.blogspot.be 13
http://kv-emptypages.blogspot.fi 11
https://twitter.com 8
http://kv-emptypages.blogspot.se 7
http://kv-emptypages.blogspot.dk 5
http://kv-emptypages.blogspot.jp 5
http://kv-emptypages.blogspot.co.il 5
http://kv-emptypages.blogspot.in 4
http://kv-emptypages.blogspot.ch 4
http://kv-emptypages.blogspot.cz 4
http://feedly.com 3
http://kv-emptypages.blogspot.sk 3
https://www.linkedin.com 3
http://draft.blogger.com 2
http://feedproxy.google.com 2
http://www.feedspot.com 2
http://kv-emptypages.blogspot.com.au 2
http://kv-emptypages.blogspot.hk 2
http://www.linkedin.com 2
http://kv-emptypages.blogspot.com.br 2
http://kv-emptypages.blogspot.ro 2
http://www.newsblur.com 1
http://digg.com 1
http://kv-emptypages.blogspot.tw 1
http://www.kv-emptypages.blogspot.com.ar 1
http://kv-emptypages.blogspot.hu 1
http://www.kv-emptypages.blogspot.com.es 1
http://kv-emptypages.blogspot.no 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Proz  Virtual Conference Post-editing MT overview Proz Virtual Conference Post-editing MT overview Presentation Transcript

  • Understanding Post-Editing Kirti Vashee kirti.vashee@asiaonline.netCopyright © 2011, Asia Online Pte Ltd
  • • Growth in Word Volume for Traditional Localization Projects • Faster Turnaround Time Requirements • Changing Translation Price-Value Expectations • Increasing Acceptance of MT by Enterprise Buyers • New Rapidly Growing Types of Content – Patents & Scientific Content – Customer Support & Care Content – Customer Conversations – User Generated ContentCopyright © 2011, Asia Online Pte Ltd
  • Human Example Words Corporate Brochures 2,000 Corporate Product Brochures 10,000 Products Software Products 50,000 User Interface Manuals / Online Help 200,000 User Documentation Existing Markets $31.4B New Markets Enterprise Information HR / Training / Reports 500,000 Communications Email / IM 10,000,000 Support / Knowledge Base Call Center / Help Desk 20,000,000+ User Generated Content Blogs / Reviews 50,000,000+ MachineCopyright © 2011, Asia Online Pte Ltd
  • • Traditional Localization Projects – Documentation and Localization • Focused on improving translation productivity • Same quality deliverable but faster and cheaper • New MT Enabled Projects – Patents & Scientific Content • Huge Volume – Hundreds of millions of words – Customer Support & Care Content • Very high value but short-lived • Technical Support & Knowledge base – Customer Conversations • Editing work only, focused on corrections.Copyright © 2011, Asia Online Pte Ltd
  • Linguistic Target Quality (TEP Level) Translation The effort and Quality linguistic work done to raise RAW MT to target quality levels is PEMT Raw MT Output Quality Common Misperceptions • The target quality level is always the same as TEP or other HT standards • The raw MT output quality is consistent from system to system • The corrective effort is always the same from language to language • There is little or no “a priori” control on the MT output quality • MT error patterns are consistent from segment to segmentCopyright © 2011, Asia Online Pte Ltd
  • Pre-Analysis of Source Error Pattern Correction Material Linguistic Error Pattern Identification Unknown Word Handling Profiling and Development of Linguistic Identification of Key Target Quality Rules Patterns Expansion of vocabulary Terminology Development of TL Style & Standards Expression Data Development Corrective Feedback Process Translation Quality Uneven Raw MT Output Quality Development Source Cleanup Raw Corrections Amplification • MT engines (especially SMT-based ones) get better with feedback • MT is not exactly the T of the TEP process • MT engines require upfront investments and analysis for best results • MT engines differ from language to language (FIGS easier than CJK) • MT error patterns can vary from segment to segmentCopyright © 2011, Asia Online Pte Ltd
  • How do you pay post-editorsfairly if each engine is different?Tools Needed:• Effective Quality metrics – Automated – Human• Confidence scores – Scores on a 0-100 scale – Can be mapped to fuzzy TM match equivalents• Post Edit Quality Analysis – After editing is complete or even while editing is in progress, effort can be easily measuredCopyright © 2011, Asia Online Pte Ltd
  • MT System Characteristics – Productivity ImplicationsQualityFree Online Engines Can be useful in some languages but often lower productivity than using TM alone and impossible to adapt to specific needs 1,000 to 3,000 Words/ Day per human editor Average segment quality = 40% - 50% TM Fuzzy MatchHuman TEP Process Typically produce 2,500 Words / Day per translatorLow Quality - Moses Less than 5% of these systems can outperform free online MT and best case productivity may be in the 3,000 Words/Day range Average segment quality = 50% - 60% TM Fuzzy MatchAverage Expert These systems can provide 5,000 to 7,000 Words/Day per editorSystem Average segment quality = 60% - 75% TM Fuzzy MatchSuperior Expert These systems can provide 9,000 to 12,000 Words/Day per editor Average segment quality = 70% - 85% TM Fuzzy MatchExceptional MT These systems can provide 12,000+ Words/Day per editor Average segment quality = 80% - 90% TM Fuzzy MatchCopyright © 2011, Asia Online Pte Ltd
  • Data Preparation Data Cleaning Translate Training Data Collections Diagnostics and Fine Tuning Quality Assurance Language Pair Foundation Data Customer Translation Data Domain Foundation Data and Linguistic AssetsCopyright © 2011, Asia Online Pte Ltd
  • Your Data Client Asia Online Bilingual Translation Memories In domain historical translations in source 1 Identify and target language. Language Pair Identify Target Language Monolingual Data 2 Top Level Monolingual target language text and URLs of Domain in-domain websites. 3 Upload Your 4 Process DataExtra Data (If Available) Data Bilingual Dictionaries and Glossaries In domain and client specific glossaries and Receive Tuning dictionaries. 5 and Test Set Source Language Non-Translatable Terms File Source language terms such as product names and place names that should not be translated. 6 Select Best 7 3000 Segments Train Engine Source Material To Be Translated Source material can be analyzed and processed to further improve quality. Ready to Translate Style Guides Quality Rules can be added to match client style guide requirements. 8 ImprovementCopyright © 2011, Asia Online Pte Ltd Plan
  • LP Source Human Reference Customized FoundationJA-EN なお, 以下の座標系の定義は Definitions pertaining to the Furthermore, the definition of the Furthermore, the following 以下の通り。 coordinate systems are given coordinate systems are as follows. coordinate system as defined. below.JA-EN せん断試験の管理特性を規定 Are the control characteristics of Are the control characteristics of Shear test criterion for defining し判断基準は明確か shearing test defined to specify shear test defined to specify characteristics of the clear? criteria for judgement clearly? criteria for judgement clearly?JA-EN ベントチューブスポット溶接の Is the strength of spot-welds on Is the strength of spot-welds on It is the intensity of the welding 強度は確認しているか vent tubes checked? vent tubes checked? spot vent tubes?EN-DE An alternate host can start the Alternative Gastgeber können das Alternative Gastgeber können das Stellvertretendes Gastgeber meeting and act as the host. Meeting starten und als Gastgeber Meeting starten und als Gastgeber beginnen können und so zu tun, als handeln. handeln. die Tagung des Aufnahmelandes.EN-DE You can publish a recorded Sie können eine aufgezeichnete Sie können eine aufgezeichnete Sie können eine namentliche training session that was Schulungssitzung veröffentlichen, schulungssitzung veröffentlichen, Fortbildungsveranstaltung created with WebEx Recorder. die mit dem WebEx-Rekorder die mit dem WebEx-Rekorder veröffentlichen, mit WebEx aufgezeichnet wurde. erstellt wurde. Fahrtenschreiber.EN-DE Once customer approves your Wenn der Kunde Ihre Anforderung Wenn der Kunde Ihre Anforderung Wenn Verbraucher stimmt ihrem request, the customer can select genehmigt, kann er eine genehmigt, kann der Kunde eine Antrag, der Kunde auswählen an application to share. Applikation zum Teilen auswählen. Applikation zum Teilen auswählen. können, einen Antrag zu teilen.EN-ES Remove the steel ball from the Retire la bola de acero de la Retire la bola de acero de la Eliminar la bola de acero de la main oil gallery before cleaning. canalización de aceite principal canalización de aceite principal limpieza galería antes de petróleo. antes de limpiar. antes de la limpieza.EN-ES Continuously with the ignition Continuamente con el encendido Continuamente con el encendido Continuamente con la ignición en on and the propulsion system conectado y el sistema de en posición on y el sistema de activo y el sistema de propulsión. active. propulsión activo. propulsión activo.EN-ES The average response time goal El objetivo del tiempo de respuesta El objetivo del tiempo de La meta media del tiempo de is assigned a specific time goal. medio se asigna a un objetivo de respuesta medio se asigna a un respuesta se asigna una meta del tiempo específico. objetivo de tiempo específico. momento específico. Customization teaches an engine how to translate using YOUR style and vocabulary Copyright © 2011, Asia Online Pte Ltd
  • A method of distilling a polymerizable vinyl compound selected from the group consisting of acrolein, methacrolein, acrylic acid, methacrylec acid, hydroxyethyl acrylate, hydroxyethyl methacrylate, hydroxypropyl acrylate, hydroxypropyl methacrylate, glycidyl acrylate and glycidyl methacrylate, the method comprising distilling the polymerizable vinyl compound in the presence of a polymerization inhibitor using a distillation tower having perforated trays without downcomers and wherein the temperature of the inner wall of the tower is maintained at a temperature sufficient to prevent the condensation of the vapor being distilled, whereby the polymerizable vinyl compound is distilled without the formation of polymer. Actual sample of Japanese to English MT output • Requires a significant terminology database effort • Special handling for long sentences • Monolingual target language analysis • Linguistic parsingCopyright © 2011, Asia Online Pte Ltd
  • Copyright © 2011, Asia Online Pte Ltd
  • • Training of post-editors – New Skills – MT Post Editing Is Different to HT Proof Editing • Different error patterns and different ways to resolve issues • Some LSPs are creating e-learning courses for post editors • 3 Kinds of Post Editors – Professional Bilingual MT Post Editors: • Often with domain expertise, these editors have been trained to understand issues with MT and not only correct the error in the sentence, but also create learning material – Early Career Post Editors: • Editing work only, focused on corrections – Monolingual Post Editors • Experts in the domain, but may not be fluently bilingual • With a mature engine, this approach will often deliver the best, most natural sounding resultsCopyright © 2011, Asia Online Pte Ltd
  • Metrics That Really Count Productivity is the• Productivity – Words translated per day per Best Quality Measure Raw MT often has a greater number of human resource errors than first pass human translation• Margin – improvement in the profit margin is but: 1. MT errors are easy to see and easy critical to greater use and adoption to fix• Consistency – Writing style and terminology (i.e. simple grammar/ word order).  MT + Human delivers higher quality than a 2. MT provides more accurate and human only approach consistent terminology 3. Human errors may be fewer, but harder to see and harder to fix. MT with more total errors is oftenOther “Useful” Quality Indicators faster to edit and fix than first passAutomated Metrics (Good indicators, but not absolute) human translations with fewer number• BLEU (Bilingual Evaluation Understudy) of errors.• F-Measure (F1 Score or F-Score)Manual Quality Metrics (Most not designed for MT, more Marginfor HT) Time• Edit Distance (Does not take into account complexity of edit)• SAE-J2450 (Industry specific)Copyright © 2011, Asia Online Pte Ltd
  • Standard TEP Excellent Average Excellent Moses Expert Expert Translated Words / 2,500 3,000 6,000 9,000 Day Hourly Rate $45 $45 $45 $45 Word Rate 15 cents 12 cents 10 cents 7.5 cents Daily Cost at Hourly Rate $360 $360 $360 $360 Daily Cost at Word Rate $375 $360 $600 $675 500,000 Word Project Hourly Cost $72,000.00 $ 60,000.00 $30,000.00 $20,000.00 Word Rate Cost $75,000.00 $ 60,000.00 $50,000.00 $37,500.00 Man Days 200.00 166.67 83.33 55.56Copyright © 2011, Asia Online Pte Ltd
  • Translator 1 Translator 2 Translator 3 Translator 4 Human Only MT + Post Editing Words Per Day 0 2,000 4,000 6,000 8,000 10,000 12,000• Productivity improvement results differ by translator. The above data is derived by studying 4 different translators productivity used only human and then with the addition of MT + human post editing by professionals• Weaker translators often tend to benefit more from technology• Customization is key to minimizing translator frustration• Rapid measurement and assessment of quality is key to profitabilityCopyright © 2011, Asia Online Pte Ltd
  • Incremental Improvement TrainingCopyright © 2011, Asia Online Pte Ltd
  • 1. Customize 2. Measure Create a new custom engine Measure the quality of the using foundation data and engine for rating and future your own language assets improvement comparisons 4. Manage 3. Improve Manage translation projects Provide corrective feedback while generating corrective removing potential for data for quality improvement. translation errors.Copyright © 2011, Asia Online Pte Ltd
  • Machine Translate Compare and Score S Original C Translation R Human Source Candidate ReferenceNote: Multiple machine 3 Measurement Tools C translation candidates can • Human Quality Assessment be scored at one time to • Automated Quality Metrics compare against each other. • Sentence Evaluation E.g. Asia Online, Google, Systran – Original Source: S • The original sentences that are to be translated. – Human Reference R • The gold standard of what a high quality human translation would look like. – Translation Candidate C • This is the translated output from the machine translation system that you are comparing.Copyright © 2011, Asia Online Pte Ltd
  • • The test set being measured: Different test sets will give very different scores. Very small test sets can give misleading results. • How many human reference translations were used: If there is more than one human reference translation, the resulting BLEU score will be higher. • The complexity of the language pair: Spanish is a simpler language in terms of grammar and structure than Finnish or Chinese. • The complexity of the domain: A patent has more complex text and structure than a children’s story book. It is not practical to use two different test sets and conclude that one translation engine is better than the other. • The capitalization of the segments being measured: When comparing metrics, the most common form of measurement is Case Insensitive. • The size of the test set: Use 1,000 or more BLIND segments to get good assessments • The measurement software: There are many measurement tools for translation quality. Each may vary slightly with respect to how a score is calculated It is clear from the above list of variations that a BLEU score number by itself has no real meaning.Copyright © 2011, Asia Online Pte Ltd
  • Evaluation Criteria of MT output Read the MT output first. Then read the source text (ST). Your understanding is not improved by the reading of the ST because the MT Excellent (4) output is satisfactory and would not need to be modified (grammatically correct/proper terminology is used/maybe not stylistically perfect but fulfills the main objective, i.e. transferring accurately all information.) Read the MT output first. Then read the source text. Your understanding is Good (3) not improved by the reading of the ST even though the MT output contains minor grammatical mistakes .You would not need to refer to the ST to correct these mistakes. Read the MT output first. Then read the source text. Your understanding is Medium (2) improved by the reading of the ST, due to significant errors in the MT output . You would have to re-read the ST a few times to correct these errors in the MT output. Read the MT output first. Then read the source text. Your understanding Poor (1) only derives from the reading of the ST, as you could not understand the MT output. It contained serious errors. You could only produce a translation by dismissing most of the MT output and/or re-translating from scratch.Copyright © 2011, Asia Online Pte Ltd
  • Human evaluators can develop custom error taxonomy to help identify key error pattern problems .Copyright © 2011, Asia Online Pte Ltd
  • Before Machine Translation Source text is processed and modified. Pre-Translation Corrections (PTC) - A list of terms that adjust the source text fixing common issues and making it more suitable for translation. Non-Translatable Terms (NTT) After Machine Translation - A list of monolingual terms that are used to ensure key terms are not Target text is processed and modified. translated. Post Translation Adjustment (PTA) Runtime Glossary (GLO) - A list of terms in the target language that - A list of bilingual terms that are used to modify the translated output. This is very ensure terminology is translated a useful for normalization of target terms. specific way. Each of the above runtime customizations can be applied in 2 forms: Default: Applied to all jobs. Job Specific: A different set of customizations can be applied for different clients.Copyright © 2011, Asia Online Pte Ltd
  • Original Source Corrected Source PrecisionTMWorkstations Precision™ Workstations ChinaSingaporeSydney China, Singapore, Sydney Hyper-VTM Hyper-V™ 6TBExternal 6TB External w/ with TO Q1 TO QUESTION 1 — <wall/>:<wall/> (d)|"(?=[ ](HD|disp|SAS|SATA)) ${1}-inch • Support for case sensitive and case insensitive matches • Support for regular expressionsCopyright © 2011, Asia Online Pte Ltd
  • Term New York Times PCs Limited Asia Online Pte Ltd Fortune 500 John Jacob Microsoft Office Cisco Local Director Man Yee WaiCopyright © 2011, Asia Online Pte Ltd
  • Original Source Specified Translation Portugal-Portuguese Portugais (Portugal) Independent Software Vendor (ISV) éditeurs de logiciels indépendants (ISV) South Holland Province La Province Hollande-Méridionale Proof of Concept (POC) engagement mission de validation technique HBA adaptateur de bus hôte Fine print Clauses complémentaires Standup HBA adapter pour adaptateur de bus hôte HBA standup adapter pour adaptateur de bus hôteCopyright © 2011, Asia Online Pte Ltd
  • Original Target Adjusted Target double port 2 port double-port 2 port deux port 2 port deux-port 2 port I5 i5 e/s E/S cloud computing Cloud Computing ompm OMPMCopyright © 2011, Asia Online Pte Ltd
  • Additional Training Data Runtime Improvements Each custom engine is a living engine and Fine tuning to specific formats constantly improves with use. There are many new and style guide requirements kinds of data sources that can improve an engine’s can be performed at runtime translation quality. without retraining the engine. Posted Edited Machine Translations • Pre-Translation Corrections Post editing of raw MT rapidly improves translation quality. • Non-Translatable Terms Data Manufacturing • Runtime Glossary Language Studio™ will analyze edits and other data and • Post-Translation Adjustments manufacture new data to improve quality. These features enable: Bilingual Translation Memories • Normalization of terms Additional in domain historical translations in source and target language that were not included in earlier training. • Control of preferred terminology • Mapping of complex rules as Target Language Monolingual Data specified in the style guide Additional monolingual target language text and URLs of in-domain websites that were not included in earlier training. Bilingual Dictionaries and Glossaries Additional in domain and client specific glossaries and dictionaries that were not included in earlier training. Source Language Non-Translatable Terms Additional source language terms that should not be translated that were not included in earlier training.Copyright © 2011, Asia Online Pte Ltd
  • Example of Training Data More initial data provided for training results in greater vocabulary and grammatical coverage above the Sufficient Data Threshold and less post editing feedback required. Data Volume Sufficient Data Threshold Data Shortfall Post Edited Feedback and Generated Data to Fill Gaps Gaps in Topic Coverage • Training data can often have gaps in coverage and an excess of data in other areas. • Gaps in coverage reduce translation quality. • Gaps can quickly be filled via post editing the machine translated output and submitting the data back to the system for further learning. • Many gaps can be filled with monolingual data only. • Further gaps can be identified and resolved by analyzing the text that is to be translated for high frequency terms and unknown words • In some cases incorrect data may be statistically more relevant. Post editing will raise the relevance of the correct grammar.Copyright © 2011, Asia Online Pte Ltd
  • The quick brown fox over jumps the lazy dog The quick brown fox jumps over the lazy dogAdditional corrective data generated Buddha jumps over the wall Siemens Wind Power CEO jumps over to Repower by Language Studio™ Pro Judge jumps over bench in courtroom melee Military surveillance bot jumps over 25 foot walls Robbie Maddison jumps over Tower Bridge on motorbike Man jumps over Grand Canyon Cow jumps over Moon With IE9 in sight, Firefox jumps over 50% market share mark Long jumper Brian Thomas jumps over a car to raise money Rally car jumps over a crazy fan! Kobe jumps over a speeding Aston Martin A deer jumps over a motorcycle A woman jogging in a California state park jumps over a 100-foot cliff to get away from attacker An Afghan Army soldier jumps over a irrigation canal while conducting a foot patrol Language Studio™ Pro analyzes corrections and generates other examples that include the corrected phrase to fill gaps in grammatical patterns. Each post edited correction is amplified producing many other corrective patterns, improving future translations.Copyright © 2011, Asia Online Pte Ltd
  • 1 Original Machine Raw Machine Human Source File Translate Translations Post Editing 2 1 2 Post Edited Translations Incremental Send Raw MT and Quality Data Analysis and Post Edited Translations Improvement Manufacturing back to Asia Online Supported File TypesCopyright © 2011, Asia Online Pte Ltd
  • Client Requirements An existing technology client that has large (100K+ docs) English knowledge base and technical support document repository and wishes to make this self-support content multilingual Translate Subset Edit Subset Train Initial Engine (~5000 docs) Unedited documents Repeat can be retranslated as multiple times as Required engine improves Translated Output Translate Documents Improve EngineCopyright © 2011, Asia Online Pte Ltd
  • Key Human Feedback Correct Mistranslation Targeted Corrections of Bad Learning Syntax/Grammar Terminology Correct Spelling and Spelling Terminology Punctuation Correct Initial System Correct Correct Human Feedback can raise the raw output to previously unseen quality levelsCopyright © 2011, Asia Online Pte Ltd
  • Post Editing Cost 6 Cost Per Word MT learns from post editing feedback and quality of 5 Post Editing (Human Translation) translation constantly improves. 4 Cost of post editing progressively reduces as MT quality 3 increases after each engine learning iteration. 2 1 MT Post Editing 1 2 3 4 5 6 Engine Learning Iteration Post Editing Effort Reduces Over Time Publication Quality Target The post editing and cleanup effort gets easier as the Quality MT engine improves. Post Editing Effort Initial efforts should focus on error analysis and correction of a representative sample data set. Raw MT Quality Each successive project should get easier and more efficient. 1 2 3 4 5 6 Engine Learning Iteration Job Duration and Human Resources MT with the same number of physical human resources Job Duration Human Translation can reduce the time required to complete the job (job + Human Post Editing duration) vs. human only. MT + human post editing reduces overall project MT duration by multiples of human only approach. + Human Post EditingCopyright © 2011, Asia Online Pte Ltd Human Resources
  • Initial System put into production Changes are collected and Trained Internal Experts added to initial corpus to drive begin initial error analysis continuous retraining and correction process All editors and users allowed Experienced editors to suggest changes which goes also allowed to make through vetting process changes Publication Quality Target Post-editing effort and cost can be managed by Quality Post Editing Effort improving the quality and performance of the MT engine via corrective linguistic feedback Raw MT Quality 1 2 3 4 5 6 Engine Learning IterationCopyright © 2011, Asia Online Pte Ltd
  • • Hunnect: Eastern European Language Focus • First Engine – Customized, without any additional engine feedback • Domain: IT / Engineering • Words: 25,000 • Measurements: – Cost – Timeframe – Quality • Quality of client delivery with machine translation + human approach must be the same or better as a human only TEP approach.Copyright © 2011, Asia Online Pte Ltd
  • 100% 25,000 Words Translation Editing Proofing90%80% 10 Days 3 Days 2 Days70%60% Cost50% Translation Post Editing Proofing 46% Time Saving40% (7 Days) With PEMT Approach30% 1 Day 5 Days 2 Days20%10% TimeCopyright © 2011, Asia Online Pte Ltd 38
  • Margin Margin 25% Proofing Proofing 5% 45% Margin Editing TEP Editing 20% 5% Proofing Translation 30% MT Post Editing 27% Cost Saving Human Translation 50% 20% Machine TranslationCopyright © 2011, Asia Online Pte Ltd 39
  • • LSP: Sajan • End Client Profile: – Large global multinational corporation in the IT domain. – Has developed its own proprietary MT system that has been developed over many years. • Project Goals – Eliminate the need for full TEP translation and limit it to MT + Post-editing • Language Pair: – English -> Simplified Chinese. – English -> European Spanish. • Domain: IT • 2nd Iteration of Customized Engine – Customized initial engine, followed by an incremental improvement based on client feedback. • Data – Client provided millions of TM phrase pairs for training – 26% were rejected in cleaning process as unsuitable for SMT training.Copyright © 2011, Asia Online Pte Ltd
  • • Quality – Client performed their own metrics – Asia Online Language Studio™ was 5 BLEU points better than the clients own MT solution. – Significant quality improvement after 60% Cost Saving providing feedback – 65 BLEU score. – Chinese scored better than first pass human translation as per end client’s feedback • Result – Client extremely impressed with result 70% Time Saving especially when compared to the output of their own MT engine. – Client has commissioned Sajan to work with more languages LRC have uploaded slides and video presentation from the conference: Slides: http://bit.ly/r6BPkT Video: http://bit.ly/trsyhgCopyright © 2011, Asia Online Pte Ltd
  • Linguistic Steering Pattern Identification, Corpus Analysis, Linguistic Problem Solver, Quality Assessment, Linguistic Asset Development and Test & Tuning Set Development MT-Savvy Translators & Editors Rapid Error Identification / Correction Manufacture Corrective Data and Drive Early Development of MT Engines Less Skilled Editors to Correct Target Language Content Can be Monolingual, Students, Housewives Monolingual Data Cleanup N-gram Resolution and PreparationCopyright © 2011, Asia Online Pte Ltd
  •  Corpus Analysis & Preparation  Pattern Identification  Linguistic Structural Analysis  Linguistic Problem Solving  Linguistic Production Process Management  Translation & MT Engine Quality Assessment  Rapid Quality Assessment  Effective Use and Development of Automated Measurements  Steering Guidance to MT Developers  Rapid Error Detection & Correction  Open minded translators  Better translator workbenches and tools  Skilled monolinguals with subject matter expertise (SME)  Community Management  Recruiting different types of editors  Quality ManagementCopyright © 2011, Asia Online Pte Ltd
  • • Better quality MT systems developed by experts working together with linguists will produce the best ROI • Low initial investment is not the best way to evaluate an MT strategy as these cheap systems often produce marginal benefits • Careful metric based evaluation is the best way to evaluate different strategies • Quality is most likely to be a product of systems developed in collaboration with experts (MT + Language) • Long-term defensible competitive advantage comes from the best systems Be Wary of Any Instant and Free SolutionsCopyright © 2011, Asia Online Pte Ltd
  • Any LSP not using MT in 5 years time will be marginalized or be a niche player. In 5 years time, leading LSPs will be translating more content in 1 year than in the previous 5 years combined. There will be more demand for translators than ever before, but roles will evolve and change.Copyright © 2011, Asia Online Pte Ltd
  • www.kv-emptypages.blogspot.com Understanding Post-Editing Kirti Vashee – kirti.vashee@asiaonline.net Follow on Twitter: @kvashee Join the Automated Language Translation Group in LinkedInCopyright © 2011, Asia Online Pte Ltd