Your SlideShare is downloading. ×
0
HP Use Case - MT Reversed Analysis
HP Use Case - MT Reversed Analysis
HP Use Case - MT Reversed Analysis
HP Use Case - MT Reversed Analysis
HP Use Case - MT Reversed Analysis
HP Use Case - MT Reversed Analysis
HP Use Case - MT Reversed Analysis
HP Use Case - MT Reversed Analysis
HP Use Case - MT Reversed Analysis
HP Use Case - MT Reversed Analysis
HP Use Case - MT Reversed Analysis
HP Use Case - MT Reversed Analysis
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

HP Use Case - MT Reversed Analysis

246

Published on

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
246
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide
  • PEMT = MT Post-Editing
  • FR = Full RateSame principle for both TM leverage and MT: By generating/proposing a “pre-translation” to the Translation supplier (= “raw MT” in the case of MT technology; = TM leveraged segment in the case of TM technology), the effort spent to get to the final translation/translated content is reduced.In ETMA, both technologies/processes are combined: TM leverage on fuzzy matches segments, MT on ‘ New words’ = ‘ No match’ segments.
  • aka Post-editing analysis.*: An important aspect is to use the same TM matching algorithm as the one that is used for evaluating and pricing the TM matches. Since the algorithm is there you do not need to buy a new tool. And it is well understood from all parties involved (since it used for calculating the translation costs).*: As a first pass, the cost model used could be the same as the one used for TM leverage analysis. A refined “reverse analysis” cost model can be obtained wit h the help of Productivity gains evaluation (see slide xx) .
  • (*): Alternate method is to use the existing bilingual files (if there is not a mix of MT and TM segments and if the ITD before any translation work can be downloaded pro-actively).
  • (*): Alternate method is to use the existing bilingual files (if there is not a mix of MT and TM segments and if the ITD before any translation work can be downloaded pro-actively).TODO: ADD A SECTION describing the PEMT Prod Gain Evaluation (see the slide deck presented to Suppliers)
  • Transcript

    • 1. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice. MT Reversed Analysis François Richard, Feb2013
    • 2. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.2 Objective Define and apply a cost-effective and repeatable method/process to measure MT Post-Editing effort within ETMA (SDL TMS 2011) Objective Reversed analysis will help in: • Gaining understanding of main factors that impact raw MT quality. • Quantifying raw MT quality issues raised by Translation suppliers. • Deciding on new MT engines strategy (domain specific engines/ New training,...). • Monitoring quality evolution. • Identifying source content adequacy. • Adjusting pricing model. 3 steps: • Describe and document the method • Share and explain the method to Translation Suppliers • Apply the method to start collecting data about MT quality so that it can be further analysed
    • 3. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.3 What is PEMT? PEMT is linguistic work/effort required to bring (modify/correct) raw MT output to a final linguistic quality equivalent to the quality obtained with a classical translation process. Many factors can influence the raw MT quality and as a result the required PEMT effort: Volume used for the assessment Content-type (Technical, legal, Marketing,…) Language pair Quality and consistency of the source Human factor including experience and motivation, ...
    • 4. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.4 TM leverage: Source Translation Low fuzzy TM match High fuzzy TM match $=60% of FR $=30% of FR $=?? % of FR $=?? % of FR Lower Quality MT Higher Quality MT MT: }PEMT
    • 5. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.5 Description of the "Reversed analysis" method • In classical TM leverage analysis (based on TM fuzzy matching algorithm), “similarities” between source content that you need to translate and source content you have stored in your TMs are evaluated; resulting in a matching leverage analysis; combined with a cost model to calculate TM translation savings. • The principle is to apply this well-understood TM leverage analysis (*) to the target content: The raw MT (target) content is analyzed against a virtual TM fed with the final translated content (what has been post-edited); resulting in a matching leverage analysis; combined to an appropriate cost model (*) to calculate MT translation savings.
    • 6. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.6 Key properties • It is a posteriori process (PEMT must be completed) • It does not incur any extra cost • It allows to evaluate each raw MT segment by assigning it a matching percentage. • It allows to calculate a cost reduction for any translation job/task. • “Target” world (instead of “Source“ world) • Post-editing : • Over-editing translation could be a risk/temptation • Definition of quality level and corresponding post-editing effort (e.g. light PE) • Provide guidelines for linguists performing the MT post-editing
    • 7. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.7 Detailed ETMA steps • Select an ETMA LW MT job that is already completed • Categorize it (Lang pair/Volume/Content-type/Format/Quality of the source) • Isolate source segments that went through LW MT (“New word” category) • Process these segments through LW MT engine only (*) - Download corresponding “raw MT” bilingual file - Reverse it. • Process these segments through ETMA updated TM (*) - Download corresponding “post-edited” bilingual file. • Create a reverse “post-edited” TM using reverse “post-edited” bilingual file • Generate the leverage analysis of the reverse “raw MT” bilingual file against the reverse “post-edited” TM
    • 8. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.8 Illustration
    • 9. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.9 Illustration
    • 10. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.10 Illustration
    • 11. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.11 Calibrating cost model • The accurate (and unique?) method for evaluating translation costs reduction from PEMT is Productivity Gain evaluation. • Productivity Gain evaluations are usually not easy to reproduce, take time and money (a few thousands $/lang). • So, the idea is to use the results of a single Productivity Gain evaluation to calibrate the reverse analysis cost model. And then repeat as many time as required the reverse analysis and associate its results to the calibrated cot model to quantify translation costs reduction.
    • 12. © Copyright 2012 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice.12

    ×