Introduction

Adequacy and Post-editing Utility

Experiments

Chanllenges in Predicting Machine Translation
Utility for Hu...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Motivation

MT research focused on improving the...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Motivation

MT research focused on improving the...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Motivation

MT research focused on improving the...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Motivation

MT research focused on improving the...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Motivation

MT research focused on improving the...
Introduction

Adequacy and Post-editing Utility

Experiments

Introduction
Problem

Muhammad AHSAN

A short Summary

Concl...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Introduction
Problem
It diccusses the problem of...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Introduction
Problem
It diccusses the problem of...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Introduction
Problem
It diccusses the problem of...
Introduction

Adequacy and Post-editing Utility

Experiments

Background and Related work
Research Projects

Muhammad AHSA...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Background and Related work
Research Projects
DA...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Background and Related work
Research Projects
DA...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Background and Related work
Research Projects
DA...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Background and Related work
Research Projects
Sp...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Background and Related work
Research Projects
Sp...
Introduction

Adequacy and Post-editing Utility

Experiments

Adequacy and Ranking
Traditional Focus

Muhammad AHSAN

A sh...
Introduction

Adequacy and Post-editing Utility

Experiments

Adequacy and Ranking
Traditional Focus
Research focus on tra...
Introduction

Adequacy and Post-editing Utility

Experiments

Adequacy and Ranking
Traditional Focus
Research focus on tra...
Introduction

Adequacy and Post-editing Utility

Experiments

Adequacy and Ranking
Traditional Focus
Research focus on tra...
Introduction

Adequacy and Post-editing Utility

Experiments

Adequacy and Ranking
Traditional Focus
Research focus on tra...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Adequacy and Ranking
Traditional Focus
Research ...
Introduction

Adequacy and Post-editing Utility

Experiments

Post Editing Utility
New Perspective

Muhammad AHSAN

A shor...
Introduction

Adequacy and Post-editing Utility

Experiments

Post Editing Utility
New Perspective
Adequacy and ranking fo...
Introduction

Adequacy and Post-editing Utility

Experiments

Post Editing Utility
New Perspective
Adequacy and ranking fo...
Introduction

Adequacy and Post-editing Utility

Experiments

Post Editing Utility
New Perspective
Adequacy and ranking fo...
Introduction

Adequacy and Post-editing Utility

Experiments

Post Editing Utility
New Perspective
Adequacy and ranking fo...
Introduction

Adequacy and Post-editing Utility

Experiments

Automatic Metrics
MT Evaluation
Majority MT evaluation relie...
Introduction

Adequacy and Post-editing Utility

Experiments

Automatic Metrics
MT Evaluation
Majority MT evaluation relie...
Introduction

Adequacy and Post-editing Utility

Experiments

Automatic Metrics
MT Evaluation
Majority MT evaluation relie...
Introduction

Adequacy and Post-editing Utility

Experiments

Automatic Metrics
MT Evaluation
Majority MT evaluation relie...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Automatic Metrics
MT Evaluation
Majority MT eval...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Automatic Metrics
MT Evaluation
Majority MT eval...
Introduction

Adequacy and Post-editing Utility

Experiments

Experimental Setup
Empirically evaluating behavious of human...
Introduction

Adequacy and Post-editing Utility

Experiments

Experimental Setup
Empirically evaluating behavious of human...
Introduction

Adequacy and Post-editing Utility

Experiments

Experimental Setup
Empirically evaluating behavious of human...
Introduction

Adequacy and Post-editing Utility

Experiments

Experimental Setup
Empirically evaluating behavious of human...
Introduction

Adequacy and Post-editing Utility

Experiments

Experimental Setup
Empirically evaluating behavious of human...
Introduction

Adequacy and Post-editing Utility

Experiments

Experimental Setup
Empirically evaluating behavious of human...
Introduction

Adequacy and Post-editing Utility

Experiments

Experimental Setup
Empirically evaluating behavious of human...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Autometic Metric Scores
BLEU, TER and Meteor are...
Introduction

Adequacy and Post-editing Utility

Experiments

Predicting Translation Utility
Experimental Findings

Muhamm...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Predicting Translation Utility
Experimental Find...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Predicting Translation Utility
Experimental Find...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Predicting Translation Utility
Experimental Find...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Experimental Results
Conclusion
Empirical analys...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Experimental Results
Conclusion
Empirical analys...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Experimental Results
Conclusion
Empirical analys...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Experimental Results
Conclusion
Empirical analys...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

Experimental Results
Conclusion
Empirical analys...
Introduction

Adequacy and Post-editing Utility

Experiments

References
Important References are as under

Muhammad AHSAN...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

References
Important References are as under
Chr...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

References
Important References are as under
Chr...
Introduction

Adequacy and Post-editing Utility

Experiments

Conclusion

References
Important References are as under
Chr...
Upcoming SlideShare
Loading in …5
×

Challenges in predicting machine translation utility for human post editors.

318 views

Published on

A Short Summary.

Published in: Education, Business, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
318
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Challenges in predicting machine translation utility for human post editors.

  1. 1. Introduction Adequacy and Post-editing Utility Experiments Chanllenges in Predicting Machine Translation Utility for Human Post-Editors A short Summary Muhammad AHSAN Facult´ Sciences et Techniques e Universit´ Jean Monnet e Saint Etienne, France 42100 muhammad.ahsan@gmail.com October 28, 2013 Muhammad AHSAN A short Summary Conclusion
  2. 2. Introduction Adequacy and Post-editing Utility Experiments Conclusion Motivation MT research focused on improving the adequacy of automatic translations which is semantic similarity to reference translations as perceived by human judges. Muhammad AHSAN A short Summary
  3. 3. Introduction Adequacy and Post-editing Utility Experiments Conclusion Motivation MT research focused on improving the adequacy of automatic translations which is semantic similarity to reference translations as perceived by human judges. Focus on acheiving a reliable finished translation that can be used immediately. Muhammad AHSAN A short Summary
  4. 4. Introduction Adequacy and Post-editing Utility Experiments Conclusion Motivation MT research focused on improving the adequacy of automatic translations which is semantic similarity to reference translations as perceived by human judges. Focus on acheiving a reliable finished translation that can be used immediately. Significant advancement in MT quality and evaluation of MT system using automatic metrics. Muhammad AHSAN A short Summary
  5. 5. Introduction Adequacy and Post-editing Utility Experiments Conclusion Motivation MT research focused on improving the adequacy of automatic translations which is semantic similarity to reference translations as perceived by human judges. Focus on acheiving a reliable finished translation that can be used immediately. Significant advancement in MT quality and evaluation of MT system using automatic metrics. Under-explored area of MT to assist human translators is getting more attractive. Muhammad AHSAN A short Summary
  6. 6. Introduction Adequacy and Post-editing Utility Experiments Conclusion Motivation MT research focused on improving the adequacy of automatic translations which is semantic similarity to reference translations as perceived by human judges. Focus on acheiving a reliable finished translation that can be used immediately. Significant advancement in MT quality and evaluation of MT system using automatic metrics. Under-explored area of MT to assist human translators is getting more attractive. Post-editing activity by humans translators is aimed to improve the translation provided by MT system. Muhammad AHSAN A short Summary
  7. 7. Introduction Adequacy and Post-editing Utility Experiments Introduction Problem Muhammad AHSAN A short Summary Conclusion
  8. 8. Introduction Adequacy and Post-editing Utility Experiments Conclusion Introduction Problem It diccusses the problem of prediction challenges for post-editing effort of human translators after MT output. Empirical study about limitation of existing automatic metrics. Muhammad AHSAN A short Summary
  9. 9. Introduction Adequacy and Post-editing Utility Experiments Conclusion Introduction Problem It diccusses the problem of prediction challenges for post-editing effort of human translators after MT output. Empirical study about limitation of existing automatic metrics. Dedicated to introduce the MT evaluation problem with a new perspective for research community. Muhammad AHSAN A short Summary
  10. 10. Introduction Adequacy and Post-editing Utility Experiments Conclusion Introduction Problem It diccusses the problem of prediction challenges for post-editing effort of human translators after MT output. Empirical study about limitation of existing automatic metrics. Dedicated to introduce the MT evaluation problem with a new perspective for research community. Discussion about differences between widely used adequacy-based evaluations and post-editing scenarios. Muhammad AHSAN A short Summary
  11. 11. Introduction Adequacy and Post-editing Utility Experiments Background and Related work Research Projects Muhammad AHSAN A short Summary Conclusion
  12. 12. Introduction Adequacy and Post-editing Utility Experiments Conclusion Background and Related work Research Projects DARPA GALE program (Olive et al) included the first major machine translation evaluation campaign to use human-targeted translation edit rate HTER (Snover et al., 2006) as the primary evaluation metric. Muhammad AHSAN A short Summary
  13. 13. Introduction Adequacy and Post-editing Utility Experiments Conclusion Background and Related work Research Projects DARPA GALE program (Olive et al) included the first major machine translation evaluation campaign to use human-targeted translation edit rate HTER (Snover et al., 2006) as the primary evaluation metric. The 2010 ACL Workshop on Statistical Machine Translation (Callison-Burch et al., 2010) featured a post-editing task where human judges were shown MT outputs without source sentences or reference translations and asked to edit them based on perceived meaning. Muhammad AHSAN A short Summary
  14. 14. Introduction Adequacy and Post-editing Utility Experiments Conclusion Background and Related work Research Projects DARPA GALE program (Olive et al) included the first major machine translation evaluation campaign to use human-targeted translation edit rate HTER (Snover et al., 2006) as the primary evaluation metric. The 2010 ACL Workshop on Statistical Machine Translation (Callison-Burch et al., 2010) featured a post-editing task where human judges were shown MT outputs without source sentences or reference translations and asked to edit them based on perceived meaning. He et al. (2010) conducted a user study that integrated MT into a translation memory (TM) system used by human translators. Muhammad AHSAN A short Summary
  15. 15. Introduction Adequacy and Post-editing Utility Experiments Conclusion Background and Related work Research Projects Specia (2011) evaluates the effectiveness of using MT confidence estimation methods to predict post-editor effort as measured by HTER, editing time, and editor post-assessments. Muhammad AHSAN A short Summary
  16. 16. Introduction Adequacy and Post-editing Utility Experiments Conclusion Background and Related work Research Projects Specia (2011) evaluates the effectiveness of using MT confidence estimation methods to predict post-editor effort as measured by HTER, editing time, and editor post-assessments. Hardt and Elming (2010) show the potential benefit of tighter integration of MT into translation workflows by conducting an experiment that simulates incrementally re-training a MT system as translators provide post-edited trans- lations. Muhammad AHSAN A short Summary
  17. 17. Introduction Adequacy and Post-editing Utility Experiments Adequacy and Ranking Traditional Focus Muhammad AHSAN A short Summary Conclusion
  18. 18. Introduction Adequacy and Post-editing Utility Experiments Adequacy and Ranking Traditional Focus Research focus on translation adequacy. Muhammad AHSAN A short Summary Conclusion
  19. 19. Introduction Adequacy and Post-editing Utility Experiments Adequacy and Ranking Traditional Focus Research focus on translation adequacy. Automatic metrics like BLEU, TER and Meteor designed to correlate well with adequacy judgement. Muhammad AHSAN A short Summary Conclusion
  20. 20. Introduction Adequacy and Post-editing Utility Experiments Adequacy and Ranking Traditional Focus Research focus on translation adequacy. Automatic metrics like BLEU, TER and Meteor designed to correlate well with adequacy judgement. Judgement splits between adequacy (capture meaning) and fluency (grammetic correctness). Muhammad AHSAN A short Summary Conclusion
  21. 21. Introduction Adequacy and Post-editing Utility Experiments Adequacy and Ranking Traditional Focus Research focus on translation adequacy. Automatic metrics like BLEU, TER and Meteor designed to correlate well with adequacy judgement. Judgement splits between adequacy (capture meaning) and fluency (grammetic correctness). High correlation between adequacy and fluency. Muhammad AHSAN A short Summary Conclusion
  22. 22. Introduction Adequacy and Post-editing Utility Experiments Conclusion Adequacy and Ranking Traditional Focus Research focus on translation adequacy. Automatic metrics like BLEU, TER and Meteor designed to correlate well with adequacy judgement. Judgement splits between adequacy (capture meaning) and fluency (grammetic correctness). High correlation between adequacy and fluency. Ranking based evaluation rely on humans to rank MT outputs from best to worse. Muhammad AHSAN A short Summary
  23. 23. Introduction Adequacy and Post-editing Utility Experiments Post Editing Utility New Perspective Muhammad AHSAN A short Summary Conclusion
  24. 24. Introduction Adequacy and Post-editing Utility Experiments Post Editing Utility New Perspective Adequacy and ranking for assesing acceptibility of MT outputs. Muhammad AHSAN A short Summary Conclusion
  25. 25. Introduction Adequacy and Post-editing Utility Experiments Post Editing Utility New Perspective Adequacy and ranking for assesing acceptibility of MT outputs. Post-editing judgement for assesing utility of MT outputs. Muhammad AHSAN A short Summary Conclusion
  26. 26. Introduction Adequacy and Post-editing Utility Experiments Post Editing Utility New Perspective Adequacy and ranking for assesing acceptibility of MT outputs. Post-editing judgement for assesing utility of MT outputs. Adequacy maximize semantic similarity w.r.t reference translation. Muhammad AHSAN A short Summary Conclusion
  27. 27. Introduction Adequacy and Post-editing Utility Experiments Post Editing Utility New Perspective Adequacy and ranking for assesing acceptibility of MT outputs. Post-editing judgement for assesing utility of MT outputs. Adequacy maximize semantic similarity w.r.t reference translation. Utility minimize the human’s effort required to correct MT outputs. Muhammad AHSAN A short Summary Conclusion
  28. 28. Introduction Adequacy and Post-editing Utility Experiments Automatic Metrics MT Evaluation Majority MT evaluation relies on automatic metrics. Muhammad AHSAN A short Summary Conclusion
  29. 29. Introduction Adequacy and Post-editing Utility Experiments Automatic Metrics MT Evaluation Majority MT evaluation relies on automatic metrics. BLEU (Bilingual Evaluation Understudy) measures simple surface word n-gram precision balanced with brevity penalty. Muhammad AHSAN A short Summary Conclusion
  30. 30. Introduction Adequacy and Post-editing Utility Experiments Automatic Metrics MT Evaluation Majority MT evaluation relies on automatic metrics. BLEU (Bilingual Evaluation Understudy) measures simple surface word n-gram precision balanced with brevity penalty. BLEU correlates with Human quality judgement. Muhammad AHSAN A short Summary Conclusion
  31. 31. Introduction Adequacy and Post-editing Utility Experiments Automatic Metrics MT Evaluation Majority MT evaluation relies on automatic metrics. BLEU (Bilingual Evaluation Understudy) measures simple surface word n-gram precision balanced with brevity penalty. BLEU correlates with Human quality judgement. Translation with higher BLEU score requires more effort to post edit (Reference Table 2 in paper). Muhammad AHSAN A short Summary Conclusion
  32. 32. Introduction Adequacy and Post-editing Utility Experiments Conclusion Automatic Metrics MT Evaluation Majority MT evaluation relies on automatic metrics. BLEU (Bilingual Evaluation Understudy) measures simple surface word n-gram precision balanced with brevity penalty. BLEU correlates with Human quality judgement. Translation with higher BLEU score requires more effort to post edit (Reference Table 2 in paper). Inconsistency between adequacy judgement, BLEU scores and post-editing effort (Reference Table 2 in paper). Muhammad AHSAN A short Summary
  33. 33. Introduction Adequacy and Post-editing Utility Experiments Conclusion Automatic Metrics MT Evaluation Majority MT evaluation relies on automatic metrics. BLEU (Bilingual Evaluation Understudy) measures simple surface word n-gram precision balanced with brevity penalty. BLEU correlates with Human quality judgement. Translation with higher BLEU score requires more effort to post edit (Reference Table 2 in paper). Inconsistency between adequacy judgement, BLEU scores and post-editing effort (Reference Table 2 in paper). Best MT system for post editors selected via human judgement or BLEU score does not necessarily yield efficient translation (Reference Table 2 in paper). Muhammad AHSAN A short Summary
  34. 34. Introduction Adequacy and Post-editing Utility Experiments Experimental Setup Empirically evaluating behavious of human post editors and effectiveness of MT systems. Setting Muhammad AHSAN A short Summary Conclusion
  35. 35. Introduction Adequacy and Post-editing Utility Experiments Experimental Setup Empirically evaluating behavious of human post editors and effectiveness of MT systems. Setting 5 English-Spanish parallel documents. Muhammad AHSAN A short Summary Conclusion
  36. 36. Introduction Adequacy and Post-editing Utility Experiments Experimental Setup Empirically evaluating behavious of human post editors and effectiveness of MT systems. Setting 5 English-Spanish parallel documents. Total 90 sentences. Muhammad AHSAN A short Summary Conclusion
  37. 37. Introduction Adequacy and Post-editing Utility Experiments Experimental Setup Empirically evaluating behavious of human post editors and effectiveness of MT systems. Setting 5 English-Spanish parallel documents. Total 90 sentences. 2 different MT systems with significant lexical differences. Muhammad AHSAN A short Summary Conclusion
  38. 38. Introduction Adequacy and Post-editing Utility Experiments Experimental Setup Empirically evaluating behavious of human post editors and effectiveness of MT systems. Setting 5 English-Spanish parallel documents. Total 90 sentences. 2 different MT systems with significant lexical differences. Spanish side of translation provides reference translation. Muhammad AHSAN A short Summary Conclusion
  39. 39. Introduction Adequacy and Post-editing Utility Experiments Experimental Setup Empirically evaluating behavious of human post editors and effectiveness of MT systems. Setting 5 English-Spanish parallel documents. Total 90 sentences. 2 different MT systems with significant lexical differences. Spanish side of translation provides reference translation. Human evaluators experts and students. Muhammad AHSAN A short Summary Conclusion
  40. 40. Introduction Adequacy and Post-editing Utility Experiments Experimental Setup Empirically evaluating behavious of human post editors and effectiveness of MT systems. Setting 5 English-Spanish parallel documents. Total 90 sentences. 2 different MT systems with significant lexical differences. Spanish side of translation provides reference translation. Human evaluators experts and students. Rating from 1 (No Editing) to 4 (Re translate) Time to translate is not considered as quantifying factor for effort. Muhammad AHSAN A short Summary Conclusion
  41. 41. Introduction Adequacy and Post-editing Utility Experiments Conclusion Autometic Metric Scores BLEU, TER and Meteor are used as metrics. Refering to table 3 and table 4 given in paper Limitation of Metrics Low metric score against gold standard points at limitations which are good at detecting similar translation but bad at evaluating translation with different structure and post-edit choice. First group editors apply twice as much edits as second group but metric assigns nearly same score. Low correlation coefficient for BLEU (Table 4) and results of counter examples provided in section 3.2 nullify the notion that improvement in BLEU improve MT quality. Muhammad AHSAN A short Summary
  42. 42. Introduction Adequacy and Post-editing Utility Experiments Predicting Translation Utility Experimental Findings Muhammad AHSAN A short Summary Conclusion
  43. 43. Introduction Adequacy and Post-editing Utility Experiments Conclusion Predicting Translation Utility Experimental Findings To save human translator’s time, MT must emit good results to become starting point. Muhammad AHSAN A short Summary
  44. 44. Introduction Adequacy and Post-editing Utility Experiments Conclusion Predicting Translation Utility Experimental Findings To save human translator’s time, MT must emit good results to become starting point. An additional experiment conducted with usability ranking from 1 (No editing) to 4 (Retranslation) Muhammad AHSAN A short Summary
  45. 45. Introduction Adequacy and Post-editing Utility Experiments Conclusion Predicting Translation Utility Experimental Findings To save human translator’s time, MT must emit good results to become starting point. An additional experiment conducted with usability ranking from 1 (No editing) to 4 (Retranslation) MT ouput 90.6% usable and 9.4% non-reusable. Human experts are quick to rank usability. Human expert may also misjudge the efficiency which cost extra time and effort. Muhammad AHSAN A short Summary
  46. 46. Introduction Adequacy and Post-editing Utility Experiments Conclusion Experimental Results Conclusion Empirical analysis of challenges regarding predicting the utility of automatic translations for human post-editors. Muhammad AHSAN A short Summary
  47. 47. Introduction Adequacy and Post-editing Utility Experiments Conclusion Experimental Results Conclusion Empirical analysis of challenges regarding predicting the utility of automatic translations for human post-editors. BLEU metric to optimize system parameters quickly breaks down, as the types of translations that require less post-editing can have quite different characteristics from those that receive high BLEU scores. Muhammad AHSAN A short Summary
  48. 48. Introduction Adequacy and Post-editing Utility Experiments Conclusion Experimental Results Conclusion Empirical analysis of challenges regarding predicting the utility of automatic translations for human post-editors. BLEU metric to optimize system parameters quickly breaks down, as the types of translations that require less post-editing can have quite different characteristics from those that receive high BLEU scores. Fundamental differences between MT as a final product and MT as an intermediate step for human translators. Muhammad AHSAN A short Summary
  49. 49. Introduction Adequacy and Post-editing Utility Experiments Conclusion Experimental Results Conclusion Empirical analysis of challenges regarding predicting the utility of automatic translations for human post-editors. BLEU metric to optimize system parameters quickly breaks down, as the types of translations that require less post-editing can have quite different characteristics from those that receive high BLEU scores. Fundamental differences between MT as a final product and MT as an intermediate step for human translators. Experiments reveals shortcomings of present automatic metrics when predicting translation utility. Muhammad AHSAN A short Summary
  50. 50. Introduction Adequacy and Post-editing Utility Experiments Conclusion Experimental Results Conclusion Empirical analysis of challenges regarding predicting the utility of automatic translations for human post-editors. BLEU metric to optimize system parameters quickly breaks down, as the types of translations that require less post-editing can have quite different characteristics from those that receive high BLEU scores. Fundamental differences between MT as a final product and MT as an intermediate step for human translators. Experiments reveals shortcomings of present automatic metrics when predicting translation utility. Encouragement for machine translation community to consider the above challenges. Muhammad AHSAN A short Summary
  51. 51. Introduction Adequacy and Post-editing Utility Experiments References Important References are as under Muhammad AHSAN A short Summary Conclusion
  52. 52. Introduction Adequacy and Post-editing Utility Experiments Conclusion References Important References are as under Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the Role of BLEU in Machine Translation Research. In Proc. of EACL 2006. Muhammad AHSAN A short Summary
  53. 53. Introduction Adequacy and Post-editing Utility Experiments Conclusion References Important References are as under Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the Role of BLEU in Machine Translation Research. In Proc. of EACL 2006. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. 2007. (Meta-) Evaluation of Machine Translation. In Proc. of ACL WMT 2007. Muhammad AHSAN A short Summary
  54. 54. Introduction Adequacy and Post-editing Utility Experiments Conclusion References Important References are as under Chris Callison-Burch, Miles Osborne, and Philipp Koehn. 2006. Re-evaluating the Role of BLEU in Machine Translation Research. In Proc. of EACL 2006. Chris Callison-Burch, Cameron Fordyce, Philipp Koehn, Christof Monz, and Josh Schroeder. 2007. (Meta-) Evaluation of Machine Translation. In Proc. of ACL WMT 2007. Chris Callison-Burch, Philipp Koehn, Christof Monz, Kay Peterson, and Omar Zaidan. 2011. Findings of the 2011 Joint Workshop on Statistical Machine Translation. In Proc. of ACL WMT 2011. Muhammad AHSAN A short Summary

×