"Machine Translation 101" and the Challenge of Patents
•
2 likes•336 views
Delivered at the European Patent Office's annual Patent Information Conference (EPOPIC 2014) November 5th 2014 Warsaw, Poland. In this talk, we give an introduction as to how machine translation works and what makes certain content types and languages more difficult than others.
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (13)
Similar to "Machine Translation 101" and the Challenge of Patents
Similar to "Machine Translation 101" and the Challenge of Patents (20)
Recently uploaded
Recently uploaded (20)
"Machine Translation 101" and the Challenge of Patents
- 1. “Machine Translation 101” And The Challenge of Patents John Tinsley Director / Co-Founder EPOPIC. 5th Nov 2014, Warsaw
- 2. The need for translation 50% of all PCT applications in 2013 came from Asia
- 3. BSc in Computational Linguistics PhD in Machine Translation Language Technology consultant Founder of Iconic Translation Machines Why listen to me? Machine Translation is what I do! The world’s first and only patent specific machine translation system
- 4. § The use of computers to translate from one language into another § The use of computers to automate some, or all, of the translation process § An approach to Machine Translation, where translations for an input are estimated based on previous seen translation examples and associated (inferred) probabilities. § e.g. IPTranslator, Google Translate § Rule-based (or transfer-based): based on linguistic rules • e.g. Systran; Altavista’s Babelfish § Example-based: based on translation examples and inferred linguistic patterns Machine Translation: The Basics Machine Translation = automatic translation Statistical Machine Translation (SMT) Other approaches SMT is now by far the predominant approach
- 5. A corpus (pl. corpora) is a collection of texts, in electronic format, in a single language § document(s) § book(s) Bilingual Corpora a bilingual corpus Note source language = original language or language we’re translating from target language = language we’re translating into A bilingual corpus is a collection of corresponding texts, in multiple languages § a document & its translation § a book in multiple languages § European Parliament proceedings
- 6. Aligned Bilingual Corpora A document-aligned bilingual corpus corresponds on a document level For translation, we required sentence-aligned bilingual corpora § The sentence on line 1 in the source language text corresponds to (i.e. is a translation of) the sentence on line 1 in the target language text etc. § Often referred to as parallel aligned corpora Sentence aligned bilingual parallel corpora are essential for statistical machine translation
- 7. Learning from Previous Translations Suppose we already know (from a sentence-aligned bilingual corpus) that: § “dog” is translated as “perro” § “I have a cat” is translated as “Tengo un gato” We can theoretically translate: § “I have a dog” à “Tengo un perro” § Even though we have never seen “I have a dog” before Statistical machine translation induces information about unseen input, based on previously known translations: § Primarily co-occurrence statistics § Takes contextual information into account
- 8. Statistical Machine Translation § Example of a small sentence-aligned bilingual corpus for English-French
- 9. Statistical Machine Translation § We take some new sentence to translate
- 10. Statistical Machine Translation § From the corpus we can infer possible target (French) translations for various source (English) words § We can then select the most probable translations based on simple frequencies (co-occurrence statistics)
- 11. Statistical Machine Translation Given a previously unseen input sentence, and our collated statistics, we can estimate translation
- 12. Advanced MT All modern approaches are based on building translations for complete sentences by putting together smaller pieces of translation Previous example is very simplistic § In reality SMT systems calculate much more complex statistical models over millions of sentence pairs for a pair of languages § Upwards of 2M sentence pairs on average for large-scale systems § Word-to-word translation probabilities § Phrase-to-phrase translation probabilities § Word order probabilities § Linguistic information (are the words nouns, verbs?) § Fluency of the final output Previous example is very simplistic Other statistics calculated include
- 13. Data is Key For SMT data is key § Information (word/phrase correspondences and associated statistics) is only based on what we have seen before in the data Important that data used to train SMT systems is: § Of sufficient size § avoid sparseness/skewed statistics § Representative and relevant § contains the right type of language § High-quality § absence of misspellings, incorrect alignments etc. § Proofed by human translators training data
- 14. Why is MT Difficult? A word or a phrase can have more than one meaning (ambiguity – lexical or structural) § e.g. “bank”, “dive”, “I saw the man with the telescope” People use language creatively § New words are cropping up all the time Linguistic differences between languages § e.g. structure of Irish sentences vs. structure of English sentences: § “Tá (Is) ocras (hunger) orm (on me)” <-> “I am hungry” There can be more than one way to express the same meaning. § “New York”, “The Big Apple”, “NYC”
- 15. Why is MT Difficult? § Israeli officials are responsible for airport security. § Israel is in charge of the security at this airport. § The security work for this airport is the responsibility of the Israel government. § Israeli side was in charge of the security of this airport. § Israel is responsible for the airport’s security. § Israel is responsible for safety work at this airport. § Israel presides over the security of the airport. § Israel took charge of the airport security. § The safety of this airport is taken charge of by Israel. § This airport’s security is the responsibility of the Israeli security officials.
- 16. No single solution for all languages Number agreement: the house / the houses vs. la maison / les maisons Gender agreement: the house / the cheese vs. la maison / le frommage English - Spanish English - French
- 17. No single solution for all languages English - German English - Chinese 种水果的农民 The farmer who grows fruit [Lit: “grow fruit (particle) farmer”]
- 18. The Challenge of Patents L is an organic group selected from -CH2- (OCH2CH2)n-, -CO-NR'-, with R'=H or C1-C4 alkyl group; n=0-8; Y=F, CF3 … maximum stress of 1.2 to 3.5 N/mm<2> and a maximum elongation of 700 to 1,300% at 0[deg.] C. Long Sentences Technical constructions Largest single document: 249,322 words Longest Sentence: 1,417 words
- 19. The Challenge of Patents Very long sentences as standard Gramma1cally incomplete using nominal and telegraphic style (!) Passive forms are frequent Frequent use of subordinate clauses, par1ciples, implicit constructs Inconsistent and incorrect spelling High use of neologisms Instances of synonymy and polysemy Spurious use of punctua1on Authoring guide for “to be translated” text Patents break almost all of the rules!
- 20. Judge the quality of an MT system by comparing its output against a human-produced “reference” translation § Pros: Quick, cheap, consistent § Cons: Inflexible, cannot be used on ‘new’ input § Pros: Reliable, flexible, multi-faceted (fluency, error analyses, benchmarking) § Cons: Slow, expensive, subjective § Fluency vs. Adequacy Evaluating Machine Translation Quality Automatic Evaluation Human Evaluation Task-Based Evaluation
- 21. Evaluating Machine Translation Quality Task Based Evaluation § Standalone evaluation of MT systems is necessary to get a sense of the overall quality of a system § To determine the ultimate usability of an MT system, intrinsic task-based evaluation is required § Why? Fluency vs. Adequacy Fluency how fluent and grammatically correct the translation output is Adequacy how accurately the translation conveys the meaning of the source Output 1 The big blue house Output 2 The big house red Source La gran casa roja Task-Based Evaluation
- 22. Practical uses of Machine Translation Understand its limitations and you’ll understand its capabilities! No § Translate a patent for filing § Translate literature for publication § Translate marketing materials § Anything mission critical without review Yes § Productivity tool for professional translation § Understand foreign patents § Localisation processes and “controlled’ content § High volume, e.g. eDiscovery
- 23. We provide Machine Translation solutions with Subject Matter Expertise
- 24. We do this using Linguistic Engineering
- 25. An “ensemble” MT architecture
- 26. Data Engineering What is Linguistic Engineering? Pre-processing Post-processing Input Output Training Data
- 27. Data Engineering + Linguistic Engineering An “ensemble” architecture Chinese pre-ordering rules Statistical Post-editing Input Output Training Data Spanish med-device entity recognizer Multi-output Combination Korean pharma tokenizer Patent input classifier Client TM/terminology (optional) Japanese script normalisation German Compounding rules Moses RBMT Moses Moses
- 28. What is the value for users? Specialist solutions deliver more useable outcomes for the user Post-editing For information purposes Multilingual search Increased productivity Extract more meaning Retrieve more relevant results = = =
- 29. How this impacts translation quality 0 5 10 15 20 25 30 35 40 45 50 Iconic Google Systran Portuguese to English
Editor's Notes
- Second point is important. It has different uses and usability. The concept of FAHQMT is no more. Focus is now on HAMT and PEMT. Problems with rule-based is that they didn’t scale You need bilingual experts for each language pair SMT is the predominant approach
- Starting point for all systems is data. The most important aspect is the quality of the data…
- They are essential and the quality is crucial. The translations must be accurate and the alignment must be correct, otherwise we infer the wrong things. Introduce “noise” into our systems.
- How do we use these corpora? It’s all about learning and remembering things we’ve seen before, the same way you might go about translating something
- Ok, so the translation isn’t exactly right here. It should be “Je parle a la fille” but we haven’t seen enough examples (don’t have enough data) for reliable estimates, we’re just going on the counts of the words
- How likely a word is to translate to another word – as you have seen How likely the different phrases are to translate as one another What’s the likelihood a certain word will have a different position in the target sentence Sometimes we take into account linguistic information about the words, is it a verb, then it should go here, articles should proceed nouns, etc. Look at models of the target language and see if what we have produce makes sense (can these words go together in this order?)
- Google Translate aims to be a general system, but what happens when your translating a sports website? Quality issues can be caused by the fact that there’s a lot of other data in their models than sports news. Similarly, if I have a translation system for car manuals, it won’t be any good at translating sports websites. This is reflected in our systems at IPTranslator too where all of our models are built using patents which have been filed in multiple languages to ensure we get the style correct (patents are a bigger fish than this though)
- The simple answer is that language is complex! Which is what makes it difficult to learn but also so interesting at the same time! Who has the telescope, him or I? New words, especially in patents. And new usage of words. The verb “to tweet” didn’t exist so long ago…
- The last piece in the puzzle is understanding the languages you’re developing MT systems for. And that’s not understanding them in isolation – that’s understanding, for each language pair, what the differences are between them, e.g. many of the things we need to look out for when developing English-Spanish translation engines we don’t need to do for French-Spanish translation
- With certain language pairs, things get more complex. The processes that we need to develop are harder to develop, less studied, require smarter people! Chinese, need to identify these DE constructions so we know to move the head noun No tense, going into English, how do we know what tense? There’s no article! We have to generate it! DE particle has many translations, which one! FIRST THINGS FIRST, which ones are the words!? We need to segment the Chinese! ONLY WITH THESE SKILLS CAN YOU EXPLOIT THE TECHNOLOGY TO ITS FULLEST – AND WHAT DO WE GET IN DOING THIS? MT WITH SUBJECT MATTER EXPERTISE
- But of course it’s not just that easy. Patents for example have a range of highly complex linguistic characteristics that make this challenging, both for PROFESSIONAL translators as well as for Translation Software. Lets look for example at this patent – what’s highlighted in blue is a SINGLE sentence, (which is an individual legal claim). Additionally, we have to deal with complex technical constructions such as chemical formulae, alphanumeric sequences, even genomic and amino acid sequences. And then we have patents which introduce a whole new level of complexity on top of the language issues… Patents are hard to read, never mind translate, never mind try to teach a computer how to translate them!
- Sometimes it’s hard to tell whether the translation is bad or that’s simply how the original patent was written
- Commercial machine translation is plagued with misleading marketing with unrealistic claims and promises - Need to manage expectations When I say NO, I mean no in a fully-automatic manner with no human intervention Filing – not when meaning is CRUCIAL Publication – no, there will be errors Marketing – no, not with subtleties, idioms, etc.
- If you are hiring a professional translator for a job, beyond their language skills they also need to have subject matter expertise, particularly for technical content. The same applies to MT technology (and its providers)
- High quality data is essential for most effective approaches to MT. Clean data is engineering to build MT systems. But it is just an ingredient. You still need to cook the data for the specific language, the specific content type and writing style. This varies from language to language, domain to domain. We need to know how to cook it, we need to understand the language, the content, the style and not only take this into account, but make integral to the development process. This is linguistic engineering.
- As a developer, you cannot be dogmatic when it comes to approaches to MT. We’re not a statistical MT vendor, we don’t focus on Moses, we’re not a rule-based MT vendor. We don’t do hybrid MT. We do all of them. We call this an “ensemble” approach. Sometimes we use them all at the same time. Sometimes we only use one. It’s completely dependent on what works best for a given content type, style, and language together. e.g. for Chinese-English patent MT, maybe you need a statistical decoder, with some rules for automatic post-editing Maybe for French-English abstract translation, an SMT system along suffices. Maybe for Japanese-English titles, we can just use some rules, and maybe some machine learning based pre-processes. We study. We learn what ensemble works for a particular configuration and that’s what we implement.
- Existing vendors or MT providers use the follow process – if a client wants a machine translation system for a certain domain, say IT, they provider the vendor with training data and this gets churned through the various generic processes for each language required. The idea is that by pumping in data in the IT domain that an IT machine translation system comes out at the end. It’s true to a certain extent but the reality is that the quality often doesn’t cut the mustard. The problem with the data engineering approach is that you need A LOT of data and many clients simply don’t have it. By being completely reliant on the data, We’ve develop methods to manipulate the machine translation system by designed processes that are highly specific to the content being translated, often technical nuances, terminology etc. that needs to be specially accounted for. ***ALSO need to develop special processes for languages…
- Let’s get rid of the concept of a central MT system – statistical, hybrid or whatever. Yes we have training data and input, we’ll have some output, and some processes, but what is the journey?... Combining these factors is a delicate balance. Something the smallest change can effect things. Sometimes big changes have no effect. It really depends on your training data. That presents a challenge when the training data changes for each system that’s built. I’ll come back to this later…
- General advantages of this approach to MT
- All of these examples are using our IPTranslator systems which have been developed for patent machine translation. First, in terms of MT quality an BLEU scores, here are evaluation results for our Portuguese to English engines across 8 different patent technical areas. Now, while the BLEU scores don’t necessarily have too much meaning by themselves, there’s a clear distinction in the quality of the Iconic output compared to Google Translate and an out-of-the-box Systran engines. These engines are comparable here because we take the assumption that the client has no additional data with which to build an engine from scratch, so we need an “existing” option. These results correlated well with human assessment of adequacy, another of which we can look at here…