Increasing interpreting needs a more objective and automatic measurement. We hold a basic idea that 'translating means translating meaning' in that we can assessment interpretation quality by comparing the
meaning of the interpreting output with the source input. That is, a translation unit of a 'chunk' named
Frame which comes from frame semantics and its components named Frame Elements (FEs) which comes
from Frame Net are proposed to explore their matching rate between target and source texts. A case study in this paper verifies the usability of semi-automatic graded semantic-scoring measurement for human
simultaneous interpreting and shows how to use frame and FE matches to score. Experiments results show that the semantic-scoring metrics have a significantly correlation coefficient with human judgment.
GENERATING SUMMARIES USING SENTENCE COMPRESSION AND STATISTICAL MEASURESijnlc
In this paper, we propose a compression based multi-document summarization technique by incorporating
word bigram probability and word co-occurrence measure. First we implemented a graph based technique
to achieve sentence compression and information fusion. In the second step, we use hand-crafted rule
based syntactic constraint to prune our compressed sentences. Finally we use probabilistic measure while
exploiting word co-occurrence within a sentence to obtain our summaries. The system can generate summaries for any user-defined compression rate.
An Approach To Automatic Text Summarization Using Simplified Lesk Algorithm A...ijctcm
Text Summarization is a way to produce a text, which contains the significant portion of information of the
original text(s). Different methodologies are developed till now depending upon several parameters to find
the summary as the position, format and type of the sentences in an input text, formats of different words,
frequency of a particular word in a text etc. But according to different languages and input sources, these
parameters are varied. As result the performance of the algorithm is greatly affected. The proposed
approach summarizes a text without depending upon those parameters. Here, the relevance of the
sentences within the text is derived by Simplified Lesk algorithm and WordNet, an online dictionary. This
approach is not only independent of the format of the text and position of a sentence in a text, as the
sentences are arranged at first according to their relevance before the summarization process, the
percentage of summarization can be varied according to needs. The proposed approach gives around 80%
accurate results on 50% summarization of the original text with respect to the manually summarized result,
performed on 50 different types and lengths of texts. We have achieved satisfactory results even upto 25%
summarization of the original text.
Conceptual framework for abstractive text summarizationijnlc
As the volume of information available on the Internet increases, there is a growing need for tools helping users to find, filter and manage these resources. While more and more textual information is available on-line, effective retrieval is difficult without proper indexing and summarization of the content. One of the possible solutions to this problem is abstractive text summarization. The idea is to propose a system that will accept single document as input in English and processes the input by building a rich semantic graph and then reducing this graph for generating the final summary.
THE ABILITY OF WORD EMBEDDINGS TO CAPTURE WORD SIMILARITIESkevig
Distributed language representation has become the most widely used technique for language representation in various natural language processing tasks. Most of the natural language processing models that are based on deep learning techniques use already pre-trained distributed word representations, commonly called word embeddings. Determining the most qualitative word embeddings is of crucial importance for such models. However, selecting the appropriate word embeddings is a perplexing task since the projected embedding space is not intuitive to humans.In this paper, we explore different approaches for creating distributed word representations. We perform an intrinsic evaluation of several state-of-the-art word embedding methods. Their performance on capturing word similarities is analysed with existing benchmark datasets for word pairs similarities. The research in this paper conducts a correlation analysis between ground truth word similarities and similarities obtained by different word embedding methods.
presentation from my thesis defense on text summarization, discusses already existing state of art models along with efficiency of AMR or Abstract Meaning Representation for text summarization, we see how we can use AMRs with seq2seq models. We also discuss other techniques such as BPE or Byte Pair Encoding and its effectiveness for the task. Also we see how data augmentation with POS tags and AMRs effect the summarization with s2s learning.
A legal text usually long and complicated, it has some characteristic that make it different from other dailyuse
texts.Then, translating a legal text is generally considered to be difficult. This paper introduces an approach to split a legal sentence based on its logical structure and presents selecting appropriate
translation rules to improve phrase reordering of legal translation. We use a method which divides a English legal sentence based on its logical structure to segment the legal sentence. New features with rich linguistic and contextual information of split sentences are proposed to rule selection. We apply maximum entropy to combine rich linguistic and contextual information and integrate the features of split sentences into the legal translation, tree-based SMT system. We obtain improvements in performance for legal translation from English to Japanese over Moses and Moses-chart systems.
GENERATING SUMMARIES USING SENTENCE COMPRESSION AND STATISTICAL MEASURESijnlc
In this paper, we propose a compression based multi-document summarization technique by incorporating
word bigram probability and word co-occurrence measure. First we implemented a graph based technique
to achieve sentence compression and information fusion. In the second step, we use hand-crafted rule
based syntactic constraint to prune our compressed sentences. Finally we use probabilistic measure while
exploiting word co-occurrence within a sentence to obtain our summaries. The system can generate summaries for any user-defined compression rate.
An Approach To Automatic Text Summarization Using Simplified Lesk Algorithm A...ijctcm
Text Summarization is a way to produce a text, which contains the significant portion of information of the
original text(s). Different methodologies are developed till now depending upon several parameters to find
the summary as the position, format and type of the sentences in an input text, formats of different words,
frequency of a particular word in a text etc. But according to different languages and input sources, these
parameters are varied. As result the performance of the algorithm is greatly affected. The proposed
approach summarizes a text without depending upon those parameters. Here, the relevance of the
sentences within the text is derived by Simplified Lesk algorithm and WordNet, an online dictionary. This
approach is not only independent of the format of the text and position of a sentence in a text, as the
sentences are arranged at first according to their relevance before the summarization process, the
percentage of summarization can be varied according to needs. The proposed approach gives around 80%
accurate results on 50% summarization of the original text with respect to the manually summarized result,
performed on 50 different types and lengths of texts. We have achieved satisfactory results even upto 25%
summarization of the original text.
Conceptual framework for abstractive text summarizationijnlc
As the volume of information available on the Internet increases, there is a growing need for tools helping users to find, filter and manage these resources. While more and more textual information is available on-line, effective retrieval is difficult without proper indexing and summarization of the content. One of the possible solutions to this problem is abstractive text summarization. The idea is to propose a system that will accept single document as input in English and processes the input by building a rich semantic graph and then reducing this graph for generating the final summary.
THE ABILITY OF WORD EMBEDDINGS TO CAPTURE WORD SIMILARITIESkevig
Distributed language representation has become the most widely used technique for language representation in various natural language processing tasks. Most of the natural language processing models that are based on deep learning techniques use already pre-trained distributed word representations, commonly called word embeddings. Determining the most qualitative word embeddings is of crucial importance for such models. However, selecting the appropriate word embeddings is a perplexing task since the projected embedding space is not intuitive to humans.In this paper, we explore different approaches for creating distributed word representations. We perform an intrinsic evaluation of several state-of-the-art word embedding methods. Their performance on capturing word similarities is analysed with existing benchmark datasets for word pairs similarities. The research in this paper conducts a correlation analysis between ground truth word similarities and similarities obtained by different word embedding methods.
presentation from my thesis defense on text summarization, discusses already existing state of art models along with efficiency of AMR or Abstract Meaning Representation for text summarization, we see how we can use AMRs with seq2seq models. We also discuss other techniques such as BPE or Byte Pair Encoding and its effectiveness for the task. Also we see how data augmentation with POS tags and AMRs effect the summarization with s2s learning.
A legal text usually long and complicated, it has some characteristic that make it different from other dailyuse
texts.Then, translating a legal text is generally considered to be difficult. This paper introduces an approach to split a legal sentence based on its logical structure and presents selecting appropriate
translation rules to improve phrase reordering of legal translation. We use a method which divides a English legal sentence based on its logical structure to segment the legal sentence. New features with rich linguistic and contextual information of split sentences are proposed to rule selection. We apply maximum entropy to combine rich linguistic and contextual information and integrate the features of split sentences into the legal translation, tree-based SMT system. We obtain improvements in performance for legal translation from English to Japanese over Moses and Moses-chart systems.
ON THE UTILITY OF A SYLLABLE-LIKE SEGMENTATION FOR LEARNING A TRANSLITERATION...cscpconf
Source and target word segmentation and alignment is a primary step in the statistical learning of a Transliteration. Here, we analyze the benefit of a syllable-like segmentation approach for learning a transliteration from English to an Indic language, which aligns the training set word pairs in terms of sub-syllable-like units instead of individual character units. While this has been found useful in the case of dealing with Out-of-vocabulary words in English-Chinese in the presence of multiple target dialects, we asked if this would be true for Indic languages which are simpler in their phonetic representation and pronunciation. We expected this syllable-like method to perform marginally better, but we found instead that even though our proposed approach improved the Top-1 accuracy, the individual-character-unit alignment model
somewhat outperformed our approach when the Top-10 results of the system were re-ranked using language modeling approaches. Our experiments were conducted for English to Telugu transliteration (our method will apply equally well to most written Indic languages); our training consisted of a syllable-like segmentation and alignment of a large training set, on which we built a statistical model by modifying a previous character-level maximum entropy based Transliteration learning system due to Kumaran and Kellner; our testing consisted of using the same segmentation of a test English word, followed by applying the model, and reranking the resulting top 10 Telugu words. We also report the dataset creation and selection since standard datasets are not available.
We investigate one technique to produce a summary of
an original text without requiring its full semantic in-
terpretation, but instead relying on a model of the topic
progression in the text derived from lexical chains. We
present a new algorithm to compute lexical chains in
a text, merging several robust knowledge sources: the
WordNet thesaurus, a part-of-speech tagger, shallow
parser for the identification of nominal groups, and a
segmentation algorithm
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Two Level Disambiguation Model for Query TranslationIJECEIAES
Selection of the most suitable translation among all translation candidates returned by bilingual dictionary has always been quiet challenging task for any cross language query translation. Researchers have frequently tried to use word co-occurrence statistics to determine the most probable translation for user query. Algorithms using such statistics have certain shortcomings, which are focused in this paper. We propose a novel method for ambiguity resolution, named „two level disambiguation model‟. At first level disambiguation, the model properly weighs the importance of translation alternatives of query terms obtained from the dictionary. The importance factor measures the probability of a translation candidate of being selected as the final translation of a query term. This removes the problem of taking binary decision for translation candidates. At second level disambiguation, the model targets the user query as a single concept and deduces the translation of all query terms simultaneously, taking into account the weights of translation alternatives also. This is contrary to previous researches which select translation for each word in source language query independently. The experimental result with English-Hindi cross language information retrieval shows that the proposed two level disambiguation model achieved 79.53% and 83.50% of monolingual translation and 21.11% and 17.36% improvement compared to greedy disambiguation strategies in terms of MAP for short and long queries respectively.
Improving Neural Abstractive Text Summarization with Prior KnowledgeGaetano Rossiello, PhD
Abstractive text summarization is a complex task whose goal is to generate a concise version of a text without necessarily reusing the sentences from the original source, but still preserving the meaning and the key contents. In this position paper we address this issue by modeling the problem as a sequence to sequence learning and exploiting Recurrent Neural Networks (RNN). Moreover, we discuss the idea of combining RNNs and probabilistic models in a unified way in order to incorporate prior knowledge, such as linguistic features. We believe that this approach can obtain better performance than the state-of-the-art models for generating well-formed summaries.
Cross lingual similarity discrimination with translation characteristicsijaia
In cross-lingual plagiarism detection, the similarity between sentences is the basis of judgment. This paper
proposes a discriminative model trained on bilingual corpus to divide a set of sentences in target language
into two classes according their similarities to a given sentence in source language. Positive outputs of the
discriminative model are then ranked according to the similarity probabilities. The translation candidates
of the given sentence are finally selected from the top-n positive results. One of the problems in model
building is the extremely imbalanced training data, in which positive samples are the translations of the
target sentences, while negative samples or the non-translations are numerous or unknown. We train models
on four kinds of sampling sets with same translation characteristics and compare their performances.
Experiments on the open dataset of 1500 pairs of English Chinese sentences are evaluated by three metrics
with satisfying performances, much higher than the baseline system.
Sentence level sentiment polarity calculation for customer reviews by conside...eSAT Publishing House
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
This paper introduces a novel approach to tackle the existing gap on message translations in dialogue systems. Currently, submitted messages to the dialogue systems are considered as isolated sentences. Thus, missing context information impede the disambiguation of homographs words in ambiguous sentences. Our approach solves this disambiguation problem by using concepts over existing ontologies.
Abstract This paper represents a Semantic Analyzer for checking the semantic correctness of the given input text. We describe our system as the one which analyzes the text by comparing it with the meaning of the words given in the WordNet. The Semantic Analyzer thus developed not only detects and displays semantic errors in the text but it also corrects them. Keywords: Part of Speech (POS) Tagger, Morphological Analyzer, Syntactic Analyzer, Semantic Analyzer, Natural Language (NL)
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
The most integral part of our work is to extract Aspects from User Feedback and associate Sentiment and Opinion terms to them. The dataset we have at our disposal to work upon, is a set of feedback documents for various departments in a Hospital in XML format which have comments represented in tags. It contains about 65000 responses to a survey taken in a Hospital. Every response or comment is treated as a sentence or a set of them. We perform a sentence level aspect and sentiment extraction and we attempt to understand and mine User Feedback data to gather aspects from it. Further to it, we extract the sentiment mentions and evaluate them contextually for sentiment and associate those sentiment mentions with the corresponding aspects. To start with, we perform a clean up on the User Feedback data, followed by aspect extraction and sentiment polarity calculation, with the help of POS tagging and SentiWordNet filters respectively. The obtained sentiments are further classified according to a set of Linguistic rules and the scores are normalized to nullify any noise that might be present. We lay emphasis on using a rule based approach; rules being Linguistic rules that correspond to the positioning of various parts-of-speech words in a sentence.
The project re-implements the architecture of the paper Reasoning with Neural Tensor Networks for Knowledge Base Completion in Torch framework, achieving similar accuracy results with an elegant implementation in a modern language.
Below are some links for further details:
https://github.com/agarwal-shubham/Reasoning-Over-Knowledge-Base
http://darsh510.github.io/IREPROJ/
Unsupervised Quality Estimation Model for English to German Translation and I...Lifeng (Aaron) Han
• Unsupervised Quality Estimation Model for English to German Translation and Its Application in Extensive Supervised Evaluation
o Hindawi Publishing Corporation
Authors: Aaron Li-Feng Han, Derek F. Wong, Lidia S. Chao, Liangye He and Yi Lu
The Scientific World Journal, Issue: Recent Advances in Information Technology. ISSN:1537-744X. SCIE, IF=1.73. http://www.hindawi.com/journals/tswj/aip/760301/
DOCUMENT SUMMARIZATION IN KANNADA USING KEYWORD EXTRACTION cscpconf
The internet has caused a humongous growth in the amount of data available to the common
man. Summaries of documents can help find the right information and are particularly effective
when the document base is very large. Keywords are closely associated to a document as they
reflect the document's content and act as indexes for the given document. In this work, we
present a method to produce extractive summaries of documents in the Kannada language. The
algorithm extracts key words from pre-categorized Kannada documents collected from online
resources. We combine GSS (Galavotti, Sebastiani, Simi) coefficients and IDF (Inverse
Document Frequency) methods along with TF (Term Frequency) for extracting key words and
later use these for summarization. In the current implementation a document from a given category is selected from our database and depending on the number of sentences given by theuser, a summary is generated.
ON THE UTILITY OF A SYLLABLE-LIKE SEGMENTATION FOR LEARNING A TRANSLITERATION...cscpconf
Source and target word segmentation and alignment is a primary step in the statistical learning of a Transliteration. Here, we analyze the benefit of a syllable-like segmentation approach for learning a transliteration from English to an Indic language, which aligns the training set word pairs in terms of sub-syllable-like units instead of individual character units. While this has been found useful in the case of dealing with Out-of-vocabulary words in English-Chinese in the presence of multiple target dialects, we asked if this would be true for Indic languages which are simpler in their phonetic representation and pronunciation. We expected this syllable-like method to perform marginally better, but we found instead that even though our proposed approach improved the Top-1 accuracy, the individual-character-unit alignment model
somewhat outperformed our approach when the Top-10 results of the system were re-ranked using language modeling approaches. Our experiments were conducted for English to Telugu transliteration (our method will apply equally well to most written Indic languages); our training consisted of a syllable-like segmentation and alignment of a large training set, on which we built a statistical model by modifying a previous character-level maximum entropy based Transliteration learning system due to Kumaran and Kellner; our testing consisted of using the same segmentation of a test English word, followed by applying the model, and reranking the resulting top 10 Telugu words. We also report the dataset creation and selection since standard datasets are not available.
We investigate one technique to produce a summary of
an original text without requiring its full semantic in-
terpretation, but instead relying on a model of the topic
progression in the text derived from lexical chains. We
present a new algorithm to compute lexical chains in
a text, merging several robust knowledge sources: the
WordNet thesaurus, a part-of-speech tagger, shallow
parser for the identification of nominal groups, and a
segmentation algorithm
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Two Level Disambiguation Model for Query TranslationIJECEIAES
Selection of the most suitable translation among all translation candidates returned by bilingual dictionary has always been quiet challenging task for any cross language query translation. Researchers have frequently tried to use word co-occurrence statistics to determine the most probable translation for user query. Algorithms using such statistics have certain shortcomings, which are focused in this paper. We propose a novel method for ambiguity resolution, named „two level disambiguation model‟. At first level disambiguation, the model properly weighs the importance of translation alternatives of query terms obtained from the dictionary. The importance factor measures the probability of a translation candidate of being selected as the final translation of a query term. This removes the problem of taking binary decision for translation candidates. At second level disambiguation, the model targets the user query as a single concept and deduces the translation of all query terms simultaneously, taking into account the weights of translation alternatives also. This is contrary to previous researches which select translation for each word in source language query independently. The experimental result with English-Hindi cross language information retrieval shows that the proposed two level disambiguation model achieved 79.53% and 83.50% of monolingual translation and 21.11% and 17.36% improvement compared to greedy disambiguation strategies in terms of MAP for short and long queries respectively.
Improving Neural Abstractive Text Summarization with Prior KnowledgeGaetano Rossiello, PhD
Abstractive text summarization is a complex task whose goal is to generate a concise version of a text without necessarily reusing the sentences from the original source, but still preserving the meaning and the key contents. In this position paper we address this issue by modeling the problem as a sequence to sequence learning and exploiting Recurrent Neural Networks (RNN). Moreover, we discuss the idea of combining RNNs and probabilistic models in a unified way in order to incorporate prior knowledge, such as linguistic features. We believe that this approach can obtain better performance than the state-of-the-art models for generating well-formed summaries.
Cross lingual similarity discrimination with translation characteristicsijaia
In cross-lingual plagiarism detection, the similarity between sentences is the basis of judgment. This paper
proposes a discriminative model trained on bilingual corpus to divide a set of sentences in target language
into two classes according their similarities to a given sentence in source language. Positive outputs of the
discriminative model are then ranked according to the similarity probabilities. The translation candidates
of the given sentence are finally selected from the top-n positive results. One of the problems in model
building is the extremely imbalanced training data, in which positive samples are the translations of the
target sentences, while negative samples or the non-translations are numerous or unknown. We train models
on four kinds of sampling sets with same translation characteristics and compare their performances.
Experiments on the open dataset of 1500 pairs of English Chinese sentences are evaluated by three metrics
with satisfying performances, much higher than the baseline system.
Sentence level sentiment polarity calculation for customer reviews by conside...eSAT Publishing House
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
This paper introduces a novel approach to tackle the existing gap on message translations in dialogue systems. Currently, submitted messages to the dialogue systems are considered as isolated sentences. Thus, missing context information impede the disambiguation of homographs words in ambiguous sentences. Our approach solves this disambiguation problem by using concepts over existing ontologies.
Abstract This paper represents a Semantic Analyzer for checking the semantic correctness of the given input text. We describe our system as the one which analyzes the text by comparing it with the meaning of the words given in the WordNet. The Semantic Analyzer thus developed not only detects and displays semantic errors in the text but it also corrects them. Keywords: Part of Speech (POS) Tagger, Morphological Analyzer, Syntactic Analyzer, Semantic Analyzer, Natural Language (NL)
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
The most integral part of our work is to extract Aspects from User Feedback and associate Sentiment and Opinion terms to them. The dataset we have at our disposal to work upon, is a set of feedback documents for various departments in a Hospital in XML format which have comments represented in tags. It contains about 65000 responses to a survey taken in a Hospital. Every response or comment is treated as a sentence or a set of them. We perform a sentence level aspect and sentiment extraction and we attempt to understand and mine User Feedback data to gather aspects from it. Further to it, we extract the sentiment mentions and evaluate them contextually for sentiment and associate those sentiment mentions with the corresponding aspects. To start with, we perform a clean up on the User Feedback data, followed by aspect extraction and sentiment polarity calculation, with the help of POS tagging and SentiWordNet filters respectively. The obtained sentiments are further classified according to a set of Linguistic rules and the scores are normalized to nullify any noise that might be present. We lay emphasis on using a rule based approach; rules being Linguistic rules that correspond to the positioning of various parts-of-speech words in a sentence.
The project re-implements the architecture of the paper Reasoning with Neural Tensor Networks for Knowledge Base Completion in Torch framework, achieving similar accuracy results with an elegant implementation in a modern language.
Below are some links for further details:
https://github.com/agarwal-shubham/Reasoning-Over-Knowledge-Base
http://darsh510.github.io/IREPROJ/
Unsupervised Quality Estimation Model for English to German Translation and I...Lifeng (Aaron) Han
• Unsupervised Quality Estimation Model for English to German Translation and Its Application in Extensive Supervised Evaluation
o Hindawi Publishing Corporation
Authors: Aaron Li-Feng Han, Derek F. Wong, Lidia S. Chao, Liangye He and Yi Lu
The Scientific World Journal, Issue: Recent Advances in Information Technology. ISSN:1537-744X. SCIE, IF=1.73. http://www.hindawi.com/journals/tswj/aip/760301/
DOCUMENT SUMMARIZATION IN KANNADA USING KEYWORD EXTRACTION cscpconf
The internet has caused a humongous growth in the amount of data available to the common
man. Summaries of documents can help find the right information and are particularly effective
when the document base is very large. Keywords are closely associated to a document as they
reflect the document's content and act as indexes for the given document. In this work, we
present a method to produce extractive summaries of documents in the Kannada language. The
algorithm extracts key words from pre-categorized Kannada documents collected from online
resources. We combine GSS (Galavotti, Sebastiani, Simi) coefficients and IDF (Inverse
Document Frequency) methods along with TF (Term Frequency) for extracting key words and
later use these for summarization. In the current implementation a document from a given category is selected from our database and depending on the number of sentences given by theuser, a summary is generated.
This PPT prepared by honorable Md. Mahmudul Alam Sarker Vai. He is working as a Technical Officer in Bangladesh Hand loom Board, Ministry of Textiles & Jute, BTMC Bhaban (4th Floor) 7-9, Kawran Bazar, Dhaka-1215.website:www.bhb.gov.bd
Machine translation evaluation is a very important
activity in machine translation development. Automa
tic
evaluation metrics proposed in literature are inade
quate as they require one or more human reference
translations to compare them with output produced b
y machine translation. This does not always give
accurate results as a text can have several differe
nt translations. Human evaluation metrics, on the o
ther
hand, lacks inter-annotator agreement and repeatabi
lity. In this paper we have proposed a new human
evaluation metric which addresses these issues. Mor
eover this metric also provides solid grounds for
making sound assumptions on the quality of the text
produced by a machine translation.
This paper introduces the state-of-the-art machine translation (MT) evaluation survey that contains both manual and automatic evaluation methods. The traditional human evaluation criteria mainly include the intelligibility, fidelity, fluency , adequacy, comprehension, and in-formativeness. The advanced human assessments include task-oriented measures, post-editing, segment ranking, and extended criteriea, etc. We classify the automatic evaluation methods into two categories , including lexical similarity scenario and linguistic features application. The lexical similarity methods contain edit distance, precision, recall, F-measure, and word order. The linguistic features can be divided into syntactic features and semantic features respectively. The syntactic features include part of speech tag, phrase types and sentence structures, and the semantic features include named entity, synonyms , textual entailment, paraphrase, semantic roles, and language models. Subsequently , we also introduce the evaluation methods for MT evaluation including different correlation scores, and the recent quality estimation (QE) tasks for MT.
This paper differs from the existing works (Dorr et al., 2009; EuroMatrix, 2007) from several aspects, by introducing some recent development of MT evaluation measures, the different classifications from manual to automatic evaluation measures, the introduction of recent QE tasks of MT, and the concise construction of the content. For latest version, please goto: https://arxiv.org/abs/1605.04515
Natural Language Processing: A comprehensive overviewBenjaminlapid1
Natural language processing enhances human-computer interaction by bridging the language gap. Uncover its applications and techniques in this comprehensive overview. Dive in now!
MT SUMMIT13.Language-independent Model for Machine Translation Evaluation wit...Lifeng (Aaron) Han
Authors: Aaron Li-Feng Han, Derek Wong, Lidia S. Chao, Yervant Ho, Yi Lu, Anson Xing, Samuel Zeng
Proceedings of the 14th biennial International Conference of Machine Translation Summit (MT Summit 2013) pp. 215-222. Nice, France. 2 - 6 September 2013. Open tool https://github.com/aaronlifenghan/aaron-project-hlepor (Machine Translation Archive)
Translator-computer interaction in action — an observational process study of...antonellarose
The Journal of Specialised Translation Issue 25 – January 2016
106
Translator-computer interaction in action — an observational
process study of computer-aided translation
Kristine Bundgaard, Tina Paulsen Christensen and Anne Schjoldager, Aarhus
University
https://www.jostrans.org/issue25/art_bundgaard.pdf
Classification of MT-Output Using Hybrid MT-Evaluation Metrics for Post-Editi...aciijournal
Machine translation industry is working well but they have been facing problem in postediting. MT-outputs
do not correct and fluent so minor or major changes need for publishing them. Postediting performs
manually by linguists, which is expensive and time consuming. So we should select good translation for
postediting among all translations. Various MT-evaluation metrics can be used for filter the good
translations for postediting. We have shown the use of various MT-evolution metrics for selection of good
translation and their comparative study.
Classification of MT-Output Using Hybrid MT-Evaluation Metrics for Post-Editi...aciijournal
Machine translation industry is working well but they have been facing problem in postediting. MT-outputs do not correct and fluent so minor or major changes need for publishing them. Postediting performs manually by linguists, which is expensive and time consuming. So we should select good translation for postediting among all translations. Various MT-evaluation metrics can be used for filter the good translations for postediting. We have shown the use of various MT-evolution metrics for selection of good translation and their comparative study
A COMPARATIVE STUDY OF FEATURE SELECTION METHODSijnlc
Text analysis has been attracting increasing attention in this data era. Selecting effective features from datasets is a particular important part in text classification studies. Feature selection excludes irrelevant features from the classification task, reduces the dimensionality of a dataset, and improves the accuracy and performance of identification. So far, so many feature selection methods have been proposed, however,
it remains unclear which method is the most effective in practice. This article focuses on evaluating and comparing the available feature selection methods in general versatility regarding authorship attribution problems and tries to identify which method is the most effective. The discussions on general versatility of feature selection methods and its connection in selecting the appropriate features for varying data were
done. In addition, different languages, different types of features, different systems for calculating the accuracy of SVM (support vector machine), and different criteria for determining the rank of feature selection methods were used to measure the general versatility of these methods together. The analysis
results indicate the best feature selection method is different for each dataset; however, some methods can always extract useful information to discriminate the classes. The chi-square was proved to be a better method overall.
A hybrid composite features based sentence level sentiment analyzerIAESIJAI
Current lexica and machine learning based sentiment analysis approaches
still suffer from a two-fold limitation. First, manual lexicon construction and
machine training is time consuming and error-prone. Second, the
prediction’s accuracy entails sentences and their corresponding training text
should fall under the same domain. In this article, we experimentally
evaluate four sentiment classifiers, namely support vector machines (SVMs),
Naive Bayes (NB), logistic regression (LR) and random forest (RF). We
quantify the quality of each of these models using three real-world datasets
that comprise 50,000 movie reviews, 10,662 sentences, and 300 generic
movie reviews. Specifically, we study the impact of a variety of natural
language processing (NLP) pipelines on the quality of the predicted
sentiment orientations. Additionally, we measure the impact of incorporating
lexical semantic knowledge captured by WordNet on expanding original
words in sentences. Findings demonstrate that the utilizing different NLP
pipelines and semantic relationships impacts the quality of the sentiment
analyzers. In particular, results indicate that coupling lemmatization and
knowledge-based n-gram features proved to produce higher accuracy results.
With this coupling, the accuracy of the SVM classifier has improved to
90.43%, while it was 86.83%, 90.11%, 86.20%, respectively using the three
other classifiers.
A COMPARATIVE STUDY OF FEATURE SELECTION METHODSkevig
This article focuses on evaluating and comparing the available feature selection methods in general versatility regarding authorship attribution problems and tries to identify which method is the most effective. The discussions on general versatility of feature selection methods and its connection in selecting the appropriate features for varying data were done. In addition, different languages, different types of features, different systems for calculating the accuracy of SVM (support vector machine), and different criteria for determining the rank of feature selection methods were used to measure the general versatility of these methods together. The analysis results indicate the best feature selection method is different for each dataset; however, some methods can always extract useful information to discriminate the classes. The chi-square was proved to be a better method overall.
Natural Language Processing Theory, Applications and Difficultiesijtsrd
The promise of a powerful computing device to help people in productivity as well as in recreation can only be realized with proper human machine communication. Automatic recognition and understanding of spoken language is the first step toward natural human machine interaction. Research in this field has produced remarkable results, leading to many exciting expectations and new challenges. This field is known as Natural language Processing. In this paper the natural language generation and Natural language understanding is discussed. Difficulties in NLU, applications and comparison with structured programming language are also discussed here. Mrs. Anjali Gharat | Mrs. Helina Tandel | Mr. Ketan Bagade "Natural Language Processing Theory, Applications and Difficulties" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-6 , October 2019, URL: https://www.ijtsrd.com/papers/ijtsrd28092.pdf Paper URL: https://www.ijtsrd.com/engineering/computer-engineering/28092/natural-language-processing-theory-applications-and-difficulties/mrs-anjali-gharat
ACL-WMT2013.A Description of Tunable Machine Translation Evaluation Systems i...Lifeng (Aaron) Han
Proceedings of the ACL 2013 EIGHTH WORKSHOP ON STATISTICAL MACHINE TRANSLATION (ACL-WMT 2013), 8-9 August 2013. Sofia, Bulgaria. Open tool https://github.com/aaronlifenghan/aaron-project-lepor & https://github.com/aaronlifenghan/aaron-project-hlepor(ACM digital library, ACL anthology)
THE ABILITY OF WORD EMBEDDINGS TO CAPTURE WORD SIMILARITIESkevig
Distributed language representation has become the most widely used technique for language representation in various natural language processing tasks. Most of the natural language processing models that are based on deep learning techniques use already pre-trained distributed word representations, commonly called word embeddings. Determining the most qualitative word embeddings is of crucial importance for such models. However, selecting the appropriate word embeddings is a perplexing task since the projected embedding space is not intuitive to humans. In this paper, we explore different approaches for creating distributed word representations. We perform an intrinsic evaluation of several state-of-the-art word embedding methods. Their performance on capturing word similarities is analysed with existing benchmark datasets for word pairs similarities. The research in this paper conducts a correlation analysis between ground truth word similarities and similarities obtained by different word embedding methods.
Machine Translation Approaches and Design AspectsIOSR Journals
Machine translation is a sub-field of computational linguistics that investigates the use of software to
translate text or speech from one natural language to another. On a basic level, MT performs simple
substitution of words in one natural language for words in another, but that alone usually cannot produce a
good translation of a text, because recognition of whole phrases and their closest counterparts in the target
language is needed. The paper focuses on Example Based Machine Translation (EBMT) system that translates
sentences from English to Hindi. Development of a machine translation (MT) system typically demands a large
volume of computational resources. For example, rule based MT systems require extraction of syntactic and
semantic knowledge in the form of rules, statistics-based MT systems require huge parallel corpus containing
sentences in the source languages and their translations in target language. Requirement of such computational
resources is much less in respect of EBMT. This makes development of EBMT systems for English to Hindi
translation feasible, where availability of large-scale computational resources is still scarce. Example based
machine translation relies on the database for its translation. The frequency of word occurrence is important for
translation.
COLING 2012 - LEPOR: A Robust Evaluation Metric for Machine Translation with ...Lifeng (Aaron) Han
"LEPOR: A Robust Evaluation Metric for Machine Translation with Augmented Factors"
Publisher: Association for Computational LinguisticsDecember 2012
Authors: Aaron Li-Feng Han, Derek F. Wong and Lidia S. Chao
Proceedings of the 24th International Conference on Computational Linguistics (COLING 2012): Posters, pages 441–450, Mumbai, December 2012. Open tool https://github.com/aaronlifenghan/aaron-project-lepor
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
1. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
DOI: 10.5121/ijnlc.2016.5501 1
SEMI-AUTOMATIC SIMULTANEOUS INTERPRETING
QUALITY EVALUATION
Xiaojun Zhang
Division of Literature and Languages, University of Stirling, Stirling, UK
ABSTRACT
Increasing interpreting needs a more objective and automatic measurement. We hold a basic idea that
'translating means translating meaning' in that we can assessment interpretation quality by comparing the
meaning of the interpreting output with the source input. That is, a translation unit of a 'chunk' named
Frame which comes from frame semantics and its components named Frame Elements (FEs) which comes
from Frame Net are proposed to explore their matching rate between target and source texts. A case study
in this paper verifies the usability of semi-automatic graded semantic-scoring measurement for human
simultaneous interpreting and shows how to use frame and FE matches to score. Experiments results show
that the semantic-scoring metrics have a significantly correlation coefficient with human judgment.
KEYWORDS
Interpreting Quality, Frame, Frame Element, Semantics
1. INTRODUCTION
With the increasing needs of international communication, the massive global interpreting
demands rise in recent years. According to the survey of Common Sense Adversary, the portion
of interpreting market has expanded to 14.68% (on-site interpreting 11.38%, telephone
interpreting 2.22% and interpreting technology 1.08%) among the whole language service market
in 2015 and the interpreting LSPs are sharing the total US$34,778 billion language service cake
with others (DePalma et al., 2015). The huge interpreting demand may call for a more objective
and automatic measurement to evaluate the quality of human interpreting.
The traditional quantitative metrics of interpreting quality (IQ) assessment is human scoring
which is a professional evaluation by interpreting judges focusing on interpreters' performance in
the booth such as fluency and adequacy of their translations, on-site response and interpreting
skills. However, there is poor agreement on what constitutes an acceptable translation. Some
judges regard a translation as unacceptable if a single word choice is suboptimal. At the other end
of the scale, there are judges who will accept any translation that conveys the approximate
meaning of the sentence, irrespective of how many grammatical or stylistic mistakes it contains.
Without specifying more closely what is meant by "acceptable", it is difficult to compare
evaluations.
The other manual assessment method is task-oriented evaluation which is an end-to-end user
expectation test. It is the metrics that try to assess whether interpreting output is good enough that
an audience is able to understand the content of the interpreted speech. In interpreting literature,
maximum expectation (ME) is a normal-used standard to assess IQ which can meet the maximum
requirement of audience to understand the speech content very well. This manual assessment is
very vague, and it is difficult for evaluators to be consistent in their application. For example,
2. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
2
audience A may get her/his maximum expectation from the English interpreting output "Mary
told the cake is to be cut by John. " for s/he just is wondering "Who will cut the cake?" while
audience B may get nothing for s/he is focusing on "Could a cake be told?".
The automatic evaluation metrics of IQ may be borrowed from spoken language translation (SLT)
and machine translation (MT) evaluation. BLEU, METEOR, TER and other metrics can be
introduced to measure human interpreting output. The problem is: these metrics are all based on
references. And, there is no chance for interpreters to re-order their translation segments while
they are doing their interpreting job simultaneously with the speaker. That is to say, it's unfair to
evaluate human interpreting quality with machine translation metrics for interpreting itself is
disfluent (Honel and Schultz, 2005). Speech translation is normally an interactive process, and it
is natural that it should be less than completely automatic (Rayner et al., 2000). At a minimum, it
is clearly reasonable in many contexts to feed back to the audience and permit him or her to abort
translation if the interpreting output was unacceptably bad. Evaluation should take account of this
possibility.
Generalized word posterior probability (GWPP) is often used as a key metric to confidentially
estimate MT quality. Blatz et al. (2004) obtained the GWPP by calculating N-best list based on
some statistical machine translation features such as semantic feature extracted from WordNet,
shallow parsing feature and draw a conclusion that the estimation performance based on GWPP is
better than that based on linguistic features. Ueffing et al. (2003, 2007) employed the methods of
target position window, relative frequency and system models to count the GWPP and verify the
efficiency of MT confidential estimation. Xiong et al. (2010) integrated GWPP with syntactic and
lexical features into a binary classifier based on maximum entropy model to estimate the
correctness and incorrectness of the word in MT hypothesis. Du and Wang (2013) tested different
GWPP applications in MT estimation and showed a better performance when they combined the
features of GWPP and linguistics. GWPP is a word-level confidential estimation metric.
However, a meaningful chunk seems to be a better option to act as a translation unit in practice
than a separate word. Specia et al. (2009) and Bach et al. (2011) adopted more features besides
GWPP to estimate MT quality. Sakamoto et al. (2013) conducted a field test of a simultaneous
interpreting system for face-to-face services to evaluate the ‘solved task ratio’ for tasks, which is
a typical task-based evaluation. The above previous works are all focusing on MT evaluation but
not human translation assessment Actually, there is still no real consensus on how to evaluate
human translation including interpreting automatically. The most common approach is some
version of the following: The system is run on a set of previously unseen data; the results are
stored in text form; someone judges them as acceptable or unacceptable translations; and finally
the system's performance is quoted as the proportion that is acceptable. Zhang (2010) introduced
a fuzzy integrated evaluation method which applies fuzzy mathematics for multi-factor evaluation
in human translation. Here, the computer was only a calculator to perform the acceptable
proportion from human judges. Tian (2010) developed an online automatic scoring English-
Chinese translation system, YanFa, which compares a translation output with reference by
synonym matching, sentence-pattern matching and word similarity calculation respectively.
Yanfa's semantic scoring has been explored on lexical level with resources of HowNet and Cilin.
Both evaluation systems are concentrated on text translation. As to interpreting, there are some
specific features of speech should be considered sufficiently.
Lo and Wu (2011) introduced a semi-automatic metric via semantic frames, MEANT, to assess
translation utility by matching semantic role fillers, producing scores that correlate with human
judgment as well as HTER but much lower labor cost. They (Lo and Wu, 2013) also contrasted
the coefficiency on adequacy of semantic role labels (SRL) metric, syntactic based metric and n-
gram based BLEU metric with human judgment and showed significantly more SRL's effective
3. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
3
and better correlation with human judgment. SRL based metric also serves for MT but not human
interpreting.
Here, we adopt the idea of semantic frame and semantic role based evaluation method and
propose a translation unit of 'meaningful chunk' named 'frame' to explore the matched frames of
target sentence from original sentence (not from reference), and a frame component of 'semantic
role' named 'frame element (FE)' to explore the matched FEs in the target frames from the ones in
the original frames as well. We introduce this semantic-scoring measurement into human
interpreting domain and define two metrics: minimum expectation (MinE) for the frame matching
to meet the minimum expectation of the audience to one's interpreting job and maximum
expectation (MaxE) for the FE matching to meet the maximum expectation of the audience to
one's interpreting job.
The aim of this paper is to verify the usability of automatic semantic-scoring measurement of
human interpreting quality and show how to use frame and FE matches to score. Comparing with
human scoring and automatic MT evaluation metrics, automatic semantic-scoring measurement
has several distinctive features in interpreting quality assessment: (1) Scoring is based on
meaning; (2) Scoring is reference-independent; (3) Scoring is graded to meet different
requirements of audiences and (4) Less re-ordering operation in interpreting. Motivated by the
above features, we make a pilot study on this issue oriented to computer-aided interpreting system
development.
2. FRAME AND FRAME ELEMENT
A frame is a collection of facts that specify “characteristic features, attributes, and functions of a
denotatum, and its characteristic interactions with things necessarily or typically associated with
it.” (Cruse, 2001) The semantic frame comes from frame semantics which extends Charles J.
Fillmore's case grammar (Fillmore, 1968) from linguistic semantics to encyclopaedic knowledge.
The basic idea is that “meanings are relativized to scenes” (Fillmore, 1977). That is, one cannot
understand the meaning of a single word without access to all the essential knowledge that relates
to that word. Thus, a word activates, or evokes, a frame of semantic knowledge relating to the
specific concept it refers to. Frames are based on recurring experiences. The commercial
transaction frame is based on recurring experiences of commercial transactions. Words not only
highlight individual concepts, but also specify a certain perspective from which the frame is
viewed. For example, "sell" views the situation from the perspective of the seller and "buy" from
the perspective of the buyer. In this case the concept frame is applied to verbs like buy with the
intention to represent the relationships between syntax and semantics (Table 1).
Table 1: An example of concept frame buy
BUYER buy GOODS (SELLER) (PRICE)
subject object from for
Angela bought the owl from Pete for $ 10
Eddy bought them for $ 1
Penny bought a bicycle from Stephen
Frame Net (http://framenet.icsi.berkeley.edu/fndrupal/home) is based on the theory of frame
semantics (Baker, 2014). The basic idea is straightforward: that the meanings of most words can
best be understood on the basis of a semantic frame: a description of a type of event, relation, or
entity and the participants in it. For example, the concept of cooking typically involves a person
doing the cooking (Cook), the food that is to be cooked (Food), something to hold the food while
cooking (Container) and a source of heat (Heating_instrument). In the FrameNet project, this is
represented as a frame called Apply_heat, and the Cook, Food, Heating_instrument and Container
4. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
4
are called FEs . Words that evoke this frame, such as fry, bake, boil, and broil, are called lexical
units (LUs) of the Apply_heat frame. The job of FrameNet is to define the frames and to annotate
sentences to show how the FEs fit syntactically around the word that evokes the frame.
In FrameNet, the above sentence "Mary told the cake to be cut by John." will be involved in two
frames (Figure 1) and annotated as the following colorful markers of LUs (black background with
upper case) and FEs (colourful background):
Frame 1: Telling (A Speaker addresses an Addressee with a Message, which may be indirectly referred to as a Topic. )
Annotation: Mary TOLD the cake is to be cut by John.
Frame 2: Cutting (An Agent cuts a Item into Pieces using an Instrument . )
Annotation: The cake is to be CUT by John.
Figure 1: An example of Frame Net annotation
Our naïve idea of simultaneous interpreting quality measurement is that the more frames are
matched with the source sentence the more meaning is transferred in the target interpreting
sentence, and that the more FEs are matched with the source frames the more expectation is
achieved to the target audience.
3. A CASE STUDY
3.1. Data and experiment
We choose a real-situation public speech, inaugural address by President Barack Obama in 2012,
as the input source. The video, transcription and reference text translation of this inaugural are all
acquired on line easily. The video of Chinese interpreting output by a senior interpreter
(professional interpreter) is also obtained in open source. The senior interpreter works for Phoenix
Television, a Hong Kong-based Mandarin and Cantonese-language television broadcast. We got
her interpreting audio from Phoenix TV website (http://v.ifeng.com) and transcribed it manually.
Our experiment begins from the frame and FE annotations of the transcriptions of Obama's
speech and the senior interpreter's voice. We matched the frames and FEs of input and output and
scored its MinE and MaxE. Then we scored other three outputs from junior interpreters
(interpreting learners) and compared with the score of senior one to see the efficiency of this
method. Also, we compare the scores of semantic-scoring and n-gram based BLEU of reference,
different interpreters’ interpreting outputs and MT output with their human scores to view their
correlations.
3.2. Annotation
We annotated the semantic frame and their FEs manually for either there is no reliable semantic
role annotator or the disfluent simultaneous interpreting is not suit for the state-of-art semantic
annotator. As a pilot study we should grantee the gold standard of the testing data.
There are 87 sentences in total in the source text transcribed from the speech. We annotated these
sentences which are similar to Frame Net annotation; for example, the source sentence 20 (SS20)
can be annotated as Figure 2. In the following annotation examples, LUs are highlighted and FEs
are bracketed and marked.
SS20: No single person can train all the math and science teachers, we'll need to equip our children for the future or build the roads
and networks and research labs that will bring new jobs and businesses to our shores.
Frame 1: Needing
Annotation: (No single person can train all the math and science teachers) Requirement, (we)Cognizer'll NEED to (equip our children for the
future or build the roads and networks and research labs that will bring new jobs and businesses to our shores)Dependent.
Frame 2: Capability
5. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
5
Annotation: (No single person)Entity CAN (train all the math and science teachers)Event
Frame 3: Education_teaching
Annotation: TRAIN (all the math and science teachers)Fact
Frame 4: Supply
Annotation: EQUIP (our children)Recipient (for the future)Purpose_of_recipient
Frame 5: Building
Annotation: BUILD (the roads)Created_entity and (networks)Created_entity and (research labs)Created_entity that will bring new jobs and businesses
to our shores.
Frame 6: Bringing
Annotation: (that)Agent will BRING (new jobs)Theme and (businesses)Theme (to our shores)Goal
Figure 2: SS20 annotation
And, the senior interpreter's output of sentence 20 (SI20) can be annotated as Figure 3:
SI20:没有任何一个个人有能力训练出我们后代的教育需要的所有数学和科学教师,或者建造出能把新的工作和商业机会带
给我们的道路、网络、实验室。
Frame 1: Capability
Annotation: (没有任何一个个人)Entity有能力(训练出我们后代的教育需要的所有数学和科学教师)Event,或者(建造出能把新的工
作和商业机会带给我们的道路、网络、实验室)Event。
Frame 2: Existence
Annotation: 没有(任何一个个人)Entity
Frame 3: Education_teaching
Annotation: 训练出(我们后代的教育需要的所有数学和科学教师)Fact
Frame 4: Needing
Annotation: (我们后代的教育)Cognizer需要
Frame 5: Building
Annotation: 建造出能把新的工作和商业机会带给我们的(道路)Created_entity、(网络)Created_entity、(实验室)Created_entity
Frame 6: Bringing
Annotation: 把(新的工作)Theme和(商业机会)Theme带给(我们)Goal
Figure 3: SI20 annotation
For the first junior interpreter's output of sentence 20 (JI0120), we annotated it as Figure 4:
JI0120: 没有一个单独的人可以培训美国的科学家,可以保护美国孩子们的未来,不可能把我们的工作保护在我们的国土
内。
Frame 1: Capability
Annotation: (没有一个单独的人)Entity可以(培训美国的科学家)Event,可以(保护美国孩子们的未来)Event,不可能(把我们的工作
保护在我们的国土内)Event。
Frame 2: Existence
Annotation: 没有(一个单独的人)Entity
Frame 3: Education_teaching
Annotation: 培训(美国的科学家)Student
Frame 4: Protecting
Annotation: 保护(美国孩子们的未来)Asset
Frame 5: Protecting
Annotation: 把(我们的工作)Asset保护(在我们的国土内)Place
Figure 4: JI0120 annotation
JI0220 and JI0320 are annotated as Figure 5 and Figure 6:
JI0220: 没有一个单独的人能够训练所有的数学和科学家。我们需要为将来装备我们的孩子和实验室,这将带来新的工作和
商业。
Frame 1: Capability
6. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
6
Annotation: (没有一个单独的人)Entity能够(训练所有的数学和科学家)Event
Frame 2: Existence
Annotation: 没有(一个单独的人)Entity
Frame 3: Education_teaching
Annotation: 培练(所有的数学和科学家)Student
Frame 4: Needing
Annotation: (我们)Cognizer需要(为将来装备我们的孩子和实验室)Dependent
Frame 5: Supply
Annotation: (为将来) Purpose_of_Recipient装备(我们的孩子和实验室)Recipient
Frame 6: Bringing
Annotation: (这)Agent将带来(新的工作和商业)Theme
Figure 5: JI0220 annotation
JI0320: 没有一个人能够训练科学家。我们需要增强孩子,修建实验室,这将给我们的海岸带来新的工作机会。
Frame 1: Capability
Annotation: (没有一个人)Entity能够(训练科学家)Event
Frame 2: Existence
Annotation: 没有(一个人)Entity
Frame 3: Education_teaching
Annotation: 培训(科学家)Student
Frame 4: Needing
Annotation: (我们)Cognizer需要(增强孩子)Dependent,(修建实验室)Dependent
Frame 5: Cause_change_of_strenghth
Annotation: 增强(孩子)Patient
Frame 6: Building
Annotation: 修建(实验室)Create_entity
Frame 7: Bringing
Annotation: (这)Agent将给(我们的海岸)Goal带来(新的工作机会)Theme
Figure 6: JI0320 annotation
3.3. Evaluation metrics
We generate the annotation list for each source sentence and its interpreting outputs of different
interpreters. Take sentence 20 as an example, the annotation lists are in Table 2:
Table 2: Annotation list of sentence 20
Frame Frame Element (FE)
SS20 Needing Requirement, Cognizer, Dependent
Capability Entity, Event
Education_teaching Fact
Supply Recipient, Purpose_of_recipient
Building Created_entity, Created_entity, Created_entity
Bringing Agent, Theme, Theme, Goal
SI20 Capability Entity, Event, Event
Existence Entity
Education_teaching Fact
Needing Cognizer
Building Created_entity, Created_entity, Created_entity
Bringing Theme, Theme, Goal
JI0120 Capability Entity, Event, Event, Event
Existence Entity
Education_teaching Student
Protecting Asset
Protecting Asset, Place
JI0220 Capability Entity, Event
7. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
7
Existence Entity
Education_teaching Student
Needing Cognizer, Dependent
Supply Purpose_of_Recipient, Recipient
Bringing Agent, Theme
JI0320 Capability Entity, Event
Existence Entity
Education_teaching Student
Needing Cognizer, Dependent, Dependent
Cause_of_strength Patient
Building Create_entity
Bringing Agent, Goal, Theme
We set two metrics to score the interpreting outputs: MinE and MaxE. MinE refers to the
expectation value which can meet the audiences' 'basic' requirement, that is, the frames of output
can match the most frames of speaker's input. MaxE refers to the expectation value which can
meet the audiences' 'advanced' requirement, that is, the FEs of output frames can match the most
FEs of speaker's input frames. To qualify the above metrics, we define the both in terms of F-
score that balance each precision and recall analysis. Precision of MinE (PMinE) counts the number
of matched frames (Nm)in the total frames of the target sentence (Nt), as
t
m
MinE
N
N
P =
(1)
Precision of MaxE (PMaxE) counts the number of matched FEs in the matched frames (nm)in the
total FEs of the target sentence frames (nt), as
∑=
matched t
m
MaxE
n
n
P
(2)
Recall of MinE (RMinE) counts the number of matched frames (Nm)in the total frames of the
source sentence (Ns), as
s
m
MinE
N
N
R =
(3)
Recall of MaxE (RMaxE) counts the number of matched FEs in the matched frames (nm)in the
total FEs of the source sentence frames (ns), as
∑=
matched s
m
MaxE
n
n
R
(4)
F-score of MinE (FMinE) is the balance of precision of MinE and recall of MinE, as
MinEMinE
MinEMinE
MinE
RP
RP
F
+
××
=
2
(5)
F-score of MaxE (FMaxE) is the balance of precision of MaxE and recall of MaxE, as
MaxEMaxE
MaxEMaxE
MaxE
RP
RP
F
+
××
=
2
(6)
8. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
8
In the example of SI20, the number of senior interpreter's matched frames (Nm) is 5, there are 6
frames in the target sentence (Nt), so for this sentence the precision of MinE (PMinE) is 5/6; there
are 6 frames of the source sentence (Ns), so recall of MinE (RMinE) of this sentence to the senior
interpreter is 5/6, and its F-score (FMinE) is 0.83. Accordingly, in this example, the number of
matched FEs in the matched frames (nm) is 10, there are 12 FEs in the target sentence frames (nt),
so, its precision of MaxE (PMaxE) is 10/12; there are 15 FEs in the source sentence frames (ns), so
recall of MaxE (RMaxE) of this sentence to the senior interpreter is 10/15, and its F-score (FMaxE) is
0.74.
Table 3: Scores of MinE and MaxE of sentence 20
SI20 JI0120 JI0220 JI0320
PMinE 0.83 0.40 0.83 0.71
RMinE 0.83 0.33 0.83 0.83
FMinE 0.83 0.36 0.83 0.77
PMaxE 0.83 0.22 0.80 0.66
RMaxE 0.67 0.13 0.53 0.53
FMaxE 0.74 0.16 0.64 0.59
Hence, we obtain PMinE, RMinE, FMinE, PMaxE, RMaxE and FMaxE of this sentence to senior interpreter
and three junior interpreters respectively as Table 3. Table 4 is the average sentence-level F-
scores of MinE and MaxE for the four interpreting outputs:
Table 4: The average F-scores of MinE and MaxE
SI JI01 JI02 JI03
FMinE 0.82 0.41 0.78 0.73
FMaxE 0.75 0.25 0.66 0.54
There are two issues in the metric MinE: one is the order of the matched frames pairs, and the
other is the similarity of matched frames. The former is not a real issue for the interpreters has the
feasibility to deal with their output in different order. The latter should relate to the second metric,
MaxE.
It seems so cruel to the matched FEs in the unmatched frames that they will be counted zero in
the formula. Actually that's the unbalanced points in human scoring. For example, in the frame of
'Existence' of SI20 and JI0120, the FE 'Entity' matches the one in the frame of 'Capability' of SS20.
Unfortunately, the chunk "No single person" is not meaningful in the source language and it is not
a dependent frame even though its translation in the target language, "没有任何一个人", can be
annotated as the frame 'Existence'.
To the interpreting outputs, there are other two issues in the metric MaxE: one is the repetitive or
added FEs in one frame, and the other is keywords mistranslation. As we mentioned that
interpreting is disfluent, repetitive expressions are always occurred in one's interpreting, as well
as some missing expressions. The missing expressions mean the missing FEs to reduce the MaxE
value in scoring; while to the repetitive ones, or sometimes due to the experienced processing by
senior interpreters, the added expresses which can make output more smooth and fluent will add
the number of FEs in the target fram. For example, in the sentence 12, annotations as Figure 7:
9. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
9
SS12: Together, we determined that a modern economy requires railroads and highways to speed travel and commerce, schools and
colleges to train our workers.
Frame 1: Coming_to_belive
Frame 2: Needing
Annotation: (a modern economy)Cognizer REQUIRES (railroads and highways)Requirement (to speed travel)Dependent and (commerce,
schools and colleges)Requirement (to train our workers)Dependent
Frame 3: Speed-description
Frame 4: Education_teaching
SI12: 现代的民主国家需要我们建立强大的基础设施,建立强大的学校,培训我们的工人。
Frame 1: Needing
Annotation: (现代的民主国家)Cognizer需要(我们)Requirment(建立强大的基础设施)Dependent,(建立强大的学校)Dependent,(培训我们的工
人)Dependent
Frame 2: Building
Frame 3: Education_teaching
Figure 7: Annotations of sentence 12
In the matched frame 'Needing', there are 5 FEs in each of the original frame and target frame.
But the FE 'Dependent' in target frame occurs 3 times while twice in the original sentence in that
the interpreter made some smoothening treatment in his/her interpreting, which makes a trouble
for us to count recall of this FE (3/2?). In this case, we normalized that recall in the range of
[0,1]. That means, the matched FE, 'Dependent', in the above example, will be count only once by
cutting off 1 FE in the target frame.
Distinguished with text translation, interpreting quality depends on the meaning transferring of
several keywords in the speech. Terminologies, human or place names, time expressions,
numbers (values, figures, amounts...) are the main four categories keywords. If these keywords
were misunderstood by the audience, the interpreting is failure, even though their FEs were
matched with the original frame. Hence, we set a penalty in scoring, that is, if one keyword was
mistranslated, its matched FE number will reduce 1. For example, in Figure 8:
SS42: The commitments we make to each other through Medicare and Medicaid and Social Security, these things do not sap our
initiative, they strengthen us.
Frame 1: Cause_change_of_strength
Frame 2: Cause_change_of_strength
Frame 3: Commitment
Annotation: THE COMMITMENTS (we)Speaker MAKE to (each other)Addressee (through Medicare and Medicaid and Social
Security)Manner
JI02 42: 我们通过一些体系对彼此有承诺,这些东西不会挫伤我们的积极性,反而会使我们得更强大。
Frame 1: Commitment
Annotation: (我们)Speaker(通过一些体系)Manner对(彼此)Addressee有承诺
Frame 2: Cause_change_of_strength
Frame 3: Cause_change_of_strength
Figure 8: Annotations of sentence 42
In the matched frame 'Commitment', the source and the target frame share the same FE, 'Manner'
and the output seems perfect. But the target 'Manner', "一些体系 (some systems)" , is more
general in meaning than the source 'Manner', "Medicare and Medicaid and Social Security",
which are all keywords in Obama's speech. In our scoring, this FE is not matched.
3.4. Correlation with human scoring
We use the Spearman's rank correlation coefficient ρ to measure the correlation of the semantic-
scoring metrics with the human judgement and n-gram-based BLEU evaluation at sentence-level
10. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
10
and took average on the whole speech interpreting data. We invite a professional interpreting
teacher (English to Chinese) as a judge to score all the outputs from reference, senior interpreter,
three junior interpreters and MT. We obtained the reference from an open-source language-
learning website (http://news.iciba.com/study/listen/1554958.shtml) and MT output from Google
Translate. The Spearman's rank correlation coefficient ρ can be calculated using the following
equation:
)1(
6
1 2
2
−
−=
∑
nn
di
ρ (7)
where di is the difference between the ranks of the evaluation metrics and the human judgement
over of system i and n is the number of systems. The range of possible values of correlation
coefficient is [-1,1], where 1 means the systems are ranked in the same order as the human
judgement and -1 means the systems are ranked in the reverse order as the human judgment. The
higher the value for ρ indicates the more similar the ranking by the evaluation metric to the
human judgment.
Table 5 shows the raw scores of example sentence 20 under our semantic-scoring metrics, MinE
and MaxE, sentence-level BLEU and human scores with the corresponding ranks assigned to each
of the systems.
Table 5: MinE/MaxE vs. BLEU in correlation with human judgment of example sentence 20
Reference20 SI20 MT20
FMinE
Score 0.85 0.83 0.77
Rank 1 2 3
FMaxE
Score 0.80 0.74 0.59
Rank 1 2 3
BLEU
Score 1.00 0.13 0.14
Rank 1 3 2
Human
scoring
Score 90 80 60
Rank 1 2 3
Our results show that the proposed semantic-scoring metrics have higher correlation with the
human judgement than BLEU metric. Table 6 compares the average sentence-level ρ for our
proposed MinE, MaxE with BLEU. The correlation coefficient for MinE metric is 0.71 and MaxE
is 0.73, while that for BLEU is 0.24. Our proposed metrics is significantly better than BLEU.
Table 6: Average sentence-level correlation for the evaluation metrics
Metric Correlation with human judgment
MinE 0.71
MaxE 0.83
BLEU 0.24
5. CONCLUSIONS
"Translating means translating meaning. (Nida, 1986) "Comparing with text translation,
interpreting has less distinctive features such as genre (literature or technical?), culture
(domestication or foreignization?), sociological factors (post-colony or femelism?). The only
thing what interpreters are focusing on is transferring the meaning of speaker's speech to the
audience. On language structure, interpreting, especially, simultaneous interpreting allows little
time for the interpreters in the booth to organize their outputs in the normal way as the target
languages does. They always output their translation as the source language structure. So, the
meaning-focused and less re-ordering give a possibility to evaluate the interpreting automatically
and semantically.
11. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
11
Interpreting quality evaluation or estimation is an important part to the study of computer-aided
interpreting study. We propose the semantic-scoring evaluation metrics which shows a significant
advantage in correlation coefficient with human judgment. As a case study, we satisfied the
experiment results. However, it is still a long way to reach our goal of fully automatically
semantic-scoring interpreting outputs.
ACKNOWLEDGEMENTS
This work is partly supported by Centre of Translation Studies of GDUFS project (Grant No.: CTS201501)
and the Open Project Program of National Laboratory of Pattern Recognition (Grant No.: 201407353).
REFERENCES
[1] DePalma, D. A., Hegde, V., Pielmeier, H. and Robert G. The Language Services Market: 2015,
Common Sense Advisory, Inc., Cambridge (USA), 2015.
[2] Honal, M. and Schultz, T. “Automatic Disfluency Removal on Recognized Spontaneous Speech
Rapid Adaptation to Speaker-dependent Disfluencies”, Proceedings of International Conference on
Acoustics, Speech, and Signal Processing, Vol. 1, 2005, 969-972.
[3] Rayner, M., Carter, D., Bouillon, P., Digalakis, V. and Viren, M. The Spoken Language Translator,
Cambridge University Press, Cambridge (UK), 2000.
[4] Zhang, X. “The Fuzzy Integrated Evaluate Method of Translation Quality”, Proceedings of
International Conference on Fuzzy Systems and Knowledge Discovery, Vol. 2, 2010, 502-506.
[5] Tian, Y. “YanFa: An Online Automatic Scoring and Intelligent Feedback System of Student English-
Chinese Translation”, Proceedings of the 23rd International Conference on Computational
Linguistics, Demo Vol., 2010, 25-28.
[6] Blatz, J., Fitzgeral, E., Foster, G., Gandrabur, S., Goutte, C., Kulesza, A., Sanchis, A. and Ueffing, N.
“Confidence Estamation for Machine Translation”, Proceedings of the 20th International Conference
on Computational Linguistics, 2004, 513-321.
[7] Ueffing, N., Macherey, K. and Ney, H. “Confidence Measures for Statistical Machine Translation”,
Proceedings of Machine Translation Summit IX, Springer-Verlag, New Orleans, 2003, 394-401.
[8] Ueffing, N. and Ney, H. “Word-level Confidence Estimation for Machine Translation”,
Computational Linguistics, 2007, 33(1):9-40.
[9] Xiong, D., Zhang, M. and Li, H. “Error Detection for Statistical Machine Translation Using
Linguistic Features”, Proceedings of the 48th Annual Meeting of the Association for Computational
Linguistics, 2010, 604-611.
[10] Du, J. and Wang, S. “Error Detection for Statistical Machine Translation Based on Feature
Comparison and Maximum Entropy Model Classifier”, Acta Scientiarum Naturalium Universitatis
Pekinensis, 2013, 43(1):801-807.
[11] Specia, L., Cancedda, N., Dymetman, M., Turchi, M. and Cristanini, N. “Estimating the Sentence-
level Quality of Machine Translation Systems”, Proceedings of the 13th The European Association
for Machine Translation, 2009, 28-35.
[12] Bach, N., Huang, F. and AI-Onaizan, Y. “Goodness: A Method for Measuring Machine Translation
Confidence”, Proceedings of the 49th Annual Meeting of the Association for Computational
Linguistics, 2011, 211-219.
[13] Sakamoto, A., Watanabe, N., Kamatani, S. and Sumita, K. “Development of a Simultaneous
Interpreting System for Face-to-face Services and its Evaluation Experiment in Real Situation”,
Proceedings of Machine Translation Summit XIV, 2013, 85-92.
[14] Lo, C. and Wu, D. “MEANT: An Inexpensive, High-accuracy, Semi-automatic Metric for Evaluating
Translation Utility via Semantic Frames”, Proceedings of the 49th Annual Meeting of the Association
for Computational Linguistics, 2011, 220-229.
[15] Lo, C. and Wu, D. “Semantic vs. Syntactic vs. N-gram Structure for Machine Translation
Evaluation”, Proceedings of the 4th Workshop on Syntax and Structure in Statistical Translation,
2010, 52-60.
[16] Cruse, A. “Meaning in Language”, An Introduction to Semantics and Pragmatics (2nd Edition),
Oxford University Press, New York, 2004.
12. International Journal on Natural Language Computing (IJNLC) Vol. 5, No.5, October 2016
12
[17] Fillmore, C. J. “The Case for Case”, Universals in Linguistic Theory. Holt, Rinehart, and Winston,
New York, 1968, 1-88.
[18] Fillmore, C. J. “The Case for Case Reopened”, Syntax and Semantics 8: Grammatical Relations,
Academic Press, New York, 1977, 59-81.
[19] Baker, C. F. “FrameNet: A Knowledge Base for Natural Language Processing”, Proceedings of
Frame Semantics in NLP: A Workshop in Honor of Chuck Fillmore (1929–2014), 2014, 1-5.
[20] Waard, J.D. and Nida, E.A. From One Language to Another, Thomas Nelson Publishers, Tennessee,
1986.
Authors
Dr. Xiaojun Zhang is a lecturer in translation and interpreting studies at Division of
Literature and Languages, Faculty of Art and Humanities in University of
Stirling. His researches include translation technologies such as computer-aided
translation (CAT) and machine translation (MT), and translation and interpreting
studies from their cognitive and psycholinguistic aspects.