This paper describes machine translation systems submitted by the IIITT team for the LoResMT 2021 shared task on translating between English-Marathi and English-Irish, which are low-resource languages. They fine-tuned IndicTrans, a pretrained multilingual NMT model, on additional parallel corpora for English-Marathi translation. For English-Irish, they used a pretrained Helsinki-NLP model. Their systems achieved BLEU scores of 24.2, 25.8 and 34.6 for English-Marathi, Irish-English and English-Irish respectively, ranking highly in the shared task.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Design and Development of a Malayalam to English Translator- A Transfer Based...Waqas Tariq
This paper describes a transfer based scheme for translating Malayalam, a Dravidian language, to English. This system inputs Malayalam sentences and outputs equivalent English sentences. The system comprises of a preprocessor for splitting the compound words, a morphological parser for context disambiguation and chunking, a syntactic structure transfer module and a bilingual dictionary. All the modules are morpheme based to reduce dictionary size. The system does not rely on a stochastic approach and it is based on a rule-based architecture along with various linguistic knowledge components of both Malayalam and English. The system uses two sets of rules: rules for Malayalam morphology and rules for syntactic structure transfer from Malayalam to English. The system is designed using artificial intelligence techniques.
HINDI AND MARATHI TO ENGLISH MACHINE TRANSLITERATION USING SVMijnlc
The document describes a machine transliteration system that transliterates Hindi and Marathi names and words to English using support vector machines (SVM). It segments source language names into phonetic units, and trains an SVM classifier using phonetic units and n-grams as features to label each unit with its English transliteration. The system achieves good accuracy for Hindi-English and Marathi-English transliteration.
Marathi-English CLIR using detailed user query and unsupervised corpus-based WSDIJERA Editor
With rapid growth of multilingual information on the Internet, Cross Language Information Retrieval (CLIR) is becoming need of the day. It helps user to query in their native language and retrieve information in any language. But the performance of CLIR is poor as compared to monolingual retrieval due to lexical ambiguity, mismatching of query terms and out-of-vocabulary words. In this paper, we have proposed an algorithm for improving the performance of Marathi-English CLIR system. The system first finds possible translations of input query in target language, disambiguates them and then gives English queries to search engine for relevant document retrieval. The disambiguation is based on unsupervised corpus-based method which uses English dictionary as additional resource. The experiment is performed on FIRE 2011 (Forum of Information Retrieval Evaluation) dataset using “Title” and “Description” fields as inputs. The experimental results show that proposed approach gives better performance of Marathi-English CLIR system with good precision level.
Information retrieval (IR) system aims to retrieve
relevant documents to a user query where the query is a set of
keywords. Cross-language information retrieval (CLIR) is a
retrieval process in which the user fires queries in one language to
retrieve information from another language. The growing
requirement on the Internet for users to access information
expressed in language other than their own has led to Cross
Language Information Retrieval (CLIR) becoming established as
a major topic in IR.
Survey on Indian CLIR and MT systems in Marathi LanguageEditor IJCATR
Cross Language Information Retrieval (CLIR) deals with retrieving relevant information stored in a language different from
the language of user’s query. This helps users to express the information need in their native languages. Machine translation based (MTbased)
approach of CLIR uses existing machine translation techniques to provide automatic translation of queries. This paper covers the
research work done in CLIR and MT systems for Marathi language in India.
A COMPREHENSIVE ANALYSIS OF STEMMERS AVAILABLE FOR INDIC LANGUAGES ijnlc
Stemming is the process of term conflation. It conflates all the word variants to a common form called as stem. It plays significant role in numerous Natural Language Processing (NLP) applications like morphological analysis, parsing, document summarization, text classification, part-of-speech tagging, question-answering system, machine translation, word sense disambiguation, information retrieval (IR), etc. Each of these tasks requires some pre-processing to be done. Stemming is one of the important building blocks for all these applications. This paper, presents an overview of various stemming techniques, evaluation criteria for stemmers and various existing stemmers for Indic languages.
TUNING DARI SPEECH CLASSIFICATION EMPLOYING DEEP NEURAL NETWORKSIJCI JOURNAL
Recently, many researchers have focused on building and improving speech recognition systems to facilitate and enhance human-computer interaction. Today, Automatic Speech Recognition (ASR) system has become an important and common tool from games to translation systems, robots, and so on. However, there is still a need for research on speech recognition systems for low-resource languages. This article deals with the recognition of a separate word for Dari language, using Mel-frequency cepstral coefficients (MFCCs) feature extraction method and three different deep neural networks including Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Multilayer Perceptron (MLP), and two hybrid models of CNN and RNN. We evaluate our models on our built-in isolated Dari words corpus that consists of 1000 utterances for 20 short Dari terms. This study obtained the impressive result of 98.365% average accuracy.
IJRET : International Journal of Research in Engineering and Technology is an international peer reviewed, online journal published by eSAT Publishing House for the enhancement of research in various disciplines of Engineering and Technology. The aim and scope of the journal is to provide an academic medium and an important reference for the advancement and dissemination of research results that support high-level learning, teaching and research in the fields of Engineering and Technology. We bring together Scientists, Academician, Field Engineers, Scholars and Students of related fields of Engineering and Technology
Design and Development of a Malayalam to English Translator- A Transfer Based...Waqas Tariq
This paper describes a transfer based scheme for translating Malayalam, a Dravidian language, to English. This system inputs Malayalam sentences and outputs equivalent English sentences. The system comprises of a preprocessor for splitting the compound words, a morphological parser for context disambiguation and chunking, a syntactic structure transfer module and a bilingual dictionary. All the modules are morpheme based to reduce dictionary size. The system does not rely on a stochastic approach and it is based on a rule-based architecture along with various linguistic knowledge components of both Malayalam and English. The system uses two sets of rules: rules for Malayalam morphology and rules for syntactic structure transfer from Malayalam to English. The system is designed using artificial intelligence techniques.
HINDI AND MARATHI TO ENGLISH MACHINE TRANSLITERATION USING SVMijnlc
The document describes a machine transliteration system that transliterates Hindi and Marathi names and words to English using support vector machines (SVM). It segments source language names into phonetic units, and trains an SVM classifier using phonetic units and n-grams as features to label each unit with its English transliteration. The system achieves good accuracy for Hindi-English and Marathi-English transliteration.
Marathi-English CLIR using detailed user query and unsupervised corpus-based WSDIJERA Editor
With rapid growth of multilingual information on the Internet, Cross Language Information Retrieval (CLIR) is becoming need of the day. It helps user to query in their native language and retrieve information in any language. But the performance of CLIR is poor as compared to monolingual retrieval due to lexical ambiguity, mismatching of query terms and out-of-vocabulary words. In this paper, we have proposed an algorithm for improving the performance of Marathi-English CLIR system. The system first finds possible translations of input query in target language, disambiguates them and then gives English queries to search engine for relevant document retrieval. The disambiguation is based on unsupervised corpus-based method which uses English dictionary as additional resource. The experiment is performed on FIRE 2011 (Forum of Information Retrieval Evaluation) dataset using “Title” and “Description” fields as inputs. The experimental results show that proposed approach gives better performance of Marathi-English CLIR system with good precision level.
Information retrieval (IR) system aims to retrieve
relevant documents to a user query where the query is a set of
keywords. Cross-language information retrieval (CLIR) is a
retrieval process in which the user fires queries in one language to
retrieve information from another language. The growing
requirement on the Internet for users to access information
expressed in language other than their own has led to Cross
Language Information Retrieval (CLIR) becoming established as
a major topic in IR.
Survey on Indian CLIR and MT systems in Marathi LanguageEditor IJCATR
Cross Language Information Retrieval (CLIR) deals with retrieving relevant information stored in a language different from
the language of user’s query. This helps users to express the information need in their native languages. Machine translation based (MTbased)
approach of CLIR uses existing machine translation techniques to provide automatic translation of queries. This paper covers the
research work done in CLIR and MT systems for Marathi language in India.
A COMPREHENSIVE ANALYSIS OF STEMMERS AVAILABLE FOR INDIC LANGUAGES ijnlc
Stemming is the process of term conflation. It conflates all the word variants to a common form called as stem. It plays significant role in numerous Natural Language Processing (NLP) applications like morphological analysis, parsing, document summarization, text classification, part-of-speech tagging, question-answering system, machine translation, word sense disambiguation, information retrieval (IR), etc. Each of these tasks requires some pre-processing to be done. Stemming is one of the important building blocks for all these applications. This paper, presents an overview of various stemming techniques, evaluation criteria for stemmers and various existing stemmers for Indic languages.
TUNING DARI SPEECH CLASSIFICATION EMPLOYING DEEP NEURAL NETWORKSIJCI JOURNAL
Recently, many researchers have focused on building and improving speech recognition systems to facilitate and enhance human-computer interaction. Today, Automatic Speech Recognition (ASR) system has become an important and common tool from games to translation systems, robots, and so on. However, there is still a need for research on speech recognition systems for low-resource languages. This article deals with the recognition of a separate word for Dari language, using Mel-frequency cepstral coefficients (MFCCs) feature extraction method and three different deep neural networks including Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Multilayer Perceptron (MLP), and two hybrid models of CNN and RNN. We evaluate our models on our built-in isolated Dari words corpus that consists of 1000 utterances for 20 short Dari terms. This study obtained the impressive result of 98.365% average accuracy.
IMPROVING THE QUALITY OF GUJARATI-HINDI MACHINE TRANSLATION THROUGH PART-OF-S...ijnlc
The document proposes using part-of-speech tagging and stemming to improve Gujarati to Hindi machine translation through transliteration. It presents a system that applies stemming and POS tagging to Gujarati text before transliterating to resolve ambiguities. An evaluation of the system on 500 sentences found that transliteration and translation matched for 54.48% of Gujarati words, and overall transliteration efficiency was 93.09%. The approach aims to improve over direct transliteration for the highly inflected Gujarati language.
AUTOMATIC DETECTION AND LANGUAGE IDENTIFICATION OF MULTILINGUAL DOCUMENTSIRJET Journal
The document describes a study that aims to handle the task of detecting offensive language in multilingual documents using machine learning models. The proposed framework consists of three phases: preprocessing text, representing text using BERT models, and classifying text into offensive and non-offensive classes. The study examines different strategies for handling multilingualism, such as creating one classification model for multiple languages or using translation to convert all texts to one language before classification. Experimental results on a bilingual dataset show that the translation-based approach using Arabic BERT achieves over 93% F1-score and 91% precision for offensive language detection in multilingual texts.
ADVERSARIAL GRAMMATICAL ERROR GENERATION: APPLICATION TO PERSIAN LANGUAGEkevig
Grammatical error correction (GEC) greatly benefits from large quantities of high-quality training data.
However, the preparation of a large amount of labelled training data is time-consuming and prone to
human errors. These issues have become major obstacles in training GEC systems. Recently, the
performance of English GEC systems has drastically been enhanced by the application of deep neural
networks that generate a large amount of synthetic data from limited samples. While GEC has extensively
been studied in languages such as English and Chinese, no attempts have been made to generate synthetic
data for improving Persian GEC systems. Given the substantial grammatical and semantic differences of
the Persian language, in this paper, we propose a new deep learning framework to create large enough
synthetic sentences that are grammatically incorrect for training Persian GEC systems. A modified version
of sequence generative adversarial net with policy gradient is developed, in which the size of the model is
scaled down and the hyperparameters are tuned. The generator is trained in an adversarial framework on
a limited dataset of 8000 samples. Our proposed adversarial framework achieved bilingual evaluation
understudy (BLEU) scores of 64.5% on BLEU-2, 44.2% on BLEU-3, and 21.4% on BLEU-4, and
outperformed the conventional supervised-trained long short-term memory using maximum likelihood
estimation and recently proposed sequence labeler using neural machine translation augmentation. This
shows promise toward improving the performance of GEC systems by generating a large amount of
training data.
ADVERSARIAL GRAMMATICAL ERROR GENERATION: APPLICATION TO PERSIAN LANGUAGEkevig
Grammatical error correction (GEC) greatly benefits from large quantities of high-quality training data.
However, the preparation of a large amount of labelled training data is time-consuming and prone to
human errors. These issues have become major obstacles in training GEC systems. Recently, the
performance of English GEC systems has drastically been enhanced by the application of deep neural
networks that generate a large amount of synthetic data from limited samples. While GEC has extensively
been studied in languages such as English and Chinese, no attempts have been made to generate synthetic
data for improving Persian GEC systems. Given the substantial grammatical and semantic differences of
the Persian language, in this paper, we propose a new deep learning framework to create large enough
synthetic sentences that are grammatically incorrect for training Persian GEC systems. A modified version
of sequence generative adversarial net with policy gradient is developed, in which the size of the model is
scaled down and the hyperparameters are tuned. The generator is trained in an adversarial framework on
a limited dataset of 8000 samples. Our proposed adversarial framework achieved bilingual evaluation
understudy (BLEU) scores of 64.5% on BLEU-2, 44.2% on BLEU-3, and 21.4% on BLEU-4, and
outperformed the conventional supervised-trained long short-term memory using maximum likelihood
estimation and recently proposed sequence labeler using neural machine translation augmentation. This
shows promise toward improving the performance of GEC systems by generating a large amount of
training data.
Identification of monolingual and code-switch information from English-Kannad...IJECEIAES
Code-switching is a very common occurrence in social media communication, predominantly found in multilingual countries like India. Using more than one language in communication is known as codeswitching or code-mixing. Some of the important applications of codeswitch are machine translation (MT), shallow parsing, dialog systems, and semantic parsing. Identifying code-switch and monolingual information is useful for better communication in online networking websites. In this paper, we performed a character level n-gram approach to identify monolingual and code-switch information from English-Kannada social media data. We paralleled various machine learning techniques such as naïve Bayes (NB), support vector classifier (SVC), logistic regression (LR) and neural network (NN) on English-Kannada code-switch (EKCS) data. From the proposed approach, it is observed that the character level n-gram approach provides 1.8% to 4.1% of improvement in terms of Accuracy and 1.6% to 3.8% of improvement in F1-score. Also observed that SVC and NN techniques are outperformed in terms of accuracy (97.9%) and F1-score (98%) with character level n-gram.
This document provides an overview of natural language processing (NLP) and discusses some of its applications. It describes how NLP uses machine translation to translate between languages using techniques like bilingual and multilingual translation systems. It also discusses how question answering systems in NLP analyze natural language questions to generate answers and how text summarization automatically creates short summaries of longer documents to present the most relevant information concisely.
TUNING DARI SPEECH CLASSIFICATION EMPLOYING DEEP NEURAL NETWORKSkevig
Recently, many researchers have focused on building and improving speech recognition systems to
facilitate and enhance human-computer interaction. Today, Automatic Speech Recognition (ASR) system
has become an important and commontool from games to translation systems, robots, and so on. However,
there is still a need for research on speech recognition systems for low-resource languages. This article
deals with the recognition of a separate word for Dari language, using Mel-frequency cepstral coefficients
(MFCCs) feature extraction method and three different deep neural networks including Convolutional
Neural Network (CNN), Recurrent Neural Network (RNN), Multilayer Perceptron (MLP). We evaluate our
models on our built-in isolated Dari words corpus that consists of 1000 utterances for 20 short Dari terms.
This study obtained the impressive result of 98.365% average accuracy.
Tuning Dari Speech Classification Employing Deep Neural Networkskevig
Recently, many researchers have focused on building and improving speech recognition systems to facilitate and enhance human-computer interaction. Today, Automatic Speech Recognition (ASR) system has become an important and common tool from games to translation systems, robots, and so on. However, there is still a need for research on speech recognition systems for low-resource languages. This article deals with the recognition of a separate word for Dari language, using Mel-frequency cepstral coefficients (MFCCs) feature extraction method and three different deep neural networks including Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Multilayer Perceptron (MLP). We evaluate our models on our built-in isolated Dari words corpus that consists of 1000 utterances for 20 short Dari terms. This study obtained the impressive result of 98.365% average accuracy.
This paper presents a machine translation system that translates simple assertive English sentences to Marathi sentences. The system performs morphological analysis, part-of-speech tagging, and local word grouping to convert the meaning of the English sentence to the corresponding Marathi sentence. An English to Marathi bilingual dictionary is used for translation. The system aims to help people with primary education understand English words by providing translations to their native Marathi language.
DEVELOPMENT OF PHONEME DOMINATED DATABASE FOR LIMITED DOMAIN T-T-S IN HINDIijaia
Maximum digital information is available to fewer people who can read or understand a particular
language. The corpus is the basis for developing speech synthesis and recognition systems. In India, almost
all speech research and development affiliations are developing their own speech corpora for Hindi
language, which is the first language for more than 200 million people. The primary goal of this paper is to
review the speech corpus created by various institutes and organizations so that the scientists and language
technologists can recognize the crucial role of corpus development in the field of building ASR and TTS
systems. This aim is to bring together all the information related to the recording, volume and quality of
speech data in speech corpus to facilitate the work of researchers in the field of speech recognition and
synthesis. This paper describes development of medium size database for Metro rail passenger information
systems using HMM based technique in our organization for above application. Phoneme is chosen as
basic speech unit of the database. The result shows that a medium size database consisting of 630
utterances with 12,614 words, 11572 tokens of phonemes covering 38 phonemes are generated in our
database and it cover maximum possible phonetic context.
TOP 5 MOST CITED NATURAL LANGUAGE COMPUTING ARTICLES IN 2013kevig
Natural Language Processing is a programmed approach to analyze text that is based on both a set of theories and a set of technologies. This forum aims to bring together researchers who have designed and build software that will analyze, understand, and generate languages that humans use naturally to address computers.
A Novel Approach for Rule Based Translation of English to Marathiaciijournal
This paper presents a design for rule-based machine translation system for English to Marathi language pair. The machine translation system will take input script as English sentence and parse with the help of Stanford parser. The Stanford parser will be used for main purposes on the source side processing, in the machine translation system. English to Marathi Bilingual dictionary is going to be created. The system will take the parsed output and separate the source text word by word and searches for their corresponding target words in the bilingual dictionary. The hand coded rules are written for Marathi inflections and also reordering rules are there. After applying the reordering rules, English sentence will be syntactically reordered to suit Marathi language
A Novel Approach for Rule Based Translation of English to Marathiaciijournal
This paper presents a design for rule-based machine translation system for English to Marathi language
pair. The machine translation system will take input script as English sentence and parse with the help of
Stanford parser. The Stanford parser will be used for main purposes on the source side processing, in the
machine translation system. English to Marathi Bilingual dictionary is going to be created. The system will
take the parsed output and separate the source text word by word and searches for their corresponding
target words in the bilingual dictionary. The hand coded rules are written for Marathi inflections and also
reordering rules are there. After applying the reordering rules, English sentence will be syntactically
reordered to suit Marathi language.
A Novel Approach for Rule Based Translation of English to Marathiaciijournal
This paper presents a design for rule-based machine translation system for English to Marathi language pair. The machine translation system will take input script as English sentence and parse with the help of Stanford parser. The Stanford parser will be used for main purposes on the source side processing, in the machine translation system. English to Marathi Bilingual dictionary is going to be created. The system will take the parsed output and separate the source text word by word and searches for their corresponding target words in the bilingual dictionary. The hand coded rules are written for Marathi inflections and also reordering rules are there. After applying the reordering rules, English sentence will be syntactically reordered to suit Marathi language
Survey of machine translation systems in indiaijnlc
The work in the area of machine translation has been going on for last few decades but the promising
translation work began in the early 1990s due to advanced research in Artificial Intelligence and
Computational Linguistics. India is a multilingual and multicultural country with over 1.25 billion
population and 22 constitutionally recognized languages which are written in 12 different scripts. This
necessitates the automated machine translation system for English to Indian languages and among Indian
languages so as to exchange the information amongst people in their local language. Many usable machine
translation systems have been developed and are under development in India and around the world. The
paper focuses on different approaches used in the development of Machine Translation Systems and also
briefly described some of the Machine Translation Systems along with their features, domains and
limitations.
A Context-based Numeral Reading Technique for Text to Speech Systems IJECEIAES
This paper presents a novel technique for context based numeral reading in Indian language text to speech systems. The model uses a set of rules to determine the context of the numeral pronunciation and is being integrated with the waveform concatenation technique to produce speech out of the input text in Indian languages. For this purpose, the three Indian languages Odia, Hindi and Bengali are considered. To analyze the performance of the proposed technique, a set of experiments are performed considering different context of numeral pronunciations and the results are compared with existing syllable-based technique. The results obtained from different experiments shows the effectiveness of the proposed technique in producing intelligible speech out of the entered text utterances compared to the existing technique even with very less storage and execution time.
A NOVEL APPROACH FOR NAMED ENTITY RECOGNITION ON HINDI LANGUAGE USING RESIDUA...kevig
Many Natural Language Processing (NLP) applications involve Named Entity Recognition (NER) as an important task, where it leads to improve the overall performance of NLP applications. In this paper the Deep learning techniques are used to perform NER task on Hindi text data as it found that as compared to English NER, Hindi language NER is not sufficiently done. This is a barrier for resource-scarce languages as many resources are not readily available. Many researchers use various techniques such as rule based, machine learning based and hybrid approaches to solve this problem. Deep learning based algorithms are being developed in large scale as an innovative approach now a days for the advanced NER models which will give the best results out of it. In this paper we devise a Novel architecture based on residual network architecture for preferably Bidirectional Long Short Term Memory (BiLSTM) with fasttext word embedding layers. For this purpose we use pre-trained word embedding to represent the words in the corpus where the NER tags of the words are defined as the used annotated corpora. BiLSTM Development of an NER system for Indian languages is a comparatively difficult task. In this paper, we have done the various experiments to compare the results of NER with normal embedding and fasttext embedding layers to analyse the performance of word embedding with different batch sizes to train the deep learning models. Here we present a state-of-the-art results with said approach F1 Score measures.
ATAR: Attention-based LSTM for Arabizi transliterationIJECEIAES
A non-standard romanization of Arabic script, known as Arbizi, is widely used in Arabic online and SMS/chat communities. However, since state-of-the-art tools and applications for Arabic NLP expects Arabic to be written in Arabic script, handling contents written in Arabizi requires a special attention either by building customized tools or by transliterating them into Arabic script. The latter approach is the more common one and this work presents two significant contributions in this direction. The first one is to collect and publicly release the first large-scale “Arabizi to Arabic script” parallel corpus focusing on the Jordanian dialect and consisting of more than 25 k pairs carefully created and inspected by native speakers to ensure highest quality. Second, we present ATAR, an ATtention-based LSTM model for ARabizi transliteration. Training and testing this model on our dataset yields impressive accuracy (79%) and BLEU score (88.49).
SENTIMENT ANALYSIS OF MIXED CODE FOR THE TRANSLITERATED HINDI AND MARATHI TEXTSijnlc
The evolution of information Technology has led to the collection of large amount of data, the volume of
which has increased to the extent that in last two years the data produced is greater than all the data ever
recorded in human history. This has necessitated use of machines to understand, interpret and apply data,
without manual involvement. A lot of these texts are available in transliterated code-mixed form, which due
to the complexity are very difficult to analyze. The work already performed in this area is progressing at
great pace and this work hopes to be a way to push that work further. The designed system is an effort
which classifies Hindi as well as Marathi text transliterated (Romanized) documents automatically using
supervised learning methods (KNN), Naïve Bayes and Support Vector Machine (SVM)) and ontology based
classification; and results are compared to in order to decide which methodology is better suited in
handling of these documents. As we will see, the plain machine learning algorithm applications are just as
or in many cases are much better in performance than the more analytical approach.
The Evaluation of a Code-Switched Sepedi-English Automatic Speech Recognition...IJCI JOURNAL
Speech technology is a field that encompasses various techniques and tools used to enable machines to interact with speech, such as automatic speech recognition (ASR), spoken dialog systems, and others, allowing a device to capture spoken words through a microphone from a human speaker. End-to-end approaches such as Connectionist Temporal Classification (CTC) and attention-based methods are the most used for the development of ASR systems. However, these techniques were commonly used for research and development for many high-resourced languages with large amounts of speech data for training and evaluation, leaving low-resource languages relatively underdeveloped. While the CTC method has been successfully used for other languages, its effectiveness for the Sepedi language remains uncertain. In this study, we present the evaluation of the Sepedi-English code-switched automatic speech recognition system. This end-to-end system was developed using the Sepedi Prompted Code Switching corpus and the CTC approach. The performance of the system was evaluated using both the NCHLT Sepedi test corpus and the Sepedi Prompted Code Switching corpus. The model produced the lowest WER of 41.9%, however, the model faced challenges in recognizing the Sepedi only text.
This document is a thesis submitted by David Liebman to the State University of New York at New Paltz for the degree of Master of Science in Computer Science. The goal of the thesis is to create a chatbot using natural language processing and deep learning models. The thesis provides background on recurrent neural networks, transformers, and pre-trained language models like GPT-2. It then describes the experimental design and setup for installing chatbot models on devices like the Raspberry Pi. Several chatbot experiments are conducted using GRU, transformer, and GPT-2 models with discussion of the results.
This document summarizes a thesis on conversational chatbots with memory-based question and answer generation. The thesis was carried out by Mikael Lundell Vinkler and Peilin Yu at the Department of Science and Technology at Linköping University in Norrköping, Sweden. It presents research on developing chatbots that can answer questions based on their memory of previous conversations to have more human-like discussions.
More Related Content
Similar to Attentive_fine-tuning_of_Transformers_for_Translat.pdf
IMPROVING THE QUALITY OF GUJARATI-HINDI MACHINE TRANSLATION THROUGH PART-OF-S...ijnlc
The document proposes using part-of-speech tagging and stemming to improve Gujarati to Hindi machine translation through transliteration. It presents a system that applies stemming and POS tagging to Gujarati text before transliterating to resolve ambiguities. An evaluation of the system on 500 sentences found that transliteration and translation matched for 54.48% of Gujarati words, and overall transliteration efficiency was 93.09%. The approach aims to improve over direct transliteration for the highly inflected Gujarati language.
AUTOMATIC DETECTION AND LANGUAGE IDENTIFICATION OF MULTILINGUAL DOCUMENTSIRJET Journal
The document describes a study that aims to handle the task of detecting offensive language in multilingual documents using machine learning models. The proposed framework consists of three phases: preprocessing text, representing text using BERT models, and classifying text into offensive and non-offensive classes. The study examines different strategies for handling multilingualism, such as creating one classification model for multiple languages or using translation to convert all texts to one language before classification. Experimental results on a bilingual dataset show that the translation-based approach using Arabic BERT achieves over 93% F1-score and 91% precision for offensive language detection in multilingual texts.
ADVERSARIAL GRAMMATICAL ERROR GENERATION: APPLICATION TO PERSIAN LANGUAGEkevig
Grammatical error correction (GEC) greatly benefits from large quantities of high-quality training data.
However, the preparation of a large amount of labelled training data is time-consuming and prone to
human errors. These issues have become major obstacles in training GEC systems. Recently, the
performance of English GEC systems has drastically been enhanced by the application of deep neural
networks that generate a large amount of synthetic data from limited samples. While GEC has extensively
been studied in languages such as English and Chinese, no attempts have been made to generate synthetic
data for improving Persian GEC systems. Given the substantial grammatical and semantic differences of
the Persian language, in this paper, we propose a new deep learning framework to create large enough
synthetic sentences that are grammatically incorrect for training Persian GEC systems. A modified version
of sequence generative adversarial net with policy gradient is developed, in which the size of the model is
scaled down and the hyperparameters are tuned. The generator is trained in an adversarial framework on
a limited dataset of 8000 samples. Our proposed adversarial framework achieved bilingual evaluation
understudy (BLEU) scores of 64.5% on BLEU-2, 44.2% on BLEU-3, and 21.4% on BLEU-4, and
outperformed the conventional supervised-trained long short-term memory using maximum likelihood
estimation and recently proposed sequence labeler using neural machine translation augmentation. This
shows promise toward improving the performance of GEC systems by generating a large amount of
training data.
ADVERSARIAL GRAMMATICAL ERROR GENERATION: APPLICATION TO PERSIAN LANGUAGEkevig
Grammatical error correction (GEC) greatly benefits from large quantities of high-quality training data.
However, the preparation of a large amount of labelled training data is time-consuming and prone to
human errors. These issues have become major obstacles in training GEC systems. Recently, the
performance of English GEC systems has drastically been enhanced by the application of deep neural
networks that generate a large amount of synthetic data from limited samples. While GEC has extensively
been studied in languages such as English and Chinese, no attempts have been made to generate synthetic
data for improving Persian GEC systems. Given the substantial grammatical and semantic differences of
the Persian language, in this paper, we propose a new deep learning framework to create large enough
synthetic sentences that are grammatically incorrect for training Persian GEC systems. A modified version
of sequence generative adversarial net with policy gradient is developed, in which the size of the model is
scaled down and the hyperparameters are tuned. The generator is trained in an adversarial framework on
a limited dataset of 8000 samples. Our proposed adversarial framework achieved bilingual evaluation
understudy (BLEU) scores of 64.5% on BLEU-2, 44.2% on BLEU-3, and 21.4% on BLEU-4, and
outperformed the conventional supervised-trained long short-term memory using maximum likelihood
estimation and recently proposed sequence labeler using neural machine translation augmentation. This
shows promise toward improving the performance of GEC systems by generating a large amount of
training data.
Identification of monolingual and code-switch information from English-Kannad...IJECEIAES
Code-switching is a very common occurrence in social media communication, predominantly found in multilingual countries like India. Using more than one language in communication is known as codeswitching or code-mixing. Some of the important applications of codeswitch are machine translation (MT), shallow parsing, dialog systems, and semantic parsing. Identifying code-switch and monolingual information is useful for better communication in online networking websites. In this paper, we performed a character level n-gram approach to identify monolingual and code-switch information from English-Kannada social media data. We paralleled various machine learning techniques such as naïve Bayes (NB), support vector classifier (SVC), logistic regression (LR) and neural network (NN) on English-Kannada code-switch (EKCS) data. From the proposed approach, it is observed that the character level n-gram approach provides 1.8% to 4.1% of improvement in terms of Accuracy and 1.6% to 3.8% of improvement in F1-score. Also observed that SVC and NN techniques are outperformed in terms of accuracy (97.9%) and F1-score (98%) with character level n-gram.
This document provides an overview of natural language processing (NLP) and discusses some of its applications. It describes how NLP uses machine translation to translate between languages using techniques like bilingual and multilingual translation systems. It also discusses how question answering systems in NLP analyze natural language questions to generate answers and how text summarization automatically creates short summaries of longer documents to present the most relevant information concisely.
TUNING DARI SPEECH CLASSIFICATION EMPLOYING DEEP NEURAL NETWORKSkevig
Recently, many researchers have focused on building and improving speech recognition systems to
facilitate and enhance human-computer interaction. Today, Automatic Speech Recognition (ASR) system
has become an important and commontool from games to translation systems, robots, and so on. However,
there is still a need for research on speech recognition systems for low-resource languages. This article
deals with the recognition of a separate word for Dari language, using Mel-frequency cepstral coefficients
(MFCCs) feature extraction method and three different deep neural networks including Convolutional
Neural Network (CNN), Recurrent Neural Network (RNN), Multilayer Perceptron (MLP). We evaluate our
models on our built-in isolated Dari words corpus that consists of 1000 utterances for 20 short Dari terms.
This study obtained the impressive result of 98.365% average accuracy.
Tuning Dari Speech Classification Employing Deep Neural Networkskevig
Recently, many researchers have focused on building and improving speech recognition systems to facilitate and enhance human-computer interaction. Today, Automatic Speech Recognition (ASR) system has become an important and common tool from games to translation systems, robots, and so on. However, there is still a need for research on speech recognition systems for low-resource languages. This article deals with the recognition of a separate word for Dari language, using Mel-frequency cepstral coefficients (MFCCs) feature extraction method and three different deep neural networks including Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), Multilayer Perceptron (MLP). We evaluate our models on our built-in isolated Dari words corpus that consists of 1000 utterances for 20 short Dari terms. This study obtained the impressive result of 98.365% average accuracy.
This paper presents a machine translation system that translates simple assertive English sentences to Marathi sentences. The system performs morphological analysis, part-of-speech tagging, and local word grouping to convert the meaning of the English sentence to the corresponding Marathi sentence. An English to Marathi bilingual dictionary is used for translation. The system aims to help people with primary education understand English words by providing translations to their native Marathi language.
DEVELOPMENT OF PHONEME DOMINATED DATABASE FOR LIMITED DOMAIN T-T-S IN HINDIijaia
Maximum digital information is available to fewer people who can read or understand a particular
language. The corpus is the basis for developing speech synthesis and recognition systems. In India, almost
all speech research and development affiliations are developing their own speech corpora for Hindi
language, which is the first language for more than 200 million people. The primary goal of this paper is to
review the speech corpus created by various institutes and organizations so that the scientists and language
technologists can recognize the crucial role of corpus development in the field of building ASR and TTS
systems. This aim is to bring together all the information related to the recording, volume and quality of
speech data in speech corpus to facilitate the work of researchers in the field of speech recognition and
synthesis. This paper describes development of medium size database for Metro rail passenger information
systems using HMM based technique in our organization for above application. Phoneme is chosen as
basic speech unit of the database. The result shows that a medium size database consisting of 630
utterances with 12,614 words, 11572 tokens of phonemes covering 38 phonemes are generated in our
database and it cover maximum possible phonetic context.
TOP 5 MOST CITED NATURAL LANGUAGE COMPUTING ARTICLES IN 2013kevig
Natural Language Processing is a programmed approach to analyze text that is based on both a set of theories and a set of technologies. This forum aims to bring together researchers who have designed and build software that will analyze, understand, and generate languages that humans use naturally to address computers.
A Novel Approach for Rule Based Translation of English to Marathiaciijournal
This paper presents a design for rule-based machine translation system for English to Marathi language pair. The machine translation system will take input script as English sentence and parse with the help of Stanford parser. The Stanford parser will be used for main purposes on the source side processing, in the machine translation system. English to Marathi Bilingual dictionary is going to be created. The system will take the parsed output and separate the source text word by word and searches for their corresponding target words in the bilingual dictionary. The hand coded rules are written for Marathi inflections and also reordering rules are there. After applying the reordering rules, English sentence will be syntactically reordered to suit Marathi language
A Novel Approach for Rule Based Translation of English to Marathiaciijournal
This paper presents a design for rule-based machine translation system for English to Marathi language
pair. The machine translation system will take input script as English sentence and parse with the help of
Stanford parser. The Stanford parser will be used for main purposes on the source side processing, in the
machine translation system. English to Marathi Bilingual dictionary is going to be created. The system will
take the parsed output and separate the source text word by word and searches for their corresponding
target words in the bilingual dictionary. The hand coded rules are written for Marathi inflections and also
reordering rules are there. After applying the reordering rules, English sentence will be syntactically
reordered to suit Marathi language.
A Novel Approach for Rule Based Translation of English to Marathiaciijournal
This paper presents a design for rule-based machine translation system for English to Marathi language pair. The machine translation system will take input script as English sentence and parse with the help of Stanford parser. The Stanford parser will be used for main purposes on the source side processing, in the machine translation system. English to Marathi Bilingual dictionary is going to be created. The system will take the parsed output and separate the source text word by word and searches for their corresponding target words in the bilingual dictionary. The hand coded rules are written for Marathi inflections and also reordering rules are there. After applying the reordering rules, English sentence will be syntactically reordered to suit Marathi language
Survey of machine translation systems in indiaijnlc
The work in the area of machine translation has been going on for last few decades but the promising
translation work began in the early 1990s due to advanced research in Artificial Intelligence and
Computational Linguistics. India is a multilingual and multicultural country with over 1.25 billion
population and 22 constitutionally recognized languages which are written in 12 different scripts. This
necessitates the automated machine translation system for English to Indian languages and among Indian
languages so as to exchange the information amongst people in their local language. Many usable machine
translation systems have been developed and are under development in India and around the world. The
paper focuses on different approaches used in the development of Machine Translation Systems and also
briefly described some of the Machine Translation Systems along with their features, domains and
limitations.
A Context-based Numeral Reading Technique for Text to Speech Systems IJECEIAES
This paper presents a novel technique for context based numeral reading in Indian language text to speech systems. The model uses a set of rules to determine the context of the numeral pronunciation and is being integrated with the waveform concatenation technique to produce speech out of the input text in Indian languages. For this purpose, the three Indian languages Odia, Hindi and Bengali are considered. To analyze the performance of the proposed technique, a set of experiments are performed considering different context of numeral pronunciations and the results are compared with existing syllable-based technique. The results obtained from different experiments shows the effectiveness of the proposed technique in producing intelligible speech out of the entered text utterances compared to the existing technique even with very less storage and execution time.
A NOVEL APPROACH FOR NAMED ENTITY RECOGNITION ON HINDI LANGUAGE USING RESIDUA...kevig
Many Natural Language Processing (NLP) applications involve Named Entity Recognition (NER) as an important task, where it leads to improve the overall performance of NLP applications. In this paper the Deep learning techniques are used to perform NER task on Hindi text data as it found that as compared to English NER, Hindi language NER is not sufficiently done. This is a barrier for resource-scarce languages as many resources are not readily available. Many researchers use various techniques such as rule based, machine learning based and hybrid approaches to solve this problem. Deep learning based algorithms are being developed in large scale as an innovative approach now a days for the advanced NER models which will give the best results out of it. In this paper we devise a Novel architecture based on residual network architecture for preferably Bidirectional Long Short Term Memory (BiLSTM) with fasttext word embedding layers. For this purpose we use pre-trained word embedding to represent the words in the corpus where the NER tags of the words are defined as the used annotated corpora. BiLSTM Development of an NER system for Indian languages is a comparatively difficult task. In this paper, we have done the various experiments to compare the results of NER with normal embedding and fasttext embedding layers to analyse the performance of word embedding with different batch sizes to train the deep learning models. Here we present a state-of-the-art results with said approach F1 Score measures.
ATAR: Attention-based LSTM for Arabizi transliterationIJECEIAES
A non-standard romanization of Arabic script, known as Arbizi, is widely used in Arabic online and SMS/chat communities. However, since state-of-the-art tools and applications for Arabic NLP expects Arabic to be written in Arabic script, handling contents written in Arabizi requires a special attention either by building customized tools or by transliterating them into Arabic script. The latter approach is the more common one and this work presents two significant contributions in this direction. The first one is to collect and publicly release the first large-scale “Arabizi to Arabic script” parallel corpus focusing on the Jordanian dialect and consisting of more than 25 k pairs carefully created and inspected by native speakers to ensure highest quality. Second, we present ATAR, an ATtention-based LSTM model for ARabizi transliteration. Training and testing this model on our dataset yields impressive accuracy (79%) and BLEU score (88.49).
SENTIMENT ANALYSIS OF MIXED CODE FOR THE TRANSLITERATED HINDI AND MARATHI TEXTSijnlc
The evolution of information Technology has led to the collection of large amount of data, the volume of
which has increased to the extent that in last two years the data produced is greater than all the data ever
recorded in human history. This has necessitated use of machines to understand, interpret and apply data,
without manual involvement. A lot of these texts are available in transliterated code-mixed form, which due
to the complexity are very difficult to analyze. The work already performed in this area is progressing at
great pace and this work hopes to be a way to push that work further. The designed system is an effort
which classifies Hindi as well as Marathi text transliterated (Romanized) documents automatically using
supervised learning methods (KNN), Naïve Bayes and Support Vector Machine (SVM)) and ontology based
classification; and results are compared to in order to decide which methodology is better suited in
handling of these documents. As we will see, the plain machine learning algorithm applications are just as
or in many cases are much better in performance than the more analytical approach.
The Evaluation of a Code-Switched Sepedi-English Automatic Speech Recognition...IJCI JOURNAL
Speech technology is a field that encompasses various techniques and tools used to enable machines to interact with speech, such as automatic speech recognition (ASR), spoken dialog systems, and others, allowing a device to capture spoken words through a microphone from a human speaker. End-to-end approaches such as Connectionist Temporal Classification (CTC) and attention-based methods are the most used for the development of ASR systems. However, these techniques were commonly used for research and development for many high-resourced languages with large amounts of speech data for training and evaluation, leaving low-resource languages relatively underdeveloped. While the CTC method has been successfully used for other languages, its effectiveness for the Sepedi language remains uncertain. In this study, we present the evaluation of the Sepedi-English code-switched automatic speech recognition system. This end-to-end system was developed using the Sepedi Prompted Code Switching corpus and the CTC approach. The performance of the system was evaluated using both the NCHLT Sepedi test corpus and the Sepedi Prompted Code Switching corpus. The model produced the lowest WER of 41.9%, however, the model faced challenges in recognizing the Sepedi only text.
Similar to Attentive_fine-tuning_of_Transformers_for_Translat.pdf (20)
This document is a thesis submitted by David Liebman to the State University of New York at New Paltz for the degree of Master of Science in Computer Science. The goal of the thesis is to create a chatbot using natural language processing and deep learning models. The thesis provides background on recurrent neural networks, transformers, and pre-trained language models like GPT-2. It then describes the experimental design and setup for installing chatbot models on devices like the Raspberry Pi. Several chatbot experiments are conducted using GRU, transformer, and GPT-2 models with discussion of the results.
This document summarizes a thesis on conversational chatbots with memory-based question and answer generation. The thesis was carried out by Mikael Lundell Vinkler and Peilin Yu at the Department of Science and Technology at Linköping University in Norrköping, Sweden. It presents research on developing chatbots that can answer questions based on their memory of previous conversations to have more human-like discussions.
This paper describes the architecture of Genuine2, an open domain chatbot for Alexa based on generative models. The chatbot uses multiple response generators coordinated by a dialogue manager to improve coherence and topic continuity. Various mechanisms were implemented to control the quality of responses, including toxicity detection, emotion analysis, and avoiding repetitive or incorrect answers. Evaluation results and analysis are also presented.
The document discusses the development of a chatbot using deep learning models like sequence-to-sequence learning with LSTM. It proposes using a movie dialog corpus consisting of over 220,000 conversational exchanges to train the chatbot model. Transfer learning is used with pre-trained word embeddings to improve the model. The chatbot implementation uses an LSTM encoder-decoder architecture in the sequence-to-sequence model.
This document discusses algorithms used in chatbots. It provides background on early chatbots like ELIZA and ALICE that used pattern matching. It then discusses several algorithms commonly used in modern chatbots: sequence-to-sequence models, pattern matching, LSTM, HEIM, naive Bayes, and natural language processing. The document aims to introduce these design techniques to provide an overview of how chatbots have advanced from early rule-based systems to more intelligent designs using machine learning.
The document discusses developing an open domain chatbot using sequence modeling and machine translation techniques. It provides background on early rule-based chatbots and modern data-driven approaches. The proposed methodology collects data, performs word embeddings, uses an encoder-decoder model with attention to generate responses, and evaluates the model using metrics like F1 score.
End-to-end pipeline agility - Berlin Buzzwords 2024Lars Albertsson
We describe how we achieve high change agility in data engineering by eliminating the fear of breaking downstream data pipelines through end-to-end pipeline testing, and by using schema metaprogramming to safely eliminate boilerplate involved in changes that affect whole pipelines.
A quick poll on agility in changing pipelines from end to end indicated a huge span in capabilities. For the question "How long time does it take for all downstream pipelines to be adapted to an upstream change," the median response was 6 months, but some respondents could do it in less than a day. When quantitative data engineering differences between the best and worst are measured, the span is often 100x-1000x, sometimes even more.
A long time ago, we suffered at Spotify from fear of changing pipelines due to not knowing what the impact might be downstream. We made plans for a technical solution to test pipelines end-to-end to mitigate that fear, but the effort failed for cultural reasons. We eventually solved this challenge, but in a different context. In this presentation we will describe how we test full pipelines effectively by manipulating workflow orchestration, which enables us to make changes in pipelines without fear of breaking downstream.
Making schema changes that affect many jobs also involves a lot of toil and boilerplate. Using schema-on-read mitigates some of it, but has drawbacks since it makes it more difficult to detect errors early. We will describe how we have rejected this tradeoff by applying schema metaprogramming, eliminating boilerplate but keeping the protection of static typing, thereby further improving agility to quickly modify data pipelines without fear.
The Ipsos - AI - Monitor 2024 Report.pdfSocial Samosa
According to Ipsos AI Monitor's 2024 report, 65% Indians said that products and services using AI have profoundly changed their daily life in the past 3-5 years.
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataKiwi Creative
Harness the power of AI-backed reports, benchmarking and data analysis to predict trends and detect anomalies in your marketing efforts.
Peter Caputa, CEO at Databox, reveals how you can discover the strategies and tools to increase your growth rate (and margins!).
From metrics to track to data habits to pick up, enhance your reporting for powerful insights to improve your B2B tech company's marketing.
- - -
This is the webinar recording from the June 2024 HubSpot User Group (HUG) for B2B Technology USA.
Watch the video recording at https://youtu.be/5vjwGfPN9lw
Sign up for future HUG events at https://events.hubspot.com/b2b-technology-usa/
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...Social Samosa
The Modern Marketing Reckoner (MMR) is a comprehensive resource packed with POVs from 60+ industry leaders on how AI is transforming the 4 key pillars of marketing – product, place, price and promotions.
The Building Blocks of QuestDB, a Time Series Databasejavier ramirez
Talk Delivered at Valencia Codes Meetup 2024-06.
Traditionally, databases have treated timestamps just as another data type. However, when performing real-time analytics, timestamps should be first class citizens and we need rich time semantics to get the most out of our data. We also need to deal with ever growing datasets while keeping performant, which is as fun as it sounds.
It is no wonder time-series databases are now more popular than ever before. Join me in this session to learn about the internal architecture and building blocks of QuestDB, an open source time-series database designed for speed. We will also review a history of some of the changes we have gone over the past two years to deal with late and unordered data, non-blocking writes, read-replicas, or faster batch ingestion.
1. Attentive fine-tuning of Transformers for
Translation of low-resourced languages
@LoResMT 2021
Karthik Puranik1 Adeep Hande1 Ruba Priyadharshini2 Thenmozi Durairaj3
Anbukkarasi Sampath4 Kingston Pal Thamburaj5 Bharathi Raja Chakravarthi6
1
Department of Computer Science, Indian Institute of Information Technology Tiruchirappalli
2
ULTRA Arts and Science College, Madurai, Tamil Nadu, India
3
Sri Sivasubramaniya Nadar College of Engineering, Tamil Nadu, India
4
Kongu Engineering College, Erode, Tamil Nadu, India
5
Sultan Idris Education University, Tanjong Malim, Perak, Malaysia
6
Insight SFI Research Centre for Data Analytics, National University of Ireland Galway
{{karthikp, adeeph}18c@iiitt.ac.in, rubapriyadharshini.a@gmail.com, theni d@ssn.edu.in,
anbu.1318@gmail.com, fkingston@gmail.com, bharathi.raja@insight-centre.org}
Abstract
This paper reports the Machine Translation (MT) systems submitted by the IIITT team for the
English→Marathi and English⇔Irish language pairs LoResMT 2021 shared task. The task
focuses on getting exceptional translations for rather low-resourced languages like Irish and
Marathi. We fine-tune IndicTrans, a pretrained multilingual NMT model for English→Marathi,
using external parallel corpus as input for additional training. We have used a pretrained
Helsinki-NLP Opus MT English⇔Irish model for the latter language pair. Our approaches
yield relatively promising results on the BLEU metrics. Under the team name IIITT, our sys-
tems ranked 1, 1, and 2 in English→Marathi, Irish→English, and English→Irish respectively.
The codes for our systems are published1
.
1 Introduction
Today, a large number of text and written materials are present in English. However, with
roughly around 6,500 languages in the world2
(Chakravarthi, 2020; Hande et al., 2021a;
Sarveswaran et al., 2021), every native monoglot should not be deprived of this knowledge
and information. The manual translation is a tedious job involving much time and human re-
sources, giving rise to Machine Translation (MT). Machine Translation involves the automated
translation of text from one language to another by using various algorithms and resources to
produce quality translation predictions (Pathak and Pakray, 2018; Krishnamurthy, 2015, 2019).
Neural Machine Translation (NMT) brought about a great improvement in the field of MT
by overcoming flaws of rule-based and statistical machine translation (SMT) (Revanuru et al.,
2017; Achchuthan and Sarveswaran, 2015; Parameswari et al., 2012; Thenmozhi et al., 2018;
Kumar et al., 2020b). NMT incorporates the training of neural networks on parallel corpora to
predict the likeliness of a sequence of words. sequence-to-sequence neural models (seq2seq)
1https://github.com/karthikpuranik11/LoResMT
2https://blog.busuu.com/most-spoken-languages-in-the-world/
arXiv:2108.08556v1
[cs.CL]
19
Aug
2021
2. (Sutskever et al., 2014; Kalchbrenner and Blunsom, 2013) are the widely adopted as the stan-
dard approach by both industrial and research communities (Jadhav, 2020a; Bojar et al., 2016;
Cheng et al., 2016).
Even though NMT performs exceptionally well for all the languages, it requires a tremen-
dous amount of parallel corpus to produce meaningful and successful translations (Kumar et al.,
2020a). With little research on low resourced languages, finding a quality parallel corpus to
train the models can be arduous. The two low-resourced languages worked on in this paper
are Marathi (mr) and Irish (ga). With about 120 million Marathi speakers in Maharashtra and
other states of India, Marathi is recognized as one of the 22 scheduled languages of India3
. The
structural dissimilarity which occurs while translating from English (Subject-Verb-Object) to
Marathi (Subject-Object-Verb) or vice versa adds up to issues faced while translation (Garje,
2014). The Irish language was recognized as the first official language of Ireland and also by
the EU(Dowling et al., 2020a; Scannell, 2007). Belonging to the Goidelic language family and
the Celtic family(Scannell, 2007; Lynn et al., 2015), Irish is also claimed as one of the low re-
sourced languages due to its limited resources by the META-NET report (Dhonnchadha et al.,
2012; Scannell, 2006).
Our paper represents the work conducted for the LoResMT @ MT Summit 20214
shared
task to build MT systems for the low-resourced Marathi and Irish languages on COVID-19
related parallel corpus. We implement Transformer-based (Vaswani et al., 2017) NMT models
to procure BLEU scores(Papineni et al., 2002) of 24.2, 25.8, and 34.6 in English→Marathi,
Irish→English, and English→Irish respectively.
2 Related works
Neural Machine Translation has been exhaustively studied over the years (Kalchbrenner and
Blunsom, 2013), with several intuitive approaches involving collective learning to align and
translate (Bahdanau et al., 2016), and a language-independent attention bridge for multilingual
translational systems (Vázquez et al., 2019). There have been several approaches to NMT, with
zero-shot translational systems, between language pairs that have not seen the parallel training
data during training (Johnson et al., 2017). The introduction of artificial tokens has reduced
the architectural changes in the decoder (Ha et al., 2016). There have been some explorations
towards neural machine translation in low resource languages, with the development of a multi-
source translational system that targets the English string for any source language (Zoph and
Knight, 2016).
There has been subsequent research undertaken by researchers for machine translation in
low-resource Indian languages. Chakravarthi et al. surveyed orthographic information in ma-
chine translation, examining the orthography’s influence on machine translation and extended
it to under-resourced Dravidian languages (Chakravarthi et al., 2019a). Another approach of
leveraging the information contained in rule-based machine translation systems to improve ma-
chine translation of low-resourced languages was employed (Torregrosa et al., 2019). Several
approaches involving the improvement of WordNet for low-resourced languages have been ex-
plored (Chakravarthi et al., 2018, 2019b). Chakravarthi et al. constructed MMDravi, a multilin-
gual multimodal machine translation dataset for low-resourced Dravidian languages, extending
it from the Flickr30K dataset, and generating translations for the captions using phonetic tran-
scriptions (U Hegde et al., 2021).
There have been relatively fewer approaches experimented with and benchmarked when it
comes to translating from Marathi to English and vice versa. (Aharoni et al., 2019) tried to build
multilingual NMT systems, comprising 103 distinct languages and 204 translational directions
3https://en.wikipedia.org/wiki/Marathi_language
4https://sites.google.com/view/loresmt/
3. simultaneously. (Jadhav, 2020b; Puranik et al., 2021) developed a machine translation system
for Marathi to English using transformers on a parallel corpus. Other works on improving
machine translation include (Adi Narayana Reddy et al., 2021) proposing an English-Marathi
NMT using local attention. The same can be stated to Irish, as it is a poorly resourced language,
as the quality of the MT outputs have struggled to achieve the same level as well-supported lan-
guages (Dowling et al., 2016; Rehm and Uszkoreit, 2012). In recent years, several researchers
tried to overcome the resource barrier by creating artificial parallel data through back-translation
(Poncelas et al., 2018), exploiting out-of-domain data (Imankulova et al., 2019), and leveraging
other better-resourced languages as a pivot (Dowling et al., 2020b; Wu and Wang, 2007).
3 Dataset
We use the dataset provided by the organizers of LoResMT @ MT Summit 2021. The datasets
can be found here5
. It is a parallel corpus for English and the low resourced language, i.e., Irish
and Marathi, mostly containing text related to COVID-19 (Ojha et al., 2021).
Language pair English⇔Irish English⇔Marathi
Train 8,112 20,933
Dev 502 500
Test 1,000 1,000
Total 9,614 22,433
Table 1: Number of sentences distribution
We have used bible-uedin6
(Christodoulopoulos and Steedman, 2015) an external dataset
for Marathi. It is a multilingual parallel corpus dataset containing translations of the Bible in
102 languages(Christodoulopoulos and Steedman, 2014) and shows the possibility of using the
Bible for research and machine translation. English-Marathi corpus contains 60,495 sentences.
CVIT PIB7
(Philip et al., 2020) has also been used for the purpose of this research. It contains
1,14,220 parallel corpora for English-Marathi.
4 Methodology
4.1 IndicTrans
Fairseq PyTorch8
(Ott et al., 2019) is an open-source machine learning library supported as a
sequence modeling toolkit. Custom models can be trained for various tasks, including sum-
marization, language, translation, and other generation tasks. Training on fairseq enables com-
petent batching, mixed-precision training, multi-GPU and multi-machine training. IndicTrans
(Ramesh et al., 2021), a Transformer-4x multilingual NMT model by AI4Bharat, is trained on
the Samanantar dataset. The architecture of our approach is displayed in Fig.1. Samanantar9
is the most extensive collection of parallel corpora for Indic languages available for public use.
It includes 46.9 million sentence pairs between English and 11 Indian languages. IndicTrans
is claimed to successfully outperform the existing best performing models on a wide variety
of benchmarks. Even commercial translation systems and existing publicly available systems
were surpassed for the majority of the languages. IndicTrans is based on fairseq, and it was
5https://github.com/loresmt/loresmt-2021
6https://opus.nlpl.eu/JRC-Acquis.php
7http://preon.iiit.ac.in/˜jerin/bhasha/
8https://github.com/pytorch/fairseq
9https://indicnlp.ai4bharat.org/samanantar/
4. Figure 1: Our approach for the English→Marathi language pair.
fine-tuned on the Marathi training dataset provided by the organizers and the external datasets.
The model was fine-tuned with the cross-entropy criterion to compute the loss function, Adam
optimizer (Zhang, 2018), dropout of 0.2, fp16 (Micikevicius et al., 2017), maximum tokens of
256 for better learning, and a learning rate if 3e-5 in GPU. The process was conducted for a
maximum of 3 epochs.
4.2 Helsinki-NLP Opus-MT
OPUS-MT (Tiedemann and Thottingal, 2020) supports both bilingual and multilingual mod-
els. It is a project that focuses on the development of free resources and tools for machine
translation. The current status is a repository of over 1,000 pretrained neural MT mod-
els. We fine-tune a transformer-align model that was fine-tuned for the Tatoeba-Challenge10
.
Helsinki-NLP/opus-mt-en-ga model from the HuggingFace Transformers (Wolf et al., 2020)
for English→ Irish and Helsinki-NLP/opus-mt-ga-en for Irish→ English were used.
Language pair Method BLEU
English→Marathi IndicTrans baseline 14.0
English→Marathi IndicTrans TRA 17.8
English→Marathi IndicTrans CVIT-PIB 23.4
English→Marathi IndicTrans bible-uedin 27.7
English→Irish Opus MT 30.4
English→Irish M2M100 25.6
Irish→English Opus MT 37.2
Irish→English M2M100 30.4
Table 2: BLEU scores obtained for the various models for the development set
5 Results and Analysis
For Marathi, it is distinctly visible that our system model, i.e., IndicTrans fine-tuned on the
training data provided by the organizers or TRA and the bible-uedin dataset, gave the best BLEU
10https://github.com/Helsinki-NLP/Tatoeba-Challenge
5. Figure 2: The graph depicting the increase in val lossafter the third epoch
scores. It was surprising how the model fine-tuned on a parallel corpus of 60,495 sentences of
bible-uedin surpassed the model fine-tuned on 1,14,220 sentences from the CVIT PIB dataset.
The possible explanation is the higher correlation between the sentences of the bible-uedin
dataset with the test dataset than the CVIT PIB dataset. Another reason could be the presence
of excessive noise in the CVIT PIB dataset. The other reason for this could be noise and a lower
quality of translations in the CVIT PIB dataset compared to bible-uedin.
To infer the same, 1000 random pairs of sentences were picked from the datasets, and the
average LaBSE or language-agnostic BERT Sentence Embedding (Feng et al., 2020) scores
were found out. LaBSE gives a score between 0 and 1, depending on the quality of the transla-
tion. It was seen that the average was 0.768 for the bible-uedin dataset, while it was 0.58 for the
CVIT PIB dataset. This might have been one of the reasons for the better BLEU scores. The
model also showed constant overfitting after the second and third epoch, as the BLEU scores
reduced considerably as they reached the 6th epoch. The BLEU scores decreased by a differ-
ence of 6.The validation loss starts increasing after the third epoch, thus, showing the overfitting
occurring in training. So, the model was fine-tuned for three epochs while maintaining a low
learning rate of around 3e-5 to get a BLEU score of 24.2.
Language pair BLEU CHRF TER Rank
English→Marathi 24.2 0.59 0.597 1
Irish→English 34.6 0.61 0.711 1
English→Irish 25.8 0.53 0.629 2
Table 3: Result and ranks obtained for the test dataset(Popović, 2015; Snover et al., 2006)
Training a model to predict for a low-resourced language was highly challenging due to
the absence of prominent pretrained models (Kalyan et al., 2021; Yasaswini et al., 2021; Hande
6. et al., 2021b). However, as an experiment, two models from HuggingFace Transformers11
,
M2M100 (Fan et al., 2020) and Opus-MT from Helsinki NLP (Tiedemann, 2020) were com-
pared. For the dev data, Opus MT produced a BLEU score of 30.4 while M2M100 gave 25.62
for translations from English to Irish and 37.2 and 30.37 respectively for Irish to English transla-
tions. Probably, the individual models pretrained on numerous datasets gave Opus MT an edge
over M2M100. This led us to submit the Opus MT model for the LoResMT Shared task 2021.
The model gave exceptional BLEU scores of 25.8 for English to Irish, which ranked second in
the shared task, while 34.6 for Irish to English stood first.
6 Conclusion
It is arduous and unyielding to get accurate translations for low-resourced languages due to
limited datasets and pretrained models. However, our paper puts forward a few methods to
better the already existing accuracies. Ranked 1, 1, and 2 in English→Marathi, Irish→English,
and English→Irish respectively in the LoResMT 2021 shared task, IndicTrans fine-tuned on the
bible-uedin, and the dataset provided by the organizers manages to surpass the other models due
to its high correlation with the test set and minimal noise for the Marathi language. The Irish
language task was dominated by the Opus MT model by Helsinki-NLP, outperforming other
Transformer models, M2M100.
References
Achchuthan, Y. and Sarveswaran, K. (2015). Language localisation of Tamil using statistical machine
translation. In 2015 Fifteenth International Conference on Advances in ICT for Emerging Regions
(ICTer), pages 125–129. IEEE.
Adi Narayana Reddy, K., Shyam Chandra Prasad, G., Rajashekar Reddy, A., Naveen Kumar, L., and Kan-
naiah (2021). English-marathi neural machine translation using local attention. In Garg, D., Wong, K.,
Sarangapani, J., and Gupta, S. K., editors, Advanced Computing, pages 280–287, Singapore. Springer
Singapore.
Aharoni, R., Johnson, M., and Firat, O. (2019). Massively multilingual neural machine translation. In Pro-
ceedings of the 2019 Conference of the North American Chapter of the Association for Computational
Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 3874–3884,
Minneapolis, Minnesota. Association for Computational Linguistics.
Bahdanau, D., Cho, K., and Bengio, Y. (2016). Neural machine translation by jointly learning to align and
translate.
Bojar, O., Chatterjee, R., Federmann, C., Graham, Y., Haddow, B., Huck, M., Jimeno Yepes, A., Koehn, P.,
Logacheva, V., Monz, C., Negri, M., Névéol, A., Neves, M., Popel, M., Post, M., Rubino, R., Scarton,
C., Specia, L., Turchi, M., Verspoor, K., and Zampieri, M. (2016). Findings of the 2016 conference on
machine translation. In Proceedings of the First Conference on Machine Translation: Volume 2, Shared
Task Papers, pages 131–198, Berlin, Germany. Association for Computational Linguistics.
Chakravarthi, B. R. (2020). Leveraging orthographic information to improve machine translation of
under-resourced languages. PhD thesis, NUI Galway.
Chakravarthi, B. R., Arcan, M., and McCrae, J. P. (2018). Improving wordnets for under-resourced lan-
guages using machine translation. In Proceedings of the 9th Global Wordnet Conference, pages 77–86.
11https://huggingface.co/transformers/
7. Chakravarthi, B. R., Arcan, M., and McCrae, J. P. (2019a). Comparison of different orthographies for
machine translation of under-resourced dravidian languages. In 2nd Conference on Language, Data
and Knowledge (LDK 2019). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik.
Chakravarthi, B. R., Arcan, M., and McCrae, J. P. (2019b). Wordnet gloss translation for under-resourced
languages using multilingual neural machine translation. In Proceedings of the Second Workshop on
Multilingualism at the Intersection of Knowledge Bases and Machine Translation, pages 1–7.
Chakravarthi, B. R., Priyadharshini, R., Stearns, B., Jayapal, A. K., Sridevy, S., Arcan, M., Zarrouk,
M., and McCrae, J. P. (2019c). Multilingual multimodal machine translation for dravidian languages
utilizing phonetic transcription. In Proceedings of the 2nd Workshop on Technologies for MT of Low
Resource Languages, pages 56–63.
Chakravarthi, B. R., Rani, P., Arcan, M., and McCrae, J. P. (2021). A survey of orthographic information
in machine translation. SN Computer Science, 2(4):1–19.
Cheng, Y., Xu, W., He, Z., He, W., Wu, H., Sun, M., and Liu, Y. (2016). Semi-supervised learning
for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for
Computational Linguistics (Volume 1: Long Papers), pages 1965–1974, Berlin, Germany. Association
for Computational Linguistics.
Christodoulopoulos, C. and Steedman, M. (2014). A massively parallel corpus: the bible in 100 languages.
Language Resources and Evaluation, 49:1–21.
Christodoulopoulos, C. and Steedman, M. (2015). A massively parallel corpus: the bible in 100 languages.
Language Resources and Evaluation, 49:375 – 395.
Dhonnchadha, E., Judge, J., Chasaide, A., Dhubhda, R., and Scannell, K. (2012). The irish language in
the digital age: An ghaeilge sa ré dhigiteach.
Dowling, M., Castilho, S., Moorkens, J., Lynn, T., and Way, A. (2020a). A human evaluation of english-
irish statistical and neural machine translation.
Dowling, M., Castilho, S., Moorkens, J., Lynn, T., and Way, A. (2020b). A human evaluation of English-
Irish statistical and neural machine translation. In Proceedings of the 22nd Annual Conference of the
European Association for Machine Translation, pages 431–440, Lisboa, Portugal. European Associa-
tion for Machine Translation.
Dowling, M., Judge, J., Lynn, T., and Graham, Y. (2016). English to irish machine translation with
automatic post-editing.
Fan, A., Bhosale, S., Schwenk, H., Ma, Z., El-Kishky, A., Goyal, S., Baines, M., Celebi, O., Wenzek, G.,
Chaudhary, V., Goyal, N., Birch, T., Liptchinsky, V., Edunov, S., Grave, E., Auli, M., and Joulin, A.
(2020). Beyond english-centric multilingual machine translation.
Feng, F., Yang, Y., Cer, D., Arivazhagan, N., and Wang, W. (2020). Language-agnostic bert sentence
embedding.
Garje, G. (2014). Marathi to english machine translation for simple sentences. International Journal of
Science and Research (IJSR), 3.
Ha, T.-L., Niehues, J., and Waibel, A. (2016). Toward multilingual neural machine translation with uni-
versal encoder and decoder. ArXiv, abs/1611.04798.
8. Hande, A., Puranik, K., Priyadharshini, R., and Chakravarthi, B. R. (2021a). Domain identification of
scientific articles using transfer learning and ensembles. In Trends and Applications in Knowledge Dis-
covery and Data Mining: PAKDD 2021 Workshops, WSPA, MLMEIN, SDPRA, DARAI, and AI4EPT,
Delhi, India, May 11, 2021 Proceedings 25, pages 88–97. Springer International Publishing.
Hande, A., Puranik, K., Priyadharshini, R., Thavareesan, S., and Chakravarthi, B. R. (2021b). Evaluat-
ing pretrained transformer-based models for covid-19 fake news detection. In 2021 5th International
Conference on Computing Methodologies and Communication (ICCMC), pages 766–772.
Imankulova, A., Dabre, R., Fujita, A., and Imamura, K. (2019). Exploiting out-of-domain parallel data
through multilingual transfer learning for low-resource neural machine translation. In Proceedings of
Machine Translation Summit XVII Volume 1: Research Track, pages 128–139, Dublin, Ireland. Euro-
pean Association for Machine Translation.
Jadhav, S. (2020a). Marathi to english neural machine translation with near perfect corpus and transform-
ers. ArXiv, abs/2002.11643.
Jadhav, S. A. (2020b). Marathi to english neural machine translation with near perfect corpus and trans-
formers.
Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg,
M., Corrado, G., Hughes, M., and Dean, J. (2017). Google’s multilingual neural machine translation
system: Enabling zero-shot translation.
Kalchbrenner, N. and Blunsom, P. (2013). Recurrent continuous translation models. In Proceedings of the
2013 Conference on Empirical Methods in Natural Language Processing, pages 1700–1709, Seattle,
Washington, USA. Association for Computational Linguistics.
Kalyan, P., Reddy, D., Hande, A., Priyadharshini, R., Sakuntharaj, R., and Chakravarthi, B. R. (2021).
IIITT at CASE 2021 task 1: Leveraging pretrained language models for multilingual protest detection.
In Challenges and Applications of Automated Extraction of Socio-political Events from Text, pages
98–104, Online. Association for Computational Linguistics.
Krishnamurthy, P. (2015). Development of Telugu-Tamil transfer-based machine translation system: With
special reference to divergence index. In Proceedings of the 1st Deep Machine Translation Workshop,
pages 48–54, Praha, Czechia. ÚFAL MFF UK.
Krishnamurthy, P. (2019). Development of Telugu-Tamil transfer-based machine translation system: An
improvization using divergence index. Journal of Intelligent Systems, 28(3):493–504.
Kumar, A., Mundotiya, R. K., and Singh, A. K. (2020a). Unsupervised approach for zero-shot exper-
iments: Bhojpuri–Hindi and Magahi–Hindi@LoResMT 2020. In Proceedings of the 3rd Workshop
on Technologies for MT of Low Resource Languages, pages 43–46, Suzhou, China. Association for
Computational Linguistics.
Kumar, B. S., Thenmozhi, D., and Kayalvizhi, S. (2020b). Tamil paraphrase detection using encoder-
decoder neural networks. In International Conference on Computational Intelligence in Data Science,
pages 30–42. Springer.
Lynn, T., Scannell, K., and Maguire, E. (2015). Minority language twitter: Part-of-speech tagging and
analysis of irish tweets.
Micikevicius, P., Narang, S., Alben, J., Diamos, G., Elsen, E., Garcia, D., Ginsburg, B., Houston, M.,
Kuchaiev, O., Venkatesh, G., et al. (2017). Mixed precision training. arXiv preprint arXiv:1710.03740.
9. Ojha, A. K., Liu, C.-H., Kann, K., Ortega, J., Satam, S., and Fransen, T. (2021). Findings of the LoResMT
2021 Shared Task on COVID and Sign Language for Low-Resource Languages. In Proceedings of the
4th Workshop on Technologies for MT of Low Resource Languages.
Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., Grangier, D., and Auli, M. (2019). fairseq: A
fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations.
Papineni, K., Roukos, S., Ward, T., and Zhu, W. J. (2002). Bleu: a method for automatic evaluation of
machine translation.
Parameswari, K., Sreenivasulu, N., Uma Maheshwar Rao, G., and Christopher, M. (2012). Development
of Telugu-Tamil bidirectional machine translation system: A special focus on case divergence. In
proceedings of 11th International Tamil Internet conference, pages 180–191.
Pathak, A. and Pakray, D. P. (2018). Neural machine translation for indian languages. Journal of Intelligent
Systems, 28.
Philip, J., Siripragada, S., Namboodiri, V. P., and Jawahar, C. V. (2020). Revisiting low resource status of
indian languages in machine translation. 8th ACM IKDD CODS and 26th COMAD.
Poncelas, A., Shterionov, D., Way, A., Wenniger, G. M. D. B., and Passban, P. (2018). Investigating
backtranslation in neural machine translation. ArXiv, abs/1804.06189.
Popović, M. (2015). chrF: character n-gram F-score for automatic MT evaluation. In Proceedings of the
Tenth Workshop on Statistical Machine Translation, pages 392–395, Lisbon, Portugal. Association for
Computational Linguistics.
Puranik, K., Hande, A., Priyadharshini, R., Thavareesan, S., and Chakravarthi, B. R. (2021). IIITT@LT-
EDI-EACL2021-hope speech detection: There is always hope in transformers. In Proceedings of the
First Workshop on Language Technology for Equality, Diversity and Inclusion, pages 98–106, Kyiv.
Association for Computational Linguistics.
Ramesh, G., Doddapaneni, S., Bheemaraj, A., Jobanputra, M., AK, R., Sharma, A., Sahoo, S., Diddee, H.,
J, M., Kakwani, D., Kumar, N., Pradeep, A., Deepak, K., Raghavan, V., Kunchukuttan, A., Kumar, P.,
and Khapra, M. S. (2021). Samanantar: The largest publicly available parallel corpora collection for 11
indic languages.
Rehm, G. and Uszkoreit, H. (2012). The irish language in the digital age.
Revanuru, K., Turlapaty, K., and Rao, S. (2017). Neural machine translation of indian languages. In
Proceedings of the 10th Annual ACM India Compute Conference, Compute ’17, page 11–20, New
York, NY, USA. Association for Computing Machinery.
Sarveswaran, K., Dias, G., and Butt, M. (2021). Thamizhimorph: A morphological parser for the Tamil
language. Machine Translation, 35(1):37–70.
Scannell, K. P. (2006). Machine translation for closely related language pairs. In Proceedings of the Work-
shop Strategies for developing machine translation for minority languages, pages 103–109. Citeseer.
Scannell, K. P. (2007). The crúbadán project: Corpus building for under-resourced languages. In Building
and Exploring Web Corpora: Proceedings of the 3rd Web as Corpus Workshop, volume 4, pages 5–15.
Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. (2006). A study of translation edit rate
with targeted human annotation.
10. Sutskever, I., Vinyals, O., and Le, Q. V. (2014). Sequence to sequence learning with neural networks.
Thenmozhi, D., Kumar, B. S., and Aravindan, C. (2018). Deep learning approach to English-Tamil and
Hindi-Tamil verb phrase translations. In FIRE (Working Notes), pages 323–331.
Tiedemann, J. (2020). The tatoeba translation challenge – realistic data sets for low resource and multilin-
gual mt.
Tiedemann, J. and Thottingal, S. (2020). OPUS-MT – building open translation services for the world.
In Proceedings of the 22nd Annual Conference of the European Association for Machine Translation,
pages 479–480, Lisboa, Portugal. European Association for Machine Translation.
Torregrosa, D., Pasricha, N., Masoud, M., Chakravarthi, B. R., Alonso, J., Casas, N., and Arcan, M.
(2019). Leveraging rule-based machine translation knowledge for under-resourced neural machine
translation models. In Proceedings of Machine Translation Summit XVII Volume 2: Translator, Project
and User Tracks, pages 125–133, Dublin, Ireland. European Association for Machine Translation.
U Hegde, S., Hande, A., Priyadharshini, R., Thavareesan, S., and Chakravarthi, B. R. (2021). UVCE-
IIITT@DravidianLangTech-EACL2021: Tamil troll meme classification: You need to pay more at-
tention. In Proceedings of the First Workshop on Speech and Language Technologies for Dravidian
Languages, pages 180–186, Kyiv. Association for Computational Linguistics.
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin,
I. (2017). Attention is all you need.
Vázquez, R., Raganato, A., Tiedemann, J., and Creutz, M. (2019). Multilingual NMT with a language-
independent attention bridge. In Proceedings of the 4th Workshop on Representation Learning for NLP
(RepL4NLP-2019), pages 33–39, Florence, Italy. Association for Computational Linguistics.
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Fun-
towicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Le Scao, T.,
Gugger, S., Drame, M., Lhoest, Q., and Rush, A. (2020). Transformers: State-of-the-art natural lan-
guage processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language
Processing: System Demonstrations, pages 38–45, Online. Association for Computational Linguistics.
Wu, H. and Wang, H. (2007). Pivot language approach for phrase-based statistical machine translation.
In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages
856–863, Prague, Czech Republic. Association for Computational Linguistics.
Yasaswini, K., Puranik, K., Hande, A., Priyadharshini, R., Thavareesan, S., and Chakravarthi, B. R.
(2021). IIITT@DravidianLangTech-EACL2021: Transfer learning for offensive language detection
in Dravidian languages. In Proceedings of the First Workshop on Speech and Language Technologies
for Dravidian Languages, pages 187–194, Kyiv. Association for Computational Linguistics.
Zhang, Z. (2018). Improved adam optimizer for deep neural networks. In 2018 IEEE/ACM 26th Interna-
tional Symposium on Quality of Service (IWQoS), pages 1–2. IEEE.
Zoph, B. and Knight, K. (2016). Multi-source neural translation. In Proceedings of the 2016 Conference
of the North American Chapter of the Association for Computational Linguistics: Human Language
Technologies, pages 30–34, San Diego, California. Association for Computational Linguistics.