“Neural Machine Translation for low resource languages: Use case anglais - wolof“ by Sokhar Samb - Data scientist at @THEOLEX Abstract : We will dive into the different steps of developing a Wolof-English machine translation using JoeyNMT using the benchmark from Masakhane NLP. This presentation took place during a joint WiMLDS meetup between Paris & Dakar.

2. Low resources machine
translation
Presented by WiMLDS_Dakar

3. Definition
Natural language processing (NLP) is a field of Artificial Intelligence that tries to establish human-like
communication with computers. Although it can boast significant success, computers still struggle with
comprehending many facets of language, such as pragmatics, that are difficult to characterize formally.
Moreover, most of the success is achieved for popular languages like English or other languages that
have text corpora of hundreds of millions of words. But we should understand that these are only about
20 languages from approximately 7,000 languages in the world. The majority of human languages are in
dire need of tools and resources to overcome the resource barrier such that NLP can deliver more
widespread benefits. They are called low-resource languages languages, or languages lacking large
monolingual or parallel corpora and/or manually crafted linguistic resources sufficient for building
statistical NLP applications.
Unfortunately, all the African language are low resource. que to improve them in the case of low
resources.

4. Abstract
Large number of sentences from a source language to a targeted language is necessary to
train machine translation models. However, there are a few parallel corpus for low resources
languages.
In this work, we improve a neural machine translation by using data augmentation. Through the
data augmentation, we were able to increase the availability of different set of data for
our training models without sourcing for new data. The technique of data augmentation used is the
back translation method on Wolof monolingual data to generate a synthetic parallel corpus.
The synthetic parallel corpus is added to the original parallel corpus. The final model is then trained
on the augmented data with the same parameters than the initial model. We show
how this technique improves the translation model in different scenarios.

5. Data
We first consider the existing parallel
corpus (Wolof-English) from the JW300. JW300 is a
parallel corpus of over 300 languages with around
100 thousand
parallel sentences per language pair on average.
Thus, we collected 17945 parallel sentences.
These sentences are basically composed of
17723 sentences of the translation of the bible
and 222 sentences of the Gnome
making us have 17945 available parallel corpus of the
English-
Wolof. Augmenting the number of parallel corpus,
we scrap monolingual data from news website:
wolof-online1, defuwaxu2, and Wikipedia Wolof.

6. Model
We use Joey NMT model, a minimalist neural machine translation toolkit based on PyTorch that is
specifically designed for novices. Joey NMT provides many popular NMT features in a small and simple
code base, so that novices can easily and quickly learn to use it and adapt it to their needs. Despite its
focus on simplicity, Joey NMT supports classic architectures (RNNs, transformers), fast beam search,
weight tying, and more, and achieves performance comparable to more complex toolkits on standard
benchmarks. In this work, we use joeyNMT with a multi head attention.

8. NMT TECHNIQUES FOR LOW-RESOURCE
LANGUAGES
1. Back translation
Back-translation (BT) is an alternative to lever- age monolingual data. BT is simple and easy to
apply as it does not require modification to the MT training algorithms. It requires training a target-
to-source system in order to generate additional synthetic parallel data from the monolingual tar-
get data.
2. Cosine similarity
Cosine similarity is one of the metric to measure the text-similarity between two documents
irrespective of their size in Natural language Processing. ... If the Cosine similarity score is 1, it
means two vectors have the same orientation. The value closer to 0 indicates that the two
documents have less similarity.

9. Methodology and result
Wolof monolingual sentences
Cosine
similarity
Translation
model
English to
Wolof
Back
translate
to
English
Synthetic parallel corpus

10. Methodology and result
Data BLUE PPL
Without synthetic data 27.55 6.82
With synthetic dada 31.87 6.53
Model with synthetic data

11. Conclusion
In this paper we try to improve machine translation with back-translation method. We downloaded
monolingual data from new website to create synthetic data. The synthetic data is added to the
existing parallel corpus from the JW300.
To increase the quality of the synthetic data, we filter them using the cosine similarity.Doing this, we
were able to increase the BLUE scores from 27.55 to 31.87. However we encounter some challenges
especially on the existence and the quality of the monolingual. The number of monolingual sentences
used in this work is small due to the filtering with the cosine similarity and this step was necessary to
improve the quality of the back-translation model.
As future work, we planning to first of all create parallel corpus with divers theme instead of using the
translation of the bible with concern only religion. And try to find more monolingual sentence with
better quality.