Be the first to like this
Natural Language Processing is an area of Artificial Intelligence, in
particular, of Pattern Recognition. It is a multidisciplinary field that studies
human language, both oral and written. It deals with the development and
research of computational mechanisms for communication between people and
computers, using natural languages. Natural Language Processing is a reasearch
area constantly evolving, and this work focuses only on the part related to
language modeling, and its application to various tasks:
recognition/understanding of sequences and statistical machine translation.
Specifically, this thesis focus its interest on the so-called connectionist
language models (or continuos space language models), i.e., language models
based on neural networks. Their excellent performance in various Natural
Language Processing areas has motivated this study.
Because of certain computational problems suffered by connectionist language
models, the most widespread approach followed by the systems that currently
use these models, is based on two totally decoupled stages. At a first stage,
using a standard and cheaper language model, a set of feasible hypotheses,
assuming that this set is representative of the search space in which the best
hypothesis is located, is generated. In a second stage, on this set, a
connectionist language model is applied and a rescoring of the list of
hypotheses is done.
This scenario motivates scientific goals of this thesis:
- Developing techniques to reduce drastically the computational cost degrading
as less as possible the quality.
- Study the effect of a totally coupled approach that integrates neural network
language models on decoding stage.
- Developing some extensions of original model in order to improve it quality
and to fulfill context domain adaptation.
- Empirical application of neural network language models to sequence
recognition and machine translation tasks.
All developed algorithms were implemented in C++ and using Lua as scripting
language. The implementations are compared with those that are considered
standard on each of the addressed tasks. Neural network language models achieve
very interesting improvements of quality over the reference baseline systems:
- competitive results are achieved on automatic speech recognition and spoken
- improvement of state-of-the-art handwritten text recognition;
- state-of-the-art results on statistical machine translation, as was stated
with the participation on international evaluation campaigns.
On sequence recognition tasks, the integration of neural network language models
on the first decoding stage achieve very competitive computational
costs. However, their integration in machine translation tasks requires a deeper
development because the computation cost of the system is still somewhat high.