SlideShare a Scribd company logo
Deep Learning for Machine
Translation
Satoshi Enoue, Jungi Kim, Jean Senellart, SYSTRAN
SYSTRAN Through Machine Translation
History
Rule Base Machine Translation
Example-Based Machine
Translation
Phrase Based Machine Translation
Syntax Based Machine Translation
Neural Machine
Translation
Hybrid Machine Translation
SYSTRAN
197
1968
SYSTRAN (SYStem
TRANslation)
founded by Dr.
Toma in La Jolla,
California (USA)
1969
Provided first
MT software for
the US Air Force,
(Russian to
English)
1975
Used by NASA
for the Apollo-
Soyuz
American-Soviet
project
1975
Translation systems for
all European languages
in the European
Commission
1986
SYSTRAN is acquired
by France’s Gachot SA,
thus becoming a
French company with
a U.S. subsidiary
1995
Pioneered development of
first Windows-based MT
software
1997
First free Web-based translation
service: Altavista Babelfish. SYSTRAN
made the Internet community aware
of the usefulness and capabilities of
machine translation
2002
SYSTRAN was used on
most major Internet
Portals: Yahoo!, Google,
AltaVista, Lycos.
1996
SYSTRAN within SEIKO’s
pocket translators.
1990’s
Port technology from mainframes to
Desktop PC’s and Client-Server environments
for personal and corporate use
2014
Following acquisition by CSLI,
SYSTRAN SA forms part of the
SYSTRAN International Group
2005
Launched embedded
translation software for
mobile devices
2009
Developed first
hybrid translation
software and
solution: SES 7
Translation Server
2011
Launch of SES 7
Training Server,
first solution for
self-learning of
MT engines
2015
SES8 Translation and Training
Server – Large Models
2016
More than 140 language Pairs.
Launch of SYSTRAN.io, the
Natural Language Processing
API platform
The new game changer
• Deep Neural Network
Technologies
• Image Analysis
• Voice Recognition
• Text
• Text Generation
• Word Embeddings
• Multitask NLP
• Neural Machine Translation
• … Games
• Super Human Abilities
Sequence of fascinating results
and technologies over the last 3
years – all based on Deep Neural
Network (DNN) – covering a large
variety of domains…
26/04/2016
SYSTRAN - Copyright 2016
3
The new game changer
• Deep Neural Network
Technologies
• Image Analysis
• Voice Recognition
• Text
• Text Generation
• Word Embeddings
• Multitask NLP
• Neural Machine Translation
• … Games
26/04/2016
SYSTRAN - Copyright 2016
4
The new game changer
• Deep Neural Network
Technologies
• Image Analysis
• Voice Recognition
• Text
• Text Generation
• Word Embeddings
• Multitask NLP
• Neural Machine Translation
• … Games Abilities
• Google 2015 RNN voice search
recognition outperforms 2012
DNN models
• Baidu Deep Speech announces
16.5% improvement over
baseline and higher
performance than human in
noisy environment
26/04/2016
SYSTRAN - Copyright 2016
5
The new game changer - examples
• Deep Neural Network
Technologies
• Image Analysis
• Voice Recognition
• Text
• Text Generation
• Word Embeddings
• Multitask NLP
• Neural Machine Translation
• … Gamesn Abilities
Les yeux prenaient des redoutables, des troncs de
feu.
Toutes les prétexticheurs par ces quatre
repentilleuses avec du sergent de Digne,
débragiffés nymoeurs sur les derniers instants à
hardis, boucher, sans dénongée en plus ennérence,
ils se refecturent encore. Ils auraient déjà mangé
ses très interses.
ShakespeareVictorHugo
Char-RNN, Andrej Karpathy, 2015
공급자는 AspNetXSprchyLibrary의 인스턴스를
만들어 다른 경고를 오버터 컴퓨터에 저장할 수
있습니다.
MSDN
26/04/2016
SYSTRAN - Copyright 2016
26/04/2016
SYSTRAN - Copyright 2016
7
26/04/2016
SYSTRAN - Copyright 2016
8
The new game changer - examples
• Deep Neural Network
Technologies
• Image Analysis
• Voice Recognition
• Text
• Text Generation
• Word Embeddings
• Multitask NLP
• Neural Machine Translation
• … Games
26/04/2016 9word2vec, Google, 2013
The new game changer - examples
• Unified Neural Network
Architecture for several NLP tasks
POS tagging, chunking, NER, SRL
• Focus on avoiding task/linguistic
specific engineering
• Joint decision on the different tasks
Outperforms almost all of the state
of the art results for each individual
tasks
Natural Language Processing (Almost) from Scratch, Collobert et al., 201126/04/2016
• Deep Neural Network
Technologies
• Image Analysis
• Voice Recognition
• Text
• Text Generation
• Word Embeddings
• Multitask NLP
• Neural Machine Translation
• … Gamesn Abilities
The new game changer - examples
• Deep Neural Network
Technologies
• Image Analysis
• Voice Recognition
• Text
• Text Generation
• Word Embeddings
• Multitask NLP
• Neural Machine Translation:
sentence encoding-decoding
• … Games
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation, K. Cho et al, 2014
The new game changer - examples
• Deep Neural Network
Technologies
• Image Analysis
• Voice Recognition
• Text
• Text Generation
• Word Embeddings
• Multitask NLP
• Neural Machine : sentence encoding-
decoding
• … Games – DQN, AlphaGo
HUMAN-LEVEL CONTROL THROUGH DEEP REINFORCEMENT LEARNING, Google DeepMind, 201526/04/2016
AlphaGo, Google DeepMind, 201626/04/2016
SYSTRAN - Copyright 2016
The new game changer - examples
More and more evidence of
“super-human abilities”
Could we also reach Super-
human Machine Translation?
26/04/2016
SYSTRAN - Copyright 2016
14
The new game changer – ingredients
• MLP – multilayer perceptron
• Actually an “old concept”
• CNN
• Convolutional Neural network
• Word Embeddings
• Representing words as vectors
• RNN – GRU, LSTM
• MLP with memory
• Attention-Based models
• Ability to decide where to find
information
26/04/2016
SYSTRAN - Copyright 2016
15
The new game changer – ingredients
• MLP – multilayer perceptron
• Actually an “old concept”
• CNN
• Convolutional Neural network
• Word Embeddings
• Representing words as vectors
• RNN – GRU, LSTM
• MLP with memory
• Attention-Based models
• Ability to decide where to find
information
26/04/2016
SYSTRAN - Copyright 2016
16
The new game changer – ingredients
• MLP – multilayer perceptron
• Actually an “old concept”
• CNN
• Convolutional Neural network
• Word Embeddings
• Representing words as vectors
• RNN – GRU, LSTM
• MLP with memory
• Attention-Based models
• Ability to decide where to find
information
26/04/2016
SYSTRAN - Copyright 2016
17
The new game changer – ingredients
• MLP – multilayer perceptron
• Actually an “old concept”
• CNN
• Convolutional Neural network
• Word Embeddings
• Representing words as vectors
• RNN – GRU, LSTM
• MLP with memory
• Attention-Based models
• Ability to decide where to find
information
26/04/2016
SYSTRAN - Copyright 2016
18
The new game changer – ingredients
• MLP – multilayer perceptron
• Actually an “old concept”
• CNN
• Convolutional Neural network
• Word Embeddings
• Representing words as vectors
• RNN – GRU, LSTM
• MLP with memory
• Attention-Based models
• Ability to decide where to find
information
26/04/2016
SYSTRAN - Copyright 2016
19
All of these features are the ingredients to
Neural Machine Translation
About Neural Machine
Translation (NMT)
• The goal is to perform end-to-end translation
• Like in Speech Recognition
• The spirit is to remove all these features and have single system
• For Machine Translation – first NMT systems are encoder-decoder
• But not that magic
• Not systematic improvements over SMT baseline
• Use of ensemble systems
• Issues with sentence lengths, vocabulary size
• Solutions come back with some interest in “linguistic” characteristics
• Attention-Based model (alignment information)
• Deep Fusion with Language Model (better modelling of target language)
• Combine with word level (~ morphology)
26/04/2016
SYSTRAN - Copyright 2016
20
SYSTRAN approach to NMT
• Current Real Use-Case Requirements:
• Adaptation to (small) domain
• Help for post-editing
• Preserved speed
• Consistent results amongst multiple target languages
• Possibility to let users control translation through annotations, terminology
• …
• Toward Linguistically Motivated NN architecture
• SYSTRAN MT is composed of linguistic modules – let us start with them
• Lot of knowledge to leverage
26/04/2016
SYSTRAN - Copyright 2016
21
SYSTRAN Deep Learning Story – Part I
Language Identification
SYSTRAN LDK 1
•Statistical Classifier – 3-grams
•Heavily Feature Engineered over years
•e.g. diacritics model for latin language
•Include lexicon of frequent terms
•Quite good accuracy on news-type data
– need ~20 characters
Basic RNN
•“out-of-the-box” character level RNN
•no specific language specific
engineering
•80K words training per language
Google CLD
•Naïve Bayesian Classifier – 4-grams
•Trained on “big data”
•carefully scrapped over 100M pages
•Specific tricks for closely related
languages (Spanish/Portuguese)
•Geared for webpages - 200+ characters
Learnings: with same data RNN approach easily outperforms baseline, no
specific engineering needed… big data is not competing...
26/04/2016
SYSTRAN - Copyright 2016
22
News
Sentences
One-word
request
Ted-Talk
Sentences
Tweets
LDK 97 55.2 87.4 78.3
RNN 98.2 61.5 91.4 77.9
CLD 96.1 15.3 86 78.1
SYSTRAN Deep Learning Story – Part II
Part of Speech Tagging
Phase 1 - 1968-2014 - Handcrafting
•Manual Rule and Lexicon Coding of homography
•Closely related to Morphology description
•27 languages covered
Phase 2 - 2008-2015 – Annotating
•Train Classifier to "relearn” rules (fnTBL)
•Transfer knowledge through system output
•Maintenance through Annotation
Phase 3 - 2015- - Generalizing
•Relearn with RNN
•Joint decision (so far tokenization/part of speech
tagging) – working on morphology
•Better generalization from additional knowledge
(word embeddings)
26/04/2016
SYSTRAN - Copyright 2016
23
Learnings: Possibility to leverage ”handcrafting” and gain quality. But
learning becoming too smart – it also learns initial errors
SYSTRAN Deep Learning Story – Part III
Transliteration
26/04/2016 24
• Transliteration of person names
is depending on
• Source Language
• Target Language
• But also Name origin
• 카스파로프 = Kasparov
• 필리프 = Philippe
• Good Transliteration system
needs:
• Detection of origin
• Transliteration mechanism
•Extremely complicated – since it requires
phonetics modeling
Rule-Based
• Satisfactory but origin detection and multiple
domains
• No generalization - unseen sequence is wrong
PBMT
• Encoding-Decoding Approach
• Long distance "view" guarantee consistency of
transliteration
RNN
Learnings:
- losing reliability/traceability of the process
+ more global consistency, compactness of the solution
SYSTRAN Deep Learning Story – Part IV
Language Modeling
• RNN language model proves to overpass standard n-gram models
• No limitation in the span
• Seems to capture also better the language structure
• Better generalization due to word embedding
• Can be easily introduced in PBMT engine through rescoring
• Are still challenging pure sequence-to-sequence NMT approaches
26/04/2016 25
Learnings:
- Very long training process, several weeks of training for one language
+ Consistent quality gain, easy introduction in existing framework
Learnings from Deep Learning
• Consistent quality improvement in all the experiments/modules we
worked on
• Better leverage of existing training material
• Better generalization
• Incrementability: by design, it is immediate to feed more training data
– i.e. adapt dynamically to usage
• Globally more simple than alternative approaches and cognitively
interesting
• Fit to be combined in a global NN architecture
26/04/2016
SYSTRAN - Copyright 2016
26
Linguistically Motivated NN architecture
26/04/2016 SYSTRAN - Copyright 2016 27
Morphology
Syntactic Analysis
Sentence Encoding Sentence Decoding
RNN-LM
Word Embedding
Source Sentence …
Target Sentence …
What about Statistical Post Editing:
Learning to correct?
26/04/2016
SYSTRAN - Copyright 2016
28
• SPE was introduced as smart
alternative the SMT
• Corresponding to real MT use case for
localization
• Very little data can produce adaptation
• Reduce Human Post-Editor Work by
iteratively learning edits
• However implementation with PBMT
is not satisfactory
• PBMT does not learn to correct but to
translate
• Not incremental
• Learning to correct
• More control of the process
Toward a “translation checker”
• Change the paradigm – now human post-
editor to MT output, tomorrow
automatic post-editor to human output?
MT
HPE
Deep Learning for Machine Translation
• No doubt – it is coming:
• We will probably reach “superhuman” machine translation in coming years
• And this could become real translation assistant
• How is not yet completely clear
• From our perspective, we are working on hybrid approach = linguistically motivated
NN architecture
• More will also be coming from research world
• Still some work ahead
• Training of models is still a technological challenge
• We need the models to explain as much as to translate to become really useful – or
for language learning
• Multi-level analysis - document translation and not just sentences
• Multi-modal => could lead to full self language learning
26/04/2016
SYSTRAN - Copyright 2016
29

More Related Content

What's hot

Recurrent Neural Networks for Text Analysis
Recurrent Neural Networks for Text AnalysisRecurrent Neural Networks for Text Analysis
Recurrent Neural Networks for Text Analysis
odsc
 
[Impl] neural machine translation
[Impl] neural machine translation[Impl] neural machine translation
[Impl] neural machine translation
JaeHo Jang
 
Building a Pipeline for State-of-the-Art Natural Language Processing Using Hu...
Building a Pipeline for State-of-the-Art Natural Language Processing Using Hu...Building a Pipeline for State-of-the-Art Natural Language Processing Using Hu...
Building a Pipeline for State-of-the-Art Natural Language Processing Using Hu...
Databricks
 
Notes on attention mechanism
Notes on attention mechanismNotes on attention mechanism
Notes on attention mechanism
Khang Pham
 
RNN & LSTM: Neural Network for Sequential Data
RNN & LSTM: Neural Network for Sequential DataRNN & LSTM: Neural Network for Sequential Data
RNN & LSTM: Neural Network for Sequential Data
Yao-Chieh Hu
 
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Minh Pham
 
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog DetectorDeep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
Roelof Pieters
 
[Paper Reading] Attention is All You Need
[Paper Reading] Attention is All You Need[Paper Reading] Attention is All You Need
[Paper Reading] Attention is All You Need
Daiki Tanaka
 
Generating Sequences with Deep LSTMs & RNNS in julia
Generating Sequences with Deep LSTMs & RNNS in juliaGenerating Sequences with Deep LSTMs & RNNS in julia
Generating Sequences with Deep LSTMs & RNNS in julia
Andre Pemmelaar
 
BERT
BERTBERT
[Paper review] BERT
[Paper review] BERT[Paper review] BERT
[Paper review] BERT
JEE HYUN PARK
 
BERT: Bidirectional Encoder Representations from Transformers
BERT: Bidirectional Encoder Representations from TransformersBERT: Bidirectional Encoder Representations from Transformers
BERT: Bidirectional Encoder Representations from Transformers
Liangqun Lu
 
[246]reasoning, attention and memory toward differentiable reasoning machines
[246]reasoning, attention and memory   toward differentiable reasoning machines[246]reasoning, attention and memory   toward differentiable reasoning machines
[246]reasoning, attention and memory toward differentiable reasoning machines
NAVER D2
 
The Factoring Dead: Preparing for the Cryptopocalypse
The Factoring Dead: Preparing for the CryptopocalypseThe Factoring Dead: Preparing for the Cryptopocalypse
The Factoring Dead: Preparing for the Cryptopocalypse
Alex Stamos
 
Chris Fregly, Research Scientist, PipelineIO at MLconf ATL 2016
Chris Fregly, Research Scientist, PipelineIO at MLconf ATL 2016Chris Fregly, Research Scientist, PipelineIO at MLconf ATL 2016
Chris Fregly, Research Scientist, PipelineIO at MLconf ATL 2016
MLconf
 
Mentoring Session with Innovesia: Advance Robotics
Mentoring Session with Innovesia: Advance RoboticsMentoring Session with Innovesia: Advance Robotics
Mentoring Session with Innovesia: Advance Robotics
Dony Riyanto
 
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...
siouxhotornot
 
Intro to Data Structure & Algorithms
Intro to Data Structure & AlgorithmsIntro to Data Structure & Algorithms
Intro to Data Structure & Algorithms
Akhil Kaushik
 
Master Thesis of Computer Engineering: OpenTranslator
Master Thesis of Computer Engineering: OpenTranslatorMaster Thesis of Computer Engineering: OpenTranslator
Master Thesis of Computer Engineering: OpenTranslator
Giuseppe D'Onofrio
 
Neural Networks and Deep Learning
Neural Networks and Deep LearningNeural Networks and Deep Learning
Neural Networks and Deep Learning
Asim Jalis
 

What's hot (20)

Recurrent Neural Networks for Text Analysis
Recurrent Neural Networks for Text AnalysisRecurrent Neural Networks for Text Analysis
Recurrent Neural Networks for Text Analysis
 
[Impl] neural machine translation
[Impl] neural machine translation[Impl] neural machine translation
[Impl] neural machine translation
 
Building a Pipeline for State-of-the-Art Natural Language Processing Using Hu...
Building a Pipeline for State-of-the-Art Natural Language Processing Using Hu...Building a Pipeline for State-of-the-Art Natural Language Processing Using Hu...
Building a Pipeline for State-of-the-Art Natural Language Processing Using Hu...
 
Notes on attention mechanism
Notes on attention mechanismNotes on attention mechanism
Notes on attention mechanism
 
RNN & LSTM: Neural Network for Sequential Data
RNN & LSTM: Neural Network for Sequential DataRNN & LSTM: Neural Network for Sequential Data
RNN & LSTM: Neural Network for Sequential Data
 
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language UnderstandingBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
 
Deep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog DetectorDeep Learning as a Cat/Dog Detector
Deep Learning as a Cat/Dog Detector
 
[Paper Reading] Attention is All You Need
[Paper Reading] Attention is All You Need[Paper Reading] Attention is All You Need
[Paper Reading] Attention is All You Need
 
Generating Sequences with Deep LSTMs & RNNS in julia
Generating Sequences with Deep LSTMs & RNNS in juliaGenerating Sequences with Deep LSTMs & RNNS in julia
Generating Sequences with Deep LSTMs & RNNS in julia
 
BERT
BERTBERT
BERT
 
[Paper review] BERT
[Paper review] BERT[Paper review] BERT
[Paper review] BERT
 
BERT: Bidirectional Encoder Representations from Transformers
BERT: Bidirectional Encoder Representations from TransformersBERT: Bidirectional Encoder Representations from Transformers
BERT: Bidirectional Encoder Representations from Transformers
 
[246]reasoning, attention and memory toward differentiable reasoning machines
[246]reasoning, attention and memory   toward differentiable reasoning machines[246]reasoning, attention and memory   toward differentiable reasoning machines
[246]reasoning, attention and memory toward differentiable reasoning machines
 
The Factoring Dead: Preparing for the Cryptopocalypse
The Factoring Dead: Preparing for the CryptopocalypseThe Factoring Dead: Preparing for the Cryptopocalypse
The Factoring Dead: Preparing for the Cryptopocalypse
 
Chris Fregly, Research Scientist, PipelineIO at MLconf ATL 2016
Chris Fregly, Research Scientist, PipelineIO at MLconf ATL 2016Chris Fregly, Research Scientist, PipelineIO at MLconf ATL 2016
Chris Fregly, Research Scientist, PipelineIO at MLconf ATL 2016
 
Mentoring Session with Innovesia: Advance Robotics
Mentoring Session with Innovesia: Advance RoboticsMentoring Session with Innovesia: Advance Robotics
Mentoring Session with Innovesia: Advance Robotics
 
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...
Sioux Hot-or-Not: Functional programming: unlocking the real power of multi-c...
 
Intro to Data Structure & Algorithms
Intro to Data Structure & AlgorithmsIntro to Data Structure & Algorithms
Intro to Data Structure & Algorithms
 
Master Thesis of Computer Engineering: OpenTranslator
Master Thesis of Computer Engineering: OpenTranslatorMaster Thesis of Computer Engineering: OpenTranslator
Master Thesis of Computer Engineering: OpenTranslator
 
Neural Networks and Deep Learning
Neural Networks and Deep LearningNeural Networks and Deep Learning
Neural Networks and Deep Learning
 

Viewers also liked

NLP from scratch
NLP from scratch NLP from scratch
NLP from scratch
Bryan Gummibearehausen
 
Google's Multilingual Neural Machine Translation System
Google's Multilingual Neural Machine Translation SystemGoogle's Multilingual Neural Machine Translation System
Google's Multilingual Neural Machine Translation System
NamHyuk Ahn
 
投影片範本 Papago!mobile ppt
投影片範本 Papago!mobile ppt投影片範本 Papago!mobile ppt
投影片範本 Papago!mobile ppt
XuanJun Lin
 
Machine Translation: The Neural Frontier
Machine Translation: The Neural FrontierMachine Translation: The Neural Frontier
Machine Translation: The Neural Frontier
Iconic Translation Machines
 
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRANDeep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
TAUS - The Language Data Network
 
Recurrent neural networks
Recurrent neural networksRecurrent neural networks
Recurrent neural networks
Viacheslav Khomenko
 
Machine Translation And Computer Assisted Translation
Machine Translation And Computer Assisted TranslationMachine Translation And Computer Assisted Translation
Machine Translation And Computer Assisted Translation
Teritaa
 
Introduction to Recurrent Neural Network with Application to Sentiment Analys...
Introduction to Recurrent Neural Network with Application to Sentiment Analys...Introduction to Recurrent Neural Network with Application to Sentiment Analys...
Introduction to Recurrent Neural Network with Application to Sentiment Analys...
Artifacia
 
Transfer Learning and Fine Tuning for Cross Domain Image Classification with ...
Transfer Learning and Fine Tuning for Cross Domain Image Classification with ...Transfer Learning and Fine Tuning for Cross Domain Image Classification with ...
Transfer Learning and Fine Tuning for Cross Domain Image Classification with ...
Sujit Pal
 
What is machine translation
What is machine translationWhat is machine translation
What is machine translation
Stephen Peacock
 
[224] backend 개발자의 neural machine translation 개발기 김상경
[224] backend 개발자의 neural machine translation 개발기 김상경[224] backend 개발자의 neural machine translation 개발기 김상경
[224] backend 개발자의 neural machine translation 개발기 김상경
NAVER D2
 
[134]papago 김준석
[134]papago 김준석[134]papago 김준석
[134]papago 김준석
NAVER D2
 
Types of machine translation
Types of machine translationTypes of machine translation
Types of machine translation
Rushdi Shams
 
Deep Learning for Computer Vision: Recurrent Neural Networks (UPC 2016)
Deep Learning for Computer Vision: Recurrent Neural Networks (UPC 2016)Deep Learning for Computer Vision: Recurrent Neural Networks (UPC 2016)
Deep Learning for Computer Vision: Recurrent Neural Networks (UPC 2016)
Universitat Politècnica de Catalunya
 
Transfer Learning and Fine-tuning Deep Neural Networks
 Transfer Learning and Fine-tuning Deep Neural Networks Transfer Learning and Fine-tuning Deep Neural Networks
Transfer Learning and Fine-tuning Deep Neural Networks
PyData
 
A Brief Introduction on Recurrent Neural Network and Its Application
A Brief Introduction on Recurrent Neural Network and Its ApplicationA Brief Introduction on Recurrent Neural Network and Its Application
A Brief Introduction on Recurrent Neural Network and Its Application
Xiaohu ZHU
 
Data Science, Machine Learning and Neural Networks
Data Science, Machine Learning and Neural NetworksData Science, Machine Learning and Neural Networks
Data Science, Machine Learning and Neural Networks
BICA Labs
 
Papago/N2MT 개발이야기
Papago/N2MT 개발이야기Papago/N2MT 개발이야기
Papago/N2MT 개발이야기
NAVER D2
 
Machine Translation: What it is?
Machine Translation: What it is?Machine Translation: What it is?
Machine Translation: What it is?
Multilizer
 

Viewers also liked (19)

NLP from scratch
NLP from scratch NLP from scratch
NLP from scratch
 
Google's Multilingual Neural Machine Translation System
Google's Multilingual Neural Machine Translation SystemGoogle's Multilingual Neural Machine Translation System
Google's Multilingual Neural Machine Translation System
 
投影片範本 Papago!mobile ppt
投影片範本 Papago!mobile ppt投影片範本 Papago!mobile ppt
投影片範本 Papago!mobile ppt
 
Machine Translation: The Neural Frontier
Machine Translation: The Neural FrontierMachine Translation: The Neural Frontier
Machine Translation: The Neural Frontier
 
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRANDeep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
Deep Learning for Machine Translation, by Satoshi Enoue, SYSTRAN
 
Recurrent neural networks
Recurrent neural networksRecurrent neural networks
Recurrent neural networks
 
Machine Translation And Computer Assisted Translation
Machine Translation And Computer Assisted TranslationMachine Translation And Computer Assisted Translation
Machine Translation And Computer Assisted Translation
 
Introduction to Recurrent Neural Network with Application to Sentiment Analys...
Introduction to Recurrent Neural Network with Application to Sentiment Analys...Introduction to Recurrent Neural Network with Application to Sentiment Analys...
Introduction to Recurrent Neural Network with Application to Sentiment Analys...
 
Transfer Learning and Fine Tuning for Cross Domain Image Classification with ...
Transfer Learning and Fine Tuning for Cross Domain Image Classification with ...Transfer Learning and Fine Tuning for Cross Domain Image Classification with ...
Transfer Learning and Fine Tuning for Cross Domain Image Classification with ...
 
What is machine translation
What is machine translationWhat is machine translation
What is machine translation
 
[224] backend 개발자의 neural machine translation 개발기 김상경
[224] backend 개발자의 neural machine translation 개발기 김상경[224] backend 개발자의 neural machine translation 개발기 김상경
[224] backend 개발자의 neural machine translation 개발기 김상경
 
[134]papago 김준석
[134]papago 김준석[134]papago 김준석
[134]papago 김준석
 
Types of machine translation
Types of machine translationTypes of machine translation
Types of machine translation
 
Deep Learning for Computer Vision: Recurrent Neural Networks (UPC 2016)
Deep Learning for Computer Vision: Recurrent Neural Networks (UPC 2016)Deep Learning for Computer Vision: Recurrent Neural Networks (UPC 2016)
Deep Learning for Computer Vision: Recurrent Neural Networks (UPC 2016)
 
Transfer Learning and Fine-tuning Deep Neural Networks
 Transfer Learning and Fine-tuning Deep Neural Networks Transfer Learning and Fine-tuning Deep Neural Networks
Transfer Learning and Fine-tuning Deep Neural Networks
 
A Brief Introduction on Recurrent Neural Network and Its Application
A Brief Introduction on Recurrent Neural Network and Its ApplicationA Brief Introduction on Recurrent Neural Network and Its Application
A Brief Introduction on Recurrent Neural Network and Its Application
 
Data Science, Machine Learning and Neural Networks
Data Science, Machine Learning and Neural NetworksData Science, Machine Learning and Neural Networks
Data Science, Machine Learning and Neural Networks
 
Papago/N2MT 개발이야기
Papago/N2MT 개발이야기Papago/N2MT 개발이야기
Papago/N2MT 개발이야기
 
Machine Translation: What it is?
Machine Translation: What it is?Machine Translation: What it is?
Machine Translation: What it is?
 

Similar to Deep Learning for Machine Translation, by Jean Senellart, SYSTRAN

Understanding deep learning
Understanding deep learningUnderstanding deep learning
Understanding deep learning
Dr. Stylianos Kampakis
 
OWF14 - Big Data : The State of Machine Learning in 2014
OWF14 - Big Data : The State of Machine  Learning in 2014OWF14 - Big Data : The State of Machine  Learning in 2014
OWF14 - Big Data : The State of Machine Learning in 2014
Paris Open Source Summit
 
Building a Neural Machine Translation System From Scratch
Building a Neural Machine Translation System From ScratchBuilding a Neural Machine Translation System From Scratch
Building a Neural Machine Translation System From Scratch
Natasha Latysheva
 
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech TalksA Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
Amazon Web Services
 
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech TalksA Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
Amazon Web Services
 
P1 2017 python
P1 2017 pythonP1 2017 python
P1 2017 python
Prof. Wim Van Criekinge
 
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習 Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習
Herman Wu
 
Amazon Deep Learning
Amazon Deep LearningAmazon Deep Learning
Amazon Deep Learning
Amanda Mackay (she/her)
 
P1 2018 python
P1 2018 pythonP1 2018 python
P1 2018 python
Prof. Wim Van Criekinge
 
Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016
Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016
Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016
MLconf
 
2016 bioinformatics i_python_part_1_wim_vancriekinge
2016 bioinformatics i_python_part_1_wim_vancriekinge2016 bioinformatics i_python_part_1_wim_vancriekinge
2016 bioinformatics i_python_part_1_wim_vancriekinge
Prof. Wim Van Criekinge
 
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)AWS re:Invent 2016: Deep Learning in Alexa (MAC202)
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)
Amazon Web Services
 
Deep Learning Frameworks 2019 | Which Deep Learning Framework To Use | Deep L...
Deep Learning Frameworks 2019 | Which Deep Learning Framework To Use | Deep L...Deep Learning Frameworks 2019 | Which Deep Learning Framework To Use | Deep L...
Deep Learning Frameworks 2019 | Which Deep Learning Framework To Use | Deep L...
Simplilearn
 
Python for Data: Past, Present, Future (PyCon JP 2017 Keynote)
Python for Data: Past, Present, Future (PyCon JP 2017 Keynote)Python for Data: Past, Present, Future (PyCon JP 2017 Keynote)
Python for Data: Past, Present, Future (PyCon JP 2017 Keynote)
Peter Wang
 
Melbourne Cocoa Heads CoreML Presentation
Melbourne Cocoa Heads CoreML PresentationMelbourne Cocoa Heads CoreML Presentation
Melbourne Cocoa Heads CoreML Presentation
Hon Weng Chong
 
Hoe een efficiënte Machine of Deep Learning backend ontwikkelen?
Hoe een efficiënte Machine of Deep Learning backend ontwikkelen?Hoe een efficiënte Machine of Deep Learning backend ontwikkelen?
Hoe een efficiënte Machine of Deep Learning backend ontwikkelen?
Agentschap Innoveren & Ondernemen
 
AI and Innovations on AWS
AI and Innovations on AWSAI and Innovations on AWS
AI and Innovations on AWS
Adrian Hornsby
 
An Introduction to Natural Language Processing
An Introduction to Natural Language ProcessingAn Introduction to Natural Language Processing
An Introduction to Natural Language Processing
Tyrone Systems
 
Deep Learning and Recurrent Neural Networks in the Enterprise
Deep Learning and Recurrent Neural Networks in the EnterpriseDeep Learning and Recurrent Neural Networks in the Enterprise
Deep Learning and Recurrent Neural Networks in the Enterprise
Josh Patterson
 
Big data 4 webmonday
Big data 4 webmondayBig data 4 webmonday
Big data 4 webmonday
Daniel Koller
 

Similar to Deep Learning for Machine Translation, by Jean Senellart, SYSTRAN (20)

Understanding deep learning
Understanding deep learningUnderstanding deep learning
Understanding deep learning
 
OWF14 - Big Data : The State of Machine Learning in 2014
OWF14 - Big Data : The State of Machine  Learning in 2014OWF14 - Big Data : The State of Machine  Learning in 2014
OWF14 - Big Data : The State of Machine Learning in 2014
 
Building a Neural Machine Translation System From Scratch
Building a Neural Machine Translation System From ScratchBuilding a Neural Machine Translation System From Scratch
Building a Neural Machine Translation System From Scratch
 
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech TalksA Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
 
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech TalksA Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
A Deeper Dive into Apache MXNet - March 2017 AWS Online Tech Talks
 
P1 2017 python
P1 2017 pythonP1 2017 python
P1 2017 python
 
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習 Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習
Azure 機器學習 - 使用Python, R, Spark, CNTK 深度學習
 
Amazon Deep Learning
Amazon Deep LearningAmazon Deep Learning
Amazon Deep Learning
 
P1 2018 python
P1 2018 pythonP1 2018 python
P1 2018 python
 
Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016
Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016
Hussein Mehanna, Engineering Director, ML Core - Facebook at MLconf ATL 2016
 
2016 bioinformatics i_python_part_1_wim_vancriekinge
2016 bioinformatics i_python_part_1_wim_vancriekinge2016 bioinformatics i_python_part_1_wim_vancriekinge
2016 bioinformatics i_python_part_1_wim_vancriekinge
 
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)AWS re:Invent 2016: Deep Learning in Alexa (MAC202)
AWS re:Invent 2016: Deep Learning in Alexa (MAC202)
 
Deep Learning Frameworks 2019 | Which Deep Learning Framework To Use | Deep L...
Deep Learning Frameworks 2019 | Which Deep Learning Framework To Use | Deep L...Deep Learning Frameworks 2019 | Which Deep Learning Framework To Use | Deep L...
Deep Learning Frameworks 2019 | Which Deep Learning Framework To Use | Deep L...
 
Python for Data: Past, Present, Future (PyCon JP 2017 Keynote)
Python for Data: Past, Present, Future (PyCon JP 2017 Keynote)Python for Data: Past, Present, Future (PyCon JP 2017 Keynote)
Python for Data: Past, Present, Future (PyCon JP 2017 Keynote)
 
Melbourne Cocoa Heads CoreML Presentation
Melbourne Cocoa Heads CoreML PresentationMelbourne Cocoa Heads CoreML Presentation
Melbourne Cocoa Heads CoreML Presentation
 
Hoe een efficiënte Machine of Deep Learning backend ontwikkelen?
Hoe een efficiënte Machine of Deep Learning backend ontwikkelen?Hoe een efficiënte Machine of Deep Learning backend ontwikkelen?
Hoe een efficiënte Machine of Deep Learning backend ontwikkelen?
 
AI and Innovations on AWS
AI and Innovations on AWSAI and Innovations on AWS
AI and Innovations on AWS
 
An Introduction to Natural Language Processing
An Introduction to Natural Language ProcessingAn Introduction to Natural Language Processing
An Introduction to Natural Language Processing
 
Deep Learning and Recurrent Neural Networks in the Enterprise
Deep Learning and Recurrent Neural Networks in the EnterpriseDeep Learning and Recurrent Neural Networks in the Enterprise
Deep Learning and Recurrent Neural Networks in the Enterprise
 
Big data 4 webmonday
Big data 4 webmondayBig data 4 webmonday
Big data 4 webmonday
 

More from TAUS - The Language Data Network

TAUS Global Content Summit Amsterdam 2019 / Beyond MT. A few premature reflec...
TAUS Global Content Summit Amsterdam 2019 / Beyond MT. A few premature reflec...TAUS Global Content Summit Amsterdam 2019 / Beyond MT. A few premature reflec...
TAUS Global Content Summit Amsterdam 2019 / Beyond MT. A few premature reflec...
TAUS - The Language Data Network
 
TAUS Global Content Summit Amsterdam 2019 / Measure with DQF, Dace Dzeguze (T...
TAUS Global Content Summit Amsterdam 2019 / Measure with DQF, Dace Dzeguze (T...TAUS Global Content Summit Amsterdam 2019 / Measure with DQF, Dace Dzeguze (T...
TAUS Global Content Summit Amsterdam 2019 / Measure with DQF, Dace Dzeguze (T...
TAUS - The Language Data Network
 
TAUS Global Content Summit Amsterdam 2019 / Automatic for the People by Domin...
TAUS Global Content Summit Amsterdam 2019 / Automatic for the People by Domin...TAUS Global Content Summit Amsterdam 2019 / Automatic for the People by Domin...
TAUS Global Content Summit Amsterdam 2019 / Automatic for the People by Domin...
TAUS - The Language Data Network
 
TAUS Global Content Summit Amsterdam 2019 / The Quantum Leap: Human Parity, C...
TAUS Global Content Summit Amsterdam 2019 / The Quantum Leap: Human Parity, C...TAUS Global Content Summit Amsterdam 2019 / The Quantum Leap: Human Parity, C...
TAUS Global Content Summit Amsterdam 2019 / The Quantum Leap: Human Parity, C...
TAUS - The Language Data Network
 
TAUS Global Content Summit Amsterdam 2019 / Growing Business by Connecting Co...
TAUS Global Content Summit Amsterdam 2019 / Growing Business by Connecting Co...TAUS Global Content Summit Amsterdam 2019 / Growing Business by Connecting Co...
TAUS Global Content Summit Amsterdam 2019 / Growing Business by Connecting Co...
TAUS - The Language Data Network
 
Achieving Translation Efficiency and Accuracy for Video Content, Xiao Yuan (P...
Achieving Translation Efficiency and Accuracy for Video Content, Xiao Yuan (P...Achieving Translation Efficiency and Accuracy for Video Content, Xiao Yuan (P...
Achieving Translation Efficiency and Accuracy for Video Content, Xiao Yuan (P...
TAUS - The Language Data Network
 
Introduction Innovation Contest Shenzhen by Henri Broekmate (Lionbridge)
Introduction Innovation Contest Shenzhen by Henri Broekmate (Lionbridge)Introduction Innovation Contest Shenzhen by Henri Broekmate (Lionbridge)
Introduction Innovation Contest Shenzhen by Henri Broekmate (Lionbridge)
TAUS - The Language Data Network
 
Game Changer for Linguistic Review: Shifting the Paradigm, Klaus Fleischmann...
 Game Changer for Linguistic Review: Shifting the Paradigm, Klaus Fleischmann... Game Changer for Linguistic Review: Shifting the Paradigm, Klaus Fleischmann...
Game Changer for Linguistic Review: Shifting the Paradigm, Klaus Fleischmann...
TAUS - The Language Data Network
 
A translation memory P2P trading platform - to make global translation memory...
A translation memory P2P trading platform - to make global translation memory...A translation memory P2P trading platform - to make global translation memory...
A translation memory P2P trading platform - to make global translation memory...
TAUS - The Language Data Network
 
Shiyibao — The Most Efficient Translation Feedback System Ever, Guanqing Hao ...
Shiyibao — The Most Efficient Translation Feedback System Ever, Guanqing Hao ...Shiyibao — The Most Efficient Translation Feedback System Ever, Guanqing Hao ...
Shiyibao — The Most Efficient Translation Feedback System Ever, Guanqing Hao ...
TAUS - The Language Data Network
 
Stepes – Instant Human Translation Services for the Digital World, Carl Yao (...
Stepes – Instant Human Translation Services for the Digital World, Carl Yao (...Stepes – Instant Human Translation Services for the Digital World, Carl Yao (...
Stepes – Instant Human Translation Services for the Digital World, Carl Yao (...
TAUS - The Language Data Network
 
Farmer Lv (TrueTran)
Farmer Lv (TrueTran)Farmer Lv (TrueTran)
Farmer Lv (TrueTran)
TAUS - The Language Data Network
 
Smart Translation Resource Management: Semantic Matching, Kirk Zhang (Wiitran...
Smart Translation Resource Management: Semantic Matching, Kirk Zhang (Wiitran...Smart Translation Resource Management: Semantic Matching, Kirk Zhang (Wiitran...
Smart Translation Resource Management: Semantic Matching, Kirk Zhang (Wiitran...
TAUS - The Language Data Network
 
The Theory and Practice of Computer Aided Translation Training System, Liu Q...
 The Theory and Practice of Computer Aided Translation Training System, Liu Q... The Theory and Practice of Computer Aided Translation Training System, Liu Q...
The Theory and Practice of Computer Aided Translation Training System, Liu Q...
TAUS - The Language Data Network
 
Translation Technology Showcase in Shenzhen
Translation Technology Showcase in ShenzhenTranslation Technology Showcase in Shenzhen
Translation Technology Showcase in Shenzhen
TAUS - The Language Data Network
 
How to efficiently use large-scale TMs in translation, Jing Zhang (Tmxmall)
How to efficiently use large-scale TMs in translation, Jing Zhang (Tmxmall)How to efficiently use large-scale TMs in translation, Jing Zhang (Tmxmall)
How to efficiently use large-scale TMs in translation, Jing Zhang (Tmxmall)
TAUS - The Language Data Network
 
SDL Trados Studio 2017, Jocelyn He (SDL)
SDL Trados Studio 2017, Jocelyn He (SDL)SDL Trados Studio 2017, Jocelyn He (SDL)
SDL Trados Studio 2017, Jocelyn He (SDL)
TAUS - The Language Data Network
 
How we train post-editors - Yongpeng Wei (Lingosail)
How we train post-editors - Yongpeng Wei (Lingosail)How we train post-editors - Yongpeng Wei (Lingosail)
How we train post-editors - Yongpeng Wei (Lingosail)
TAUS - The Language Data Network
 
A use-case for getting MT into your company, Kerstin Berns (berns language c...
 A use-case for getting MT into your company, Kerstin Berns (berns language c... A use-case for getting MT into your company, Kerstin Berns (berns language c...
A use-case for getting MT into your company, Kerstin Berns (berns language c...
TAUS - The Language Data Network
 
QE integrated in XTM, by Bob Willans (XTM)
QE integrated in XTM, by Bob Willans (XTM)QE integrated in XTM, by Bob Willans (XTM)
QE integrated in XTM, by Bob Willans (XTM)
TAUS - The Language Data Network
 

More from TAUS - The Language Data Network (20)

TAUS Global Content Summit Amsterdam 2019 / Beyond MT. A few premature reflec...
TAUS Global Content Summit Amsterdam 2019 / Beyond MT. A few premature reflec...TAUS Global Content Summit Amsterdam 2019 / Beyond MT. A few premature reflec...
TAUS Global Content Summit Amsterdam 2019 / Beyond MT. A few premature reflec...
 
TAUS Global Content Summit Amsterdam 2019 / Measure with DQF, Dace Dzeguze (T...
TAUS Global Content Summit Amsterdam 2019 / Measure with DQF, Dace Dzeguze (T...TAUS Global Content Summit Amsterdam 2019 / Measure with DQF, Dace Dzeguze (T...
TAUS Global Content Summit Amsterdam 2019 / Measure with DQF, Dace Dzeguze (T...
 
TAUS Global Content Summit Amsterdam 2019 / Automatic for the People by Domin...
TAUS Global Content Summit Amsterdam 2019 / Automatic for the People by Domin...TAUS Global Content Summit Amsterdam 2019 / Automatic for the People by Domin...
TAUS Global Content Summit Amsterdam 2019 / Automatic for the People by Domin...
 
TAUS Global Content Summit Amsterdam 2019 / The Quantum Leap: Human Parity, C...
TAUS Global Content Summit Amsterdam 2019 / The Quantum Leap: Human Parity, C...TAUS Global Content Summit Amsterdam 2019 / The Quantum Leap: Human Parity, C...
TAUS Global Content Summit Amsterdam 2019 / The Quantum Leap: Human Parity, C...
 
TAUS Global Content Summit Amsterdam 2019 / Growing Business by Connecting Co...
TAUS Global Content Summit Amsterdam 2019 / Growing Business by Connecting Co...TAUS Global Content Summit Amsterdam 2019 / Growing Business by Connecting Co...
TAUS Global Content Summit Amsterdam 2019 / Growing Business by Connecting Co...
 
Achieving Translation Efficiency and Accuracy for Video Content, Xiao Yuan (P...
Achieving Translation Efficiency and Accuracy for Video Content, Xiao Yuan (P...Achieving Translation Efficiency and Accuracy for Video Content, Xiao Yuan (P...
Achieving Translation Efficiency and Accuracy for Video Content, Xiao Yuan (P...
 
Introduction Innovation Contest Shenzhen by Henri Broekmate (Lionbridge)
Introduction Innovation Contest Shenzhen by Henri Broekmate (Lionbridge)Introduction Innovation Contest Shenzhen by Henri Broekmate (Lionbridge)
Introduction Innovation Contest Shenzhen by Henri Broekmate (Lionbridge)
 
Game Changer for Linguistic Review: Shifting the Paradigm, Klaus Fleischmann...
 Game Changer for Linguistic Review: Shifting the Paradigm, Klaus Fleischmann... Game Changer for Linguistic Review: Shifting the Paradigm, Klaus Fleischmann...
Game Changer for Linguistic Review: Shifting the Paradigm, Klaus Fleischmann...
 
A translation memory P2P trading platform - to make global translation memory...
A translation memory P2P trading platform - to make global translation memory...A translation memory P2P trading platform - to make global translation memory...
A translation memory P2P trading platform - to make global translation memory...
 
Shiyibao — The Most Efficient Translation Feedback System Ever, Guanqing Hao ...
Shiyibao — The Most Efficient Translation Feedback System Ever, Guanqing Hao ...Shiyibao — The Most Efficient Translation Feedback System Ever, Guanqing Hao ...
Shiyibao — The Most Efficient Translation Feedback System Ever, Guanqing Hao ...
 
Stepes – Instant Human Translation Services for the Digital World, Carl Yao (...
Stepes – Instant Human Translation Services for the Digital World, Carl Yao (...Stepes – Instant Human Translation Services for the Digital World, Carl Yao (...
Stepes – Instant Human Translation Services for the Digital World, Carl Yao (...
 
Farmer Lv (TrueTran)
Farmer Lv (TrueTran)Farmer Lv (TrueTran)
Farmer Lv (TrueTran)
 
Smart Translation Resource Management: Semantic Matching, Kirk Zhang (Wiitran...
Smart Translation Resource Management: Semantic Matching, Kirk Zhang (Wiitran...Smart Translation Resource Management: Semantic Matching, Kirk Zhang (Wiitran...
Smart Translation Resource Management: Semantic Matching, Kirk Zhang (Wiitran...
 
The Theory and Practice of Computer Aided Translation Training System, Liu Q...
 The Theory and Practice of Computer Aided Translation Training System, Liu Q... The Theory and Practice of Computer Aided Translation Training System, Liu Q...
The Theory and Practice of Computer Aided Translation Training System, Liu Q...
 
Translation Technology Showcase in Shenzhen
Translation Technology Showcase in ShenzhenTranslation Technology Showcase in Shenzhen
Translation Technology Showcase in Shenzhen
 
How to efficiently use large-scale TMs in translation, Jing Zhang (Tmxmall)
How to efficiently use large-scale TMs in translation, Jing Zhang (Tmxmall)How to efficiently use large-scale TMs in translation, Jing Zhang (Tmxmall)
How to efficiently use large-scale TMs in translation, Jing Zhang (Tmxmall)
 
SDL Trados Studio 2017, Jocelyn He (SDL)
SDL Trados Studio 2017, Jocelyn He (SDL)SDL Trados Studio 2017, Jocelyn He (SDL)
SDL Trados Studio 2017, Jocelyn He (SDL)
 
How we train post-editors - Yongpeng Wei (Lingosail)
How we train post-editors - Yongpeng Wei (Lingosail)How we train post-editors - Yongpeng Wei (Lingosail)
How we train post-editors - Yongpeng Wei (Lingosail)
 
A use-case for getting MT into your company, Kerstin Berns (berns language c...
 A use-case for getting MT into your company, Kerstin Berns (berns language c... A use-case for getting MT into your company, Kerstin Berns (berns language c...
A use-case for getting MT into your company, Kerstin Berns (berns language c...
 
QE integrated in XTM, by Bob Willans (XTM)
QE integrated in XTM, by Bob Willans (XTM)QE integrated in XTM, by Bob Willans (XTM)
QE integrated in XTM, by Bob Willans (XTM)
 

Recently uploaded

Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
Jason Packer
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
DianaGray10
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
saastr
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
Data Hops
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
Jason Yip
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
DanBrown980551
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
Tatiana Kojar
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
Jakub Marek
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
ScyllaDB
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
Brandon Minnick, MBA
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Alpen-Adria-Universität
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
tolgahangng
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
AstuteBusiness
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
Fwdays
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
Antonios Katsarakis
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
Zilliz
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
marufrahmanstratejm
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
Hiroshi SHIBATA
 

Recently uploaded (20)

Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024Columbus Data & Analytics Wednesdays - June 2024
Columbus Data & Analytics Wednesdays - June 2024
 
What is an RPA CoE? Session 1 – CoE Vision
What is an RPA CoE?  Session 1 – CoE VisionWhat is an RPA CoE?  Session 1 – CoE Vision
What is an RPA CoE? Session 1 – CoE Vision
 
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
9 CEO's who hit $100m ARR Share Their Top Growth Tactics Nathan Latka, Founde...
 
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3FREE A4 Cyber Security Awareness  Posters-Social Engineering part 3
FREE A4 Cyber Security Awareness Posters-Social Engineering part 3
 
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
[OReilly Superstream] Occupy the Space: A grassroots guide to engineering (an...
 
5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides5th LF Energy Power Grid Model Meet-up Slides
5th LF Energy Power Grid Model Meet-up Slides
 
Skybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoptionSkybuffer SAM4U tool for SAP license adoption
Skybuffer SAM4U tool for SAP license adoption
 
Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)Main news related to the CCS TSI 2023 (2023/1695)
Main news related to the CCS TSI 2023 (2023/1695)
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyFreshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-Efficiency
 
Choosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptxChoosing The Best AWS Service For Your Website + API.pptx
Choosing The Best AWS Service For Your Website + API.pptx
 
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing InstancesEnergy Efficient Video Encoding for Cloud and Edge Computing Instances
Energy Efficient Video Encoding for Cloud and Edge Computing Instances
 
Serial Arm Control in Real Time Presentation
Serial Arm Control in Real Time PresentationSerial Arm Control in Real Time Presentation
Serial Arm Control in Real Time Presentation
 
Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |Astute Business Solutions | Oracle Cloud Partner |
Astute Business Solutions | Oracle Cloud Partner |
 
"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota"Choosing proper type of scaling", Olena Syrota
"Choosing proper type of scaling", Olena Syrota
 
TrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc Webinar - 2024 Global Privacy Survey
TrustArc Webinar - 2024 Global Privacy Survey
 
Dandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity serverDandelion Hashtable: beyond billion requests per second on a commodity server
Dandelion Hashtable: beyond billion requests per second on a commodity server
 
Programming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup SlidesProgramming Foundation Models with DSPy - Meetup Slides
Programming Foundation Models with DSPy - Meetup Slides
 
Public CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptxPublic CyberSecurity Awareness Presentation 2024.pptx
Public CyberSecurity Awareness Presentation 2024.pptx
 
Introduction of Cybersecurity with OSS at Code Europe 2024
Introduction of Cybersecurity with OSS  at Code Europe 2024Introduction of Cybersecurity with OSS  at Code Europe 2024
Introduction of Cybersecurity with OSS at Code Europe 2024
 

Deep Learning for Machine Translation, by Jean Senellart, SYSTRAN

  • 1. Deep Learning for Machine Translation Satoshi Enoue, Jungi Kim, Jean Senellart, SYSTRAN
  • 2. SYSTRAN Through Machine Translation History Rule Base Machine Translation Example-Based Machine Translation Phrase Based Machine Translation Syntax Based Machine Translation Neural Machine Translation Hybrid Machine Translation SYSTRAN 197 1968 SYSTRAN (SYStem TRANslation) founded by Dr. Toma in La Jolla, California (USA) 1969 Provided first MT software for the US Air Force, (Russian to English) 1975 Used by NASA for the Apollo- Soyuz American-Soviet project 1975 Translation systems for all European languages in the European Commission 1986 SYSTRAN is acquired by France’s Gachot SA, thus becoming a French company with a U.S. subsidiary 1995 Pioneered development of first Windows-based MT software 1997 First free Web-based translation service: Altavista Babelfish. SYSTRAN made the Internet community aware of the usefulness and capabilities of machine translation 2002 SYSTRAN was used on most major Internet Portals: Yahoo!, Google, AltaVista, Lycos. 1996 SYSTRAN within SEIKO’s pocket translators. 1990’s Port technology from mainframes to Desktop PC’s and Client-Server environments for personal and corporate use 2014 Following acquisition by CSLI, SYSTRAN SA forms part of the SYSTRAN International Group 2005 Launched embedded translation software for mobile devices 2009 Developed first hybrid translation software and solution: SES 7 Translation Server 2011 Launch of SES 7 Training Server, first solution for self-learning of MT engines 2015 SES8 Translation and Training Server – Large Models 2016 More than 140 language Pairs. Launch of SYSTRAN.io, the Natural Language Processing API platform
  • 3. The new game changer • Deep Neural Network Technologies • Image Analysis • Voice Recognition • Text • Text Generation • Word Embeddings • Multitask NLP • Neural Machine Translation • … Games • Super Human Abilities Sequence of fascinating results and technologies over the last 3 years – all based on Deep Neural Network (DNN) – covering a large variety of domains… 26/04/2016 SYSTRAN - Copyright 2016 3
  • 4. The new game changer • Deep Neural Network Technologies • Image Analysis • Voice Recognition • Text • Text Generation • Word Embeddings • Multitask NLP • Neural Machine Translation • … Games 26/04/2016 SYSTRAN - Copyright 2016 4
  • 5. The new game changer • Deep Neural Network Technologies • Image Analysis • Voice Recognition • Text • Text Generation • Word Embeddings • Multitask NLP • Neural Machine Translation • … Games Abilities • Google 2015 RNN voice search recognition outperforms 2012 DNN models • Baidu Deep Speech announces 16.5% improvement over baseline and higher performance than human in noisy environment 26/04/2016 SYSTRAN - Copyright 2016 5
  • 6. The new game changer - examples • Deep Neural Network Technologies • Image Analysis • Voice Recognition • Text • Text Generation • Word Embeddings • Multitask NLP • Neural Machine Translation • … Gamesn Abilities Les yeux prenaient des redoutables, des troncs de feu. Toutes les prétexticheurs par ces quatre repentilleuses avec du sergent de Digne, débragiffés nymoeurs sur les derniers instants à hardis, boucher, sans dénongée en plus ennérence, ils se refecturent encore. Ils auraient déjà mangé ses très interses. ShakespeareVictorHugo Char-RNN, Andrej Karpathy, 2015 공급자는 AspNetXSprchyLibrary의 인스턴스를 만들어 다른 경고를 오버터 컴퓨터에 저장할 수 있습니다. MSDN 26/04/2016 SYSTRAN - Copyright 2016
  • 9. The new game changer - examples • Deep Neural Network Technologies • Image Analysis • Voice Recognition • Text • Text Generation • Word Embeddings • Multitask NLP • Neural Machine Translation • … Games 26/04/2016 9word2vec, Google, 2013
  • 10. The new game changer - examples • Unified Neural Network Architecture for several NLP tasks POS tagging, chunking, NER, SRL • Focus on avoiding task/linguistic specific engineering • Joint decision on the different tasks Outperforms almost all of the state of the art results for each individual tasks Natural Language Processing (Almost) from Scratch, Collobert et al., 201126/04/2016 • Deep Neural Network Technologies • Image Analysis • Voice Recognition • Text • Text Generation • Word Embeddings • Multitask NLP • Neural Machine Translation • … Gamesn Abilities
  • 11. The new game changer - examples • Deep Neural Network Technologies • Image Analysis • Voice Recognition • Text • Text Generation • Word Embeddings • Multitask NLP • Neural Machine Translation: sentence encoding-decoding • … Games Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation, K. Cho et al, 2014
  • 12. The new game changer - examples • Deep Neural Network Technologies • Image Analysis • Voice Recognition • Text • Text Generation • Word Embeddings • Multitask NLP • Neural Machine : sentence encoding- decoding • … Games – DQN, AlphaGo HUMAN-LEVEL CONTROL THROUGH DEEP REINFORCEMENT LEARNING, Google DeepMind, 201526/04/2016
  • 13. AlphaGo, Google DeepMind, 201626/04/2016 SYSTRAN - Copyright 2016
  • 14. The new game changer - examples More and more evidence of “super-human abilities” Could we also reach Super- human Machine Translation? 26/04/2016 SYSTRAN - Copyright 2016 14
  • 15. The new game changer – ingredients • MLP – multilayer perceptron • Actually an “old concept” • CNN • Convolutional Neural network • Word Embeddings • Representing words as vectors • RNN – GRU, LSTM • MLP with memory • Attention-Based models • Ability to decide where to find information 26/04/2016 SYSTRAN - Copyright 2016 15
  • 16. The new game changer – ingredients • MLP – multilayer perceptron • Actually an “old concept” • CNN • Convolutional Neural network • Word Embeddings • Representing words as vectors • RNN – GRU, LSTM • MLP with memory • Attention-Based models • Ability to decide where to find information 26/04/2016 SYSTRAN - Copyright 2016 16
  • 17. The new game changer – ingredients • MLP – multilayer perceptron • Actually an “old concept” • CNN • Convolutional Neural network • Word Embeddings • Representing words as vectors • RNN – GRU, LSTM • MLP with memory • Attention-Based models • Ability to decide where to find information 26/04/2016 SYSTRAN - Copyright 2016 17
  • 18. The new game changer – ingredients • MLP – multilayer perceptron • Actually an “old concept” • CNN • Convolutional Neural network • Word Embeddings • Representing words as vectors • RNN – GRU, LSTM • MLP with memory • Attention-Based models • Ability to decide where to find information 26/04/2016 SYSTRAN - Copyright 2016 18
  • 19. The new game changer – ingredients • MLP – multilayer perceptron • Actually an “old concept” • CNN • Convolutional Neural network • Word Embeddings • Representing words as vectors • RNN – GRU, LSTM • MLP with memory • Attention-Based models • Ability to decide where to find information 26/04/2016 SYSTRAN - Copyright 2016 19 All of these features are the ingredients to Neural Machine Translation
  • 20. About Neural Machine Translation (NMT) • The goal is to perform end-to-end translation • Like in Speech Recognition • The spirit is to remove all these features and have single system • For Machine Translation – first NMT systems are encoder-decoder • But not that magic • Not systematic improvements over SMT baseline • Use of ensemble systems • Issues with sentence lengths, vocabulary size • Solutions come back with some interest in “linguistic” characteristics • Attention-Based model (alignment information) • Deep Fusion with Language Model (better modelling of target language) • Combine with word level (~ morphology) 26/04/2016 SYSTRAN - Copyright 2016 20
  • 21. SYSTRAN approach to NMT • Current Real Use-Case Requirements: • Adaptation to (small) domain • Help for post-editing • Preserved speed • Consistent results amongst multiple target languages • Possibility to let users control translation through annotations, terminology • … • Toward Linguistically Motivated NN architecture • SYSTRAN MT is composed of linguistic modules – let us start with them • Lot of knowledge to leverage 26/04/2016 SYSTRAN - Copyright 2016 21
  • 22. SYSTRAN Deep Learning Story – Part I Language Identification SYSTRAN LDK 1 •Statistical Classifier – 3-grams •Heavily Feature Engineered over years •e.g. diacritics model for latin language •Include lexicon of frequent terms •Quite good accuracy on news-type data – need ~20 characters Basic RNN •“out-of-the-box” character level RNN •no specific language specific engineering •80K words training per language Google CLD •Naïve Bayesian Classifier – 4-grams •Trained on “big data” •carefully scrapped over 100M pages •Specific tricks for closely related languages (Spanish/Portuguese) •Geared for webpages - 200+ characters Learnings: with same data RNN approach easily outperforms baseline, no specific engineering needed… big data is not competing... 26/04/2016 SYSTRAN - Copyright 2016 22 News Sentences One-word request Ted-Talk Sentences Tweets LDK 97 55.2 87.4 78.3 RNN 98.2 61.5 91.4 77.9 CLD 96.1 15.3 86 78.1
  • 23. SYSTRAN Deep Learning Story – Part II Part of Speech Tagging Phase 1 - 1968-2014 - Handcrafting •Manual Rule and Lexicon Coding of homography •Closely related to Morphology description •27 languages covered Phase 2 - 2008-2015 – Annotating •Train Classifier to "relearn” rules (fnTBL) •Transfer knowledge through system output •Maintenance through Annotation Phase 3 - 2015- - Generalizing •Relearn with RNN •Joint decision (so far tokenization/part of speech tagging) – working on morphology •Better generalization from additional knowledge (word embeddings) 26/04/2016 SYSTRAN - Copyright 2016 23 Learnings: Possibility to leverage ”handcrafting” and gain quality. But learning becoming too smart – it also learns initial errors
  • 24. SYSTRAN Deep Learning Story – Part III Transliteration 26/04/2016 24 • Transliteration of person names is depending on • Source Language • Target Language • But also Name origin • 카스파로프 = Kasparov • 필리프 = Philippe • Good Transliteration system needs: • Detection of origin • Transliteration mechanism •Extremely complicated – since it requires phonetics modeling Rule-Based • Satisfactory but origin detection and multiple domains • No generalization - unseen sequence is wrong PBMT • Encoding-Decoding Approach • Long distance "view" guarantee consistency of transliteration RNN Learnings: - losing reliability/traceability of the process + more global consistency, compactness of the solution
  • 25. SYSTRAN Deep Learning Story – Part IV Language Modeling • RNN language model proves to overpass standard n-gram models • No limitation in the span • Seems to capture also better the language structure • Better generalization due to word embedding • Can be easily introduced in PBMT engine through rescoring • Are still challenging pure sequence-to-sequence NMT approaches 26/04/2016 25 Learnings: - Very long training process, several weeks of training for one language + Consistent quality gain, easy introduction in existing framework
  • 26. Learnings from Deep Learning • Consistent quality improvement in all the experiments/modules we worked on • Better leverage of existing training material • Better generalization • Incrementability: by design, it is immediate to feed more training data – i.e. adapt dynamically to usage • Globally more simple than alternative approaches and cognitively interesting • Fit to be combined in a global NN architecture 26/04/2016 SYSTRAN - Copyright 2016 26
  • 27. Linguistically Motivated NN architecture 26/04/2016 SYSTRAN - Copyright 2016 27 Morphology Syntactic Analysis Sentence Encoding Sentence Decoding RNN-LM Word Embedding Source Sentence … Target Sentence …
  • 28. What about Statistical Post Editing: Learning to correct? 26/04/2016 SYSTRAN - Copyright 2016 28 • SPE was introduced as smart alternative the SMT • Corresponding to real MT use case for localization • Very little data can produce adaptation • Reduce Human Post-Editor Work by iteratively learning edits • However implementation with PBMT is not satisfactory • PBMT does not learn to correct but to translate • Not incremental • Learning to correct • More control of the process Toward a “translation checker” • Change the paradigm – now human post- editor to MT output, tomorrow automatic post-editor to human output? MT HPE
  • 29. Deep Learning for Machine Translation • No doubt – it is coming: • We will probably reach “superhuman” machine translation in coming years • And this could become real translation assistant • How is not yet completely clear • From our perspective, we are working on hybrid approach = linguistically motivated NN architecture • More will also be coming from research world • Still some work ahead • Training of models is still a technological challenge • We need the models to explain as much as to translate to become really useful – or for language learning • Multi-level analysis - document translation and not just sentences • Multi-modal => could lead to full self language learning 26/04/2016 SYSTRAN - Copyright 2016 29

Editor's Notes

  1. The last 3 years…
  2. In Image recognition
  3. In Voice Recognition
  4. Show X is to Y what Z is to …
  5. M
  6. M
  7. M
  8. Road Sign Recognition For some tasks
  9. Actually it is not one single technology but a mix of different technologies – what is very seducing is this remains relatively simple, and appealing
  10. Convolution Neural Network are very used in the image processing – and can be seen as consecutive layers of processing that progressively extract more and more advanced features
  11. Actually it is not one single technology but a mix of different technologies – what is very seducing is this remains relatively simple, and appealing
  12. Actually it is not one single technology but a mix of different technologies – what is very seducing is this remains relatively simple, and appealing
  13. End-to-end – is also called “sequence-to-sequence”
  14. Requirements from our customer are actually quite strong – and our goal is not to produce a generic academic NMT engine, but actual solutions for our customer requirements
  15. So we would like to share with you findings of these moves to DNN and we took for that several modules
  16. Example on Chinese
  17. So we are not yet there – but what we foresee and work on is to establish a NN architecture preserving the actual traditional linguistic workflow with specialized NN stacking up to produce machine translation From this specialization – we except several things - first we would be able to use the existing knowledge, second we would still have “checkpoints” in the process allowing to monitor translation process
  18. Alternatively, the other important research directions for us – is to improve modeling on Statistical Post-Editing introduced in 2007 as an alternative to raising SMT. SPE is corresponding to real user-case: very little data, an existing system performing well but not really adapted to the task.
  19. So SYSTRANN