Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
© 2017 ALL RIGHTS RESERVED
II – SDV Congress
Benefits of RNN (Recurrent Neural
Networks) within MachineTranslation and
NLP...
© 2017 ALL RIGHTS RESERVED
SYSTRAN overview, in one slide!
2
Pioneer in machine translation, SYSTRAN enhances
understandin...
© 2017 ALL RIGHTS RESERVED
IA & Deep Learning : a convergence with MachineTranslation
3
© 2017 ALL RIGHTS RESERVED
IA & Deep Learning enters in the business world
4
HAVING REVOLUTIONIZED SPEECH AND IMAGE RECOGN...
© 2017 ALL RIGHTS RESERVED
The challenges to apply IA & Deep Learning to NLP applications
5
WHAT MAKES NATURAL LANGUAGE PR...
© 2017 ALL RIGHTS RESERVED
At SYSTRAN, we love languages, all languages
6
1.A system for the expression of thoughts,
feeli...
© 2017 ALL RIGHTS RESERVED
MachineTranslationTechnologies before AI & Deep Learning
7
ALL MT OFFERSWERE BASED ON 2 MAINTEC...
© 2017 ALL RIGHTS RESERVED
MachineTranslationTechnologies is entering in a new world !
8
NEURAL MT PRODUCES ATRANSLATIONOV...
© 2017 ALL RIGHTS RESERVED
The CoreTechnology : Artificial Neural Networks
9
- Self learning technology inspired by human ...
© 2017 ALL RIGHTS RESERVED
The CoreTechnology : Artificial Neural Networks
- Self learning technology inspired by human
br...
© 2017 ALL RIGHTS RESERVED
The CoreTechnology – Artificial Neural Networks
1
230
5 1
2
04
1 8 3 5 6
4 0
INPUT SEQUENCE
OUT...
© 2017 ALL RIGHTS RESERVED
The CoreTechnology – Artificial Neural Networks
- Self learning technology inspired by human
br...
© 2017 ALL RIGHTS RESERVED
The Effectiveness of Recurrent Neural Networks (RNN)
The output of the DNN at a specific time t...
© 2017 ALL RIGHTS RESERVED 14
© 2017 ALL RIGHTS RESERVED 15
© 2017 ALL RIGHTS RESERVED
All-Purpose Recipe – Seq2Seq-attn
어떻게 지내요 ? <eos> How are you ?
How are you ? <eos>
+
Encoder
A...
© 2017 ALL RIGHTS RESERVED
Source sentence
Decoder
Word embeddings
Encoder
Generated target
Reference phrase
(validated tr...
© 2017 ALL RIGHTS RESERVED 18
© 2017 ALL RIGHTS RESERVED 19
© 2017 ALL RIGHTS RESERVED
FR-ENTraining – Neural MT effect
25/04/2017
20
Source SYSTRAN NMT Free Internet Portal
« Les ge...
© 2017 ALL RIGHTS RESERVED
NMT specialization: Quality Corpora are required!
For Neural MT, customization can be processed...
© 2017 ALL RIGHTS RESERVED
Evaluation of specialized PNMT vs HT
22
© 2017 ALL RIGHTS RESERVED
Neural MT – Lessons learned
23
SPEED
VOCABULARY SIZE
SENTENCE LENGTH
Technical limitations for ...
© 2017 ALL RIGHTS RESERVED
Translation at the core of the value chain
24
• Costs reduction
• Time to market
• Removing lan...
© 2017 ALL RIGHTS RESERVED
Integration in SYSTRAN Enterprise Server
SECURED HIGHLY AVAILABLE SCALABLE
User Tools Connector...
© 2017 ALL RIGHTS RESERVED
iTranslate: Integration as an embedded mobile application
•45 M downloads
•5 M translation
requ...
© 2017 ALL RIGHTS RESERVED
Ready2go!The first interpreter-guide as mobile app
27
Ready2go! Application was selected within...
© 2017 ALL RIGHTS RESERVED
MULTIMODAL CONVERGENCE
AN AUGMENTED HUMAN COMMUNICATION BEYOND LANGUAGE….
28
© 2017 ALL RIGHTS RESERVEDApril 2017 29
https://demo-pnmt.systran.net
Upcoming SlideShare
Loading in …5
×

II-SDV 2017: Applications of RNN (Recurrent Neural Networks) within Machine Translations Solutions and NLP Applications: What are the Changes for the User? What are the New Benefits?

929 views

Published on

Pierre Bernassau will present the state of the art in Artificial Intelligence and Recurrent Neural Networks applied to natural language and in particular in the Machine Translation domain. This disruptive technology does not shift previous practices based on rules and statistics, but it makes new fields possible. New fields for end-users that can apply MT technologies on new languages, text styles, documents, or messages with a ‘good-enough’ result. But also new fields in terms of good practices, where new projects, new workflow, new applications are addressed that expand de facto the MT market.

With the experience of several recent projects done by his consulting team, Pierre will explain the best practices to apply Neural Technologies within the NLP field.

Published in: Internet
  • Be the first to comment

  • Be the first to like this

II-SDV 2017: Applications of RNN (Recurrent Neural Networks) within Machine Translations Solutions and NLP Applications: What are the Changes for the User? What are the New Benefits?

  1. 1. © 2017 ALL RIGHTS RESERVED II – SDV Congress Benefits of RNN (Recurrent Neural Networks) within MachineTranslation and NLP Applications TAILORED AND SECURED MACHINE TRANSLATIONS Presenter: Pierre Bernassau Director, Client Services Date: April 24,2017 www.systrangroup.com
  2. 2. © 2017 ALL RIGHTS RESERVED SYSTRAN overview, in one slide! 2 Pioneer in machine translation, SYSTRAN enhances understanding and communication with secured end-to-end solutions Hybrid MT to deliver the best user experience Language combinations +140 200 +25% Research Ecosystem Leader in machine translation and natural language processing Employees Revenue invested in R&D Seoul Paris – R&D San Diego
  3. 3. © 2017 ALL RIGHTS RESERVED IA & Deep Learning : a convergence with MachineTranslation 3
  4. 4. © 2017 ALL RIGHTS RESERVED IA & Deep Learning enters in the business world 4 HAVING REVOLUTIONIZED SPEECH AND IMAGE RECOGNITION, AI & DEEP LEARNING NOW BRINGTHEIR CAPABILITIESTO SOLVE BUSINESS CHALLENGES
  5. 5. © 2017 ALL RIGHTS RESERVED The challenges to apply IA & Deep Learning to NLP applications 5 WHAT MAKES NATURAL LANGUAGE PROCESSINGTRICKYAND COMPLEX ?
  6. 6. © 2017 ALL RIGHTS RESERVED At SYSTRAN, we love languages, all languages 6 1.A system for the expression of thoughts, feelings, etc. by the use of spoken sounds or conventional symbols 2. Any other systematic or non systematic means of communicating, such as gesture or animal sounds 3. The faculty for the use of such systems, which is the distinguishing characteristic of man as compared with other animals 4.The language of a particular nation of people 5.The specialized vocabulary used by a particular group 6. A particular manner or style of verbal expression
  7. 7. © 2017 ALL RIGHTS RESERVED MachineTranslationTechnologies before AI & Deep Learning 7 ALL MT OFFERSWERE BASED ON 2 MAINTECHNOLOGIESWITH/WITHOUT HYBRIDAPPROACHES
  8. 8. © 2017 ALL RIGHTS RESERVED MachineTranslationTechnologies is entering in a new world ! 8 NEURAL MT PRODUCES ATRANSLATIONOVERACHIEVINGTHE CURRENT STATE OFTHE ART
  9. 9. © 2017 ALL RIGHTS RESERVED The CoreTechnology : Artificial Neural Networks 9 - Self learning technology inspired by human brain neuron network
  10. 10. © 2017 ALL RIGHTS RESERVED The CoreTechnology : Artificial Neural Networks - Self learning technology inspired by human brain neuron network - Composed of layers of artificial neurons - Layers are connected with different weights - Each artificial neuron is activated through simultaneous firing of connected neurons 5 581 9 2 3 16 5 1 8 3 5 6 6 1 INPUT SEQUENCE OUTPUT 10
  11. 11. © 2017 ALL RIGHTS RESERVED The CoreTechnology – Artificial Neural Networks 1 230 5 1 2 04 1 8 3 5 6 4 0 INPUT SEQUENCE OUTPUT 11 - Self learning technology inspired by human brain neuron network - Composed of layers of artificial neurons - Layers are connected with different weights - Each artificial neuron is activated through simultaneous firing of connected neurons
  12. 12. © 2017 ALL RIGHTS RESERVED The CoreTechnology – Artificial Neural Networks - Self learning technology inspired by human brain neuron network - Composed of layers of artificial neurons - Layers are connected with different weights - Each artificial neuron is activated through simultaneous firing of connected neurons - The generated output is compared to expected reference and corrective feedback sent backward to adjust weights and tune the network connections 4 0OUTPUT REFERENCE6 0 12
  13. 13. © 2017 ALL RIGHTS RESERVED The Effectiveness of Recurrent Neural Networks (RNN) The output of the DNN at a specific time t depends on the input of the timestep t but also on the state of the hidden layers of the timestep t-1. Recurrence in DNN allows contextual knowledge Effective to model language at different levels (local agreement, global consistency, fluency, etc…) Close to flawless and at least generally outperforms non native speaker 13
  14. 14. © 2017 ALL RIGHTS RESERVED 14
  15. 15. © 2017 ALL RIGHTS RESERVED 15
  16. 16. © 2017 ALL RIGHTS RESERVED All-Purpose Recipe – Seq2Seq-attn 어떻게 지내요 ? <eos> How are you ? How are you ? <eos> + Encoder Attention Decoder Generator 16 EMBED, ENCODE, ATTEND, PREDICT The translation process ends when the decoder “decides” to generate an end-of-sentence special word.
  17. 17. © 2017 ALL RIGHTS RESERVED Source sentence Decoder Word embeddings Encoder Generated target Reference phrase (validated translation) Attention Model 17
  18. 18. © 2017 ALL RIGHTS RESERVED 18
  19. 19. © 2017 ALL RIGHTS RESERVED 19
  20. 20. © 2017 ALL RIGHTS RESERVED FR-ENTraining – Neural MT effect 25/04/2017 20 Source SYSTRAN NMT Free Internet Portal « Les gens se ruent sur la nourriture comme des loups affamés et ensuite ils se retrouvent à exploiter du charbon en Sibérie » , - affirme-t-il. « People are going to eat food like hungry wolves and then they’re going to exploit coal in Siberia , » he says . « People are flocking to the food as hungry wolves and then they find themselves to operate coal in Siberia », - he says. Il y a encore un demi-siècle , on effectuait des expériences sur les plantes. A half-century ago , experiments were carried out on plants. Yet half a century ago it was conducting experiments on plants. Nos portes sont toujours ouvertes à ceux qui veulent ou auraient envie de nous rejoindre. Our doors are always open to those who want to join us. Our doors are always open to those who want or would want to join us. Par ailleurs , les législateurs républicains ont parrainé en 2011 des lois abolissant l'inscription des électeurs le jour du scrutin dans huit états. In 2011, republican legislators sponsored laws abolishing voter registration on election day in eight states. In addition , republican lawmakers sponsored in 2011 laws abolishing the registration of voters on election day in eight states.
  21. 21. © 2017 ALL RIGHTS RESERVED NMT specialization: Quality Corpora are required! For Neural MT, customization can be processed at three levels: before the training, during the training and after the training SYSTRAN propose a way to optimize neural networks in a post-training process, which we call “specialization.” Specialization consists of taking a generic model and adapting it to new data without fully retraining it. 21
  22. 22. © 2017 ALL RIGHTS RESERVED Evaluation of specialized PNMT vs HT 22
  23. 23. © 2017 ALL RIGHTS RESERVED Neural MT – Lessons learned 23 SPEED VOCABULARY SIZE SENTENCE LENGTH Technical limitations for training and accuracy decreases with length. Generally set to 80. Translation of a sentence involves a lot of numeric calculation and translation speed on GPU is about 8 sentences per second (RB: up to 40 sentences per second) Limited target vocabulary of about 50-100k (sub-word level) ABSENCE OF LINGUISTIC KNOWLEDGE Like SMT – NMT by default does not explicitly use linguistic knowledge but learn it by itself!
  24. 24. © 2017 ALL RIGHTS RESERVED Translation at the core of the value chain 24 • Costs reduction • Time to market • Removing language barriers • Multilingual user tools for better productivity • Domain terminology for better accuracy • Guaranteed data privacy for better security “Translation at heart improves your time to market, your lean management and your business process”
  25. 25. © 2017 ALL RIGHTS RESERVED Integration in SYSTRAN Enterprise Server SECURED HIGHLY AVAILABLE SCALABLE User Tools Connectors API YOUR TRANSLATION PROFILES Best of Breed Machine Translations Custom Resources • SYSTRAN dictionaries • Client dictionaries • Translation memories • Language models MT Engine 25
  26. 26. © 2017 ALL RIGHTS RESERVED iTranslate: Integration as an embedded mobile application •45 M downloads •5 M translation requests per day •Around 10 offline translation engines 26
  27. 27. © 2017 ALL RIGHTS RESERVED Ready2go!The first interpreter-guide as mobile app 27 Ready2go! Application was selected within 3rd Award Competition of Concours d’Innovation Numérique in France. It will be the first « inerpreter-guide » integrating all Petit Futé content and SYSTRAN translation tools: - Restaurant menus and signage translation tool - Useful popular sentences baesed on geolocalized context - Speech translation tools specialized with conversationalmodels. Development in progress: will be available as free mobile app for iPhone and Android
  28. 28. © 2017 ALL RIGHTS RESERVED MULTIMODAL CONVERGENCE AN AUGMENTED HUMAN COMMUNICATION BEYOND LANGUAGE…. 28
  29. 29. © 2017 ALL RIGHTS RESERVEDApril 2017 29 https://demo-pnmt.systran.net

×