SlideShare a Scribd company logo
Bilingual Word Representations with
Monolingual Quality in Mind
Minh-Thang Luong, Hieu Pham, Christopher D. Manning
Proceedings of NAACL-HLT 2015 Workshop
AHC-Lab
M1 Hiroyuki Fudaba
1
What are Word Representations?
Vectors representing words
• One-hot word representations
• Distributed word representations [Bengio et al. 2003]
0, 0, 0, … , 0, 1, 0, 0, 0, … , 0
1.1, 0.5, −3.2, 0.5, … , 0.4
2
Distributed Word Representations
• Vectors representing words’ syntactic / semantic features
3
2 different languages in 1 vector space
4
Why do we need bilingual word representations?
• Crosslingual document classification
5
Apple Inc. Google
apple banana
companies
fruits
アップル株式会社
りんご
Which is more appropriate?
How to do 2-in-1
• Mapping
• Learning with Joint model
6
𝑦 = 𝑊𝑥
dog
cat
犬
猫
cat
猫
dog
犬
Problem of previous work
Perform poorly on monolingual tasks
Why?
tradeoff between bilingual tasks’ performance and monolinguals’
7
Paper’s approach
Substitute words to predict surroundings
8
Which one to substitute?
1. No alignment (BiSkip-MonoAlign)
2. Align before substitution (BiSkip-UnsupAlign)
I have a dog .
私は 犬を 飼って います .
9
Which one to substitute?
1. No alignment (BiSkip-MonoAlign)
2. Align before substitution (BiSkip-UnsupAlign)
I have a dog .
私は 犬を 飼って います .
10
Bilingual Skipgram Model
11
犬
is
my
,
Delicious
Try to predict
“is my , Delicious” from “犬”
Evaluation: word similarity
• Measures semantic quality of the word vectors monolingually
e.g.
tiger cat
computer keyboard internet
12
Evaluation: CLDC
Train with language A’s vector, and predict documents with language B
13
Document classifier
(perceptron)
Result
14
Conclusion and future work
What this paper say
• Substituting words make better bilingual word representations
Future work
• Pivoting to improve performance
15
references
• [Bengio et al. 2003] A Neural Probabilistic Language Model
• [Xiaochuan et al. 2011] Cross Lingual Text Classification by Mining
Multilingual Topics from Wikipedia
16

More Related Content

Similar to [Paper Introduction] Bilingual word representations with monolingual quality in mind

Testing speaking
Testing speakingTesting speaking
Testing speaking
M B
 
English for interpersonal interaction 1
English for interpersonal interaction 1English for interpersonal interaction 1
English for interpersonal interaction 1
Syifa Fadhilah
 
What to do with a language assistant
What to do with a language assistantWhat to do with a language assistant
What to do with a language assistant
Montse Irun
 
NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA Data Science Meetup 1/19/2017 - Presentation 2NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA DATASCIENCE
 
Task- Based Language Teaching
Task- Based Language TeachingTask- Based Language Teaching
Task- Based Language Teaching
BIZ University
 
Natural language processing (nlp)
Natural language processing (nlp)Natural language processing (nlp)
Natural language processing (nlp)
Kuppusamy P
 
Natural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptxNatural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptx
SHIBDASDUTTA
 
Chapter 7
Chapter 7Chapter 7
Chapter 7
Sungwoo Kim
 
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
Kristi Reyes
 
Oral Communication in a Context Q2 Week 1-1.pptx
Oral Communication in a Context Q2 Week 1-1.pptxOral Communication in a Context Q2 Week 1-1.pptx
Oral Communication in a Context Q2 Week 1-1.pptx
MAJOANHEMBRA
 
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMU
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMURebo, the Reflection Guidance Chatbot: Research lab meeting CMU
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMU
IrmtraudWolfbauer
 
Teaching and assessing speaking
Teaching and assessing speakingTeaching and assessing speaking
Teaching and assessing speaking
Evaggelia Charalambous
 
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Saurabh Kaushik
 
agung Power point listening
agung Power point listeningagung Power point listening
agung Power point listening
agungmuarief
 
ELT201 Week 9 teaching productive skills (speaking)
ELT201 Week 9 teaching productive skills (speaking)ELT201 Week 9 teaching productive skills (speaking)
ELT201 Week 9 teaching productive skills (speaking)
Dr. Russell Rodrigo
 
Speaking pedagogy workshop
Speaking pedagogy workshopSpeaking pedagogy workshop
Speaking pedagogy workshop
Wendy Tanagho
 
Effective strategies for project work
Effective strategies for project workEffective strategies for project work
Effective strategies for project work
Center for Global Education at Asia Society
 
Effective Strategies for Project Work in the Chinese Classroom
Effective Strategies for Project Work in the Chinese ClassroomEffective Strategies for Project Work in the Chinese Classroom
Effective Strategies for Project Work in the Chinese Classroom
Center for Global Education at Asia Society
 
Euro Exam
Euro Exam Euro Exam
Euro Exam
Trendy English
 
Strategic planning
Strategic planningStrategic planning
Strategic planning
Shona Whyte
 

Similar to [Paper Introduction] Bilingual word representations with monolingual quality in mind (20)

Testing speaking
Testing speakingTesting speaking
Testing speaking
 
English for interpersonal interaction 1
English for interpersonal interaction 1English for interpersonal interaction 1
English for interpersonal interaction 1
 
What to do with a language assistant
What to do with a language assistantWhat to do with a language assistant
What to do with a language assistant
 
NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA Data Science Meetup 1/19/2017 - Presentation 2NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA Data Science Meetup 1/19/2017 - Presentation 2
 
Task- Based Language Teaching
Task- Based Language TeachingTask- Based Language Teaching
Task- Based Language Teaching
 
Natural language processing (nlp)
Natural language processing (nlp)Natural language processing (nlp)
Natural language processing (nlp)
 
Natural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptxNatural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptx
 
Chapter 7
Chapter 7Chapter 7
Chapter 7
 
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
 
Oral Communication in a Context Q2 Week 1-1.pptx
Oral Communication in a Context Q2 Week 1-1.pptxOral Communication in a Context Q2 Week 1-1.pptx
Oral Communication in a Context Q2 Week 1-1.pptx
 
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMU
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMURebo, the Reflection Guidance Chatbot: Research lab meeting CMU
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMU
 
Teaching and assessing speaking
Teaching and assessing speakingTeaching and assessing speaking
Teaching and assessing speaking
 
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
 
agung Power point listening
agung Power point listeningagung Power point listening
agung Power point listening
 
ELT201 Week 9 teaching productive skills (speaking)
ELT201 Week 9 teaching productive skills (speaking)ELT201 Week 9 teaching productive skills (speaking)
ELT201 Week 9 teaching productive skills (speaking)
 
Speaking pedagogy workshop
Speaking pedagogy workshopSpeaking pedagogy workshop
Speaking pedagogy workshop
 
Effective strategies for project work
Effective strategies for project workEffective strategies for project work
Effective strategies for project work
 
Effective Strategies for Project Work in the Chinese Classroom
Effective Strategies for Project Work in the Chinese ClassroomEffective Strategies for Project Work in the Chinese Classroom
Effective Strategies for Project Work in the Chinese Classroom
 
Euro Exam
Euro Exam Euro Exam
Euro Exam
 
Strategic planning
Strategic planningStrategic planning
Strategic planning
 

More from NAIST Machine Translation Study Group

[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
NAIST Machine Translation Study Group
 
[Paper Introduction] Distant supervision for relation extraction without labe...
[Paper Introduction] Distant supervision for relation extraction without labe...[Paper Introduction] Distant supervision for relation extraction without labe...
[Paper Introduction] Distant supervision for relation extraction without labe...
NAIST Machine Translation Study Group
 
On using monolingual corpora in neural machine translation
On using monolingual corpora in neural machine translationOn using monolingual corpora in neural machine translation
On using monolingual corpora in neural machine translation
NAIST Machine Translation Study Group
 
RNN-based Translation Models (Japanese)
RNN-based Translation Models (Japanese)RNN-based Translation Models (Japanese)
RNN-based Translation Models (Japanese)
NAIST Machine Translation Study Group
 
[Paper Introduction] Efficient top down btg parsing for machine translation p...
[Paper Introduction] Efficient top down btg parsing for machine translation p...[Paper Introduction] Efficient top down btg parsing for machine translation p...
[Paper Introduction] Efficient top down btg parsing for machine translation p...
NAIST Machine Translation Study Group
 
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
NAIST Machine Translation Study Group
 
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
NAIST Machine Translation Study Group
 
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
NAIST Machine Translation Study Group
 
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
NAIST Machine Translation Study Group
 
[Book Reading] 機械翻訳 - Section 3 No.1
[Book Reading] 機械翻訳 - Section 3 No.1[Book Reading] 機械翻訳 - Section 3 No.1
[Book Reading] 機械翻訳 - Section 3 No.1
NAIST Machine Translation Study Group
 
[Paper Introduction] Training a Natural Language Generator From Unaligned Data
[Paper Introduction] Training a Natural Language Generator From Unaligned Data[Paper Introduction] Training a Natural Language Generator From Unaligned Data
[Paper Introduction] Training a Natural Language Generator From Unaligned Data
NAIST Machine Translation Study Group
 
[Book Reading] 機械翻訳 - Section 5 No.2
[Book Reading] 機械翻訳 - Section 5 No.2[Book Reading] 機械翻訳 - Section 5 No.2
[Book Reading] 機械翻訳 - Section 5 No.2
NAIST Machine Translation Study Group
 
[Book Reading] 機械翻訳 - Section 7 No.1
[Book Reading] 機械翻訳 - Section 7 No.1[Book Reading] 機械翻訳 - Section 7 No.1
[Book Reading] 機械翻訳 - Section 7 No.1
NAIST Machine Translation Study Group
 
[Book Reading] 機械翻訳 - Section 2 No.2
 [Book Reading] 機械翻訳 - Section 2 No.2 [Book Reading] 機械翻訳 - Section 2 No.2
[Book Reading] 機械翻訳 - Section 2 No.2
NAIST Machine Translation Study Group
 

More from NAIST Machine Translation Study Group (14)

[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
 
[Paper Introduction] Distant supervision for relation extraction without labe...
[Paper Introduction] Distant supervision for relation extraction without labe...[Paper Introduction] Distant supervision for relation extraction without labe...
[Paper Introduction] Distant supervision for relation extraction without labe...
 
On using monolingual corpora in neural machine translation
On using monolingual corpora in neural machine translationOn using monolingual corpora in neural machine translation
On using monolingual corpora in neural machine translation
 
RNN-based Translation Models (Japanese)
RNN-based Translation Models (Japanese)RNN-based Translation Models (Japanese)
RNN-based Translation Models (Japanese)
 
[Paper Introduction] Efficient top down btg parsing for machine translation p...
[Paper Introduction] Efficient top down btg parsing for machine translation p...[Paper Introduction] Efficient top down btg parsing for machine translation p...
[Paper Introduction] Efficient top down btg parsing for machine translation p...
 
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
 
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
 
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
 
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
 
[Book Reading] 機械翻訳 - Section 3 No.1
[Book Reading] 機械翻訳 - Section 3 No.1[Book Reading] 機械翻訳 - Section 3 No.1
[Book Reading] 機械翻訳 - Section 3 No.1
 
[Paper Introduction] Training a Natural Language Generator From Unaligned Data
[Paper Introduction] Training a Natural Language Generator From Unaligned Data[Paper Introduction] Training a Natural Language Generator From Unaligned Data
[Paper Introduction] Training a Natural Language Generator From Unaligned Data
 
[Book Reading] 機械翻訳 - Section 5 No.2
[Book Reading] 機械翻訳 - Section 5 No.2[Book Reading] 機械翻訳 - Section 5 No.2
[Book Reading] 機械翻訳 - Section 5 No.2
 
[Book Reading] 機械翻訳 - Section 7 No.1
[Book Reading] 機械翻訳 - Section 7 No.1[Book Reading] 機械翻訳 - Section 7 No.1
[Book Reading] 機械翻訳 - Section 7 No.1
 
[Book Reading] 機械翻訳 - Section 2 No.2
 [Book Reading] 機械翻訳 - Section 2 No.2 [Book Reading] 機械翻訳 - Section 2 No.2
[Book Reading] 機械翻訳 - Section 2 No.2
 

Recently uploaded

132/33KV substation case study Presentation
132/33KV substation case study Presentation132/33KV substation case study Presentation
132/33KV substation case study Presentation
kandramariana6
 
22CYT12-Unit-V-E Waste and its Management.ppt
22CYT12-Unit-V-E Waste and its Management.ppt22CYT12-Unit-V-E Waste and its Management.ppt
22CYT12-Unit-V-E Waste and its Management.ppt
KrishnaveniKrishnara1
 
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
IEEE Aerospace and Electronic Systems Society as a Graduate Student MemberIEEE Aerospace and Electronic Systems Society as a Graduate Student Member
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
VICTOR MAESTRE RAMIREZ
 
哪里办理(csu毕业证书)查尔斯特大学毕业证硕士学历原版一模一样
哪里办理(csu毕业证书)查尔斯特大学毕业证硕士学历原版一模一样哪里办理(csu毕业证书)查尔斯特大学毕业证硕士学历原版一模一样
哪里办理(csu毕业证书)查尔斯特大学毕业证硕士学历原版一模一样
insn4465
 
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
Gino153088
 
原版制作(Humboldt毕业证书)柏林大学毕业证学位证一模一样
原版制作(Humboldt毕业证书)柏林大学毕业证学位证一模一样原版制作(Humboldt毕业证书)柏林大学毕业证学位证一模一样
原版制作(Humboldt毕业证书)柏林大学毕业证学位证一模一样
ydzowc
 
Curve Fitting in Numerical Methods Regression
Curve Fitting in Numerical Methods RegressionCurve Fitting in Numerical Methods Regression
Curve Fitting in Numerical Methods Regression
Nada Hikmah
 
Transformers design and coooling methods
Transformers design and coooling methodsTransformers design and coooling methods
Transformers design and coooling methods
Roger Rozario
 
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Sinan KOZAK
 
BRAIN TUMOR DETECTION for seminar ppt.pdf
BRAIN TUMOR DETECTION for seminar ppt.pdfBRAIN TUMOR DETECTION for seminar ppt.pdf
BRAIN TUMOR DETECTION for seminar ppt.pdf
LAXMAREDDY22
 
Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...
IJECEIAES
 
LLM Fine Tuning with QLoRA Cassandra Lunch 4, presented by Anant
LLM Fine Tuning with QLoRA Cassandra Lunch 4, presented by AnantLLM Fine Tuning with QLoRA Cassandra Lunch 4, presented by Anant
LLM Fine Tuning with QLoRA Cassandra Lunch 4, presented by Anant
Anant Corporation
 
Hematology Analyzer Machine - Complete Blood Count
Hematology Analyzer Machine - Complete Blood CountHematology Analyzer Machine - Complete Blood Count
Hematology Analyzer Machine - Complete Blood Count
shahdabdulbaset
 
学校原版美国波士顿大学毕业证学历学位证书原版一模一样
学校原版美国波士顿大学毕业证学历学位证书原版一模一样学校原版美国波士顿大学毕业证学历学位证书原版一模一样
学校原版美国波士顿大学毕业证学历学位证书原版一模一样
171ticu
 
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.pptUnit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
KrishnaveniKrishnara1
 
Welding Metallurgy Ferrous Materials.pdf
Welding Metallurgy Ferrous Materials.pdfWelding Metallurgy Ferrous Materials.pdf
Welding Metallurgy Ferrous Materials.pdf
AjmalKhan50578
 
Software Engineering and Project Management - Introduction, Modeling Concepts...
Software Engineering and Project Management - Introduction, Modeling Concepts...Software Engineering and Project Management - Introduction, Modeling Concepts...
Software Engineering and Project Management - Introduction, Modeling Concepts...
Prakhyath Rai
 
Certificates - Mahmoud Mohamed Moursi Ahmed
Certificates - Mahmoud Mohamed Moursi AhmedCertificates - Mahmoud Mohamed Moursi Ahmed
Certificates - Mahmoud Mohamed Moursi Ahmed
Mahmoud Morsy
 
2008 BUILDING CONSTRUCTION Illustrated - Ching Chapter 02 The Building.pdf
2008 BUILDING CONSTRUCTION Illustrated - Ching Chapter 02 The Building.pdf2008 BUILDING CONSTRUCTION Illustrated - Ching Chapter 02 The Building.pdf
2008 BUILDING CONSTRUCTION Illustrated - Ching Chapter 02 The Building.pdf
Yasser Mahgoub
 
Mechanical Engineering on AAI Summer Training Report-003.pdf
Mechanical Engineering on AAI Summer Training Report-003.pdfMechanical Engineering on AAI Summer Training Report-003.pdf
Mechanical Engineering on AAI Summer Training Report-003.pdf
21UME003TUSHARDEB
 

Recently uploaded (20)

132/33KV substation case study Presentation
132/33KV substation case study Presentation132/33KV substation case study Presentation
132/33KV substation case study Presentation
 
22CYT12-Unit-V-E Waste and its Management.ppt
22CYT12-Unit-V-E Waste and its Management.ppt22CYT12-Unit-V-E Waste and its Management.ppt
22CYT12-Unit-V-E Waste and its Management.ppt
 
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
IEEE Aerospace and Electronic Systems Society as a Graduate Student MemberIEEE Aerospace and Electronic Systems Society as a Graduate Student Member
IEEE Aerospace and Electronic Systems Society as a Graduate Student Member
 
哪里办理(csu毕业证书)查尔斯特大学毕业证硕士学历原版一模一样
哪里办理(csu毕业证书)查尔斯特大学毕业证硕士学历原版一模一样哪里办理(csu毕业证书)查尔斯特大学毕业证硕士学历原版一模一样
哪里办理(csu毕业证书)查尔斯特大学毕业证硕士学历原版一模一样
 
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
4. Mosca vol I -Fisica-Tipler-5ta-Edicion-Vol-1.pdf
 
原版制作(Humboldt毕业证书)柏林大学毕业证学位证一模一样
原版制作(Humboldt毕业证书)柏林大学毕业证学位证一模一样原版制作(Humboldt毕业证书)柏林大学毕业证学位证一模一样
原版制作(Humboldt毕业证书)柏林大学毕业证学位证一模一样
 
Curve Fitting in Numerical Methods Regression
Curve Fitting in Numerical Methods RegressionCurve Fitting in Numerical Methods Regression
Curve Fitting in Numerical Methods Regression
 
Transformers design and coooling methods
Transformers design and coooling methodsTransformers design and coooling methods
Transformers design and coooling methods
 
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
Optimizing Gradle Builds - Gradle DPE Tour Berlin 2024
 
BRAIN TUMOR DETECTION for seminar ppt.pdf
BRAIN TUMOR DETECTION for seminar ppt.pdfBRAIN TUMOR DETECTION for seminar ppt.pdf
BRAIN TUMOR DETECTION for seminar ppt.pdf
 
Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...Advanced control scheme of doubly fed induction generator for wind turbine us...
Advanced control scheme of doubly fed induction generator for wind turbine us...
 
LLM Fine Tuning with QLoRA Cassandra Lunch 4, presented by Anant
LLM Fine Tuning with QLoRA Cassandra Lunch 4, presented by AnantLLM Fine Tuning with QLoRA Cassandra Lunch 4, presented by Anant
LLM Fine Tuning with QLoRA Cassandra Lunch 4, presented by Anant
 
Hematology Analyzer Machine - Complete Blood Count
Hematology Analyzer Machine - Complete Blood CountHematology Analyzer Machine - Complete Blood Count
Hematology Analyzer Machine - Complete Blood Count
 
学校原版美国波士顿大学毕业证学历学位证书原版一模一样
学校原版美国波士顿大学毕业证学历学位证书原版一模一样学校原版美国波士顿大学毕业证学历学位证书原版一模一样
学校原版美国波士顿大学毕业证学历学位证书原版一模一样
 
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.pptUnit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
Unit-III-ELECTROCHEMICAL STORAGE DEVICES.ppt
 
Welding Metallurgy Ferrous Materials.pdf
Welding Metallurgy Ferrous Materials.pdfWelding Metallurgy Ferrous Materials.pdf
Welding Metallurgy Ferrous Materials.pdf
 
Software Engineering and Project Management - Introduction, Modeling Concepts...
Software Engineering and Project Management - Introduction, Modeling Concepts...Software Engineering and Project Management - Introduction, Modeling Concepts...
Software Engineering and Project Management - Introduction, Modeling Concepts...
 
Certificates - Mahmoud Mohamed Moursi Ahmed
Certificates - Mahmoud Mohamed Moursi AhmedCertificates - Mahmoud Mohamed Moursi Ahmed
Certificates - Mahmoud Mohamed Moursi Ahmed
 
2008 BUILDING CONSTRUCTION Illustrated - Ching Chapter 02 The Building.pdf
2008 BUILDING CONSTRUCTION Illustrated - Ching Chapter 02 The Building.pdf2008 BUILDING CONSTRUCTION Illustrated - Ching Chapter 02 The Building.pdf
2008 BUILDING CONSTRUCTION Illustrated - Ching Chapter 02 The Building.pdf
 
Mechanical Engineering on AAI Summer Training Report-003.pdf
Mechanical Engineering on AAI Summer Training Report-003.pdfMechanical Engineering on AAI Summer Training Report-003.pdf
Mechanical Engineering on AAI Summer Training Report-003.pdf
 

[Paper Introduction] Bilingual word representations with monolingual quality in mind

  • 1. Bilingual Word Representations with Monolingual Quality in Mind Minh-Thang Luong, Hieu Pham, Christopher D. Manning Proceedings of NAACL-HLT 2015 Workshop AHC-Lab M1 Hiroyuki Fudaba 1
  • 2. What are Word Representations? Vectors representing words • One-hot word representations • Distributed word representations [Bengio et al. 2003] 0, 0, 0, … , 0, 1, 0, 0, 0, … , 0 1.1, 0.5, −3.2, 0.5, … , 0.4 2
  • 3. Distributed Word Representations • Vectors representing words’ syntactic / semantic features 3
  • 4. 2 different languages in 1 vector space 4
  • 5. Why do we need bilingual word representations? • Crosslingual document classification 5 Apple Inc. Google apple banana companies fruits アップル株式会社 りんご Which is more appropriate?
  • 6. How to do 2-in-1 • Mapping • Learning with Joint model 6 𝑦 = 𝑊𝑥 dog cat 犬 猫 cat 猫 dog 犬
  • 7. Problem of previous work Perform poorly on monolingual tasks Why? tradeoff between bilingual tasks’ performance and monolinguals’ 7
  • 8. Paper’s approach Substitute words to predict surroundings 8
  • 9. Which one to substitute? 1. No alignment (BiSkip-MonoAlign) 2. Align before substitution (BiSkip-UnsupAlign) I have a dog . 私は 犬を 飼って います . 9
  • 10. Which one to substitute? 1. No alignment (BiSkip-MonoAlign) 2. Align before substitution (BiSkip-UnsupAlign) I have a dog . 私は 犬を 飼って います . 10
  • 11. Bilingual Skipgram Model 11 犬 is my , Delicious Try to predict “is my , Delicious” from “犬”
  • 12. Evaluation: word similarity • Measures semantic quality of the word vectors monolingually e.g. tiger cat computer keyboard internet 12
  • 13. Evaluation: CLDC Train with language A’s vector, and predict documents with language B 13 Document classifier (perceptron)
  • 15. Conclusion and future work What this paper say • Substituting words make better bilingual word representations Future work • Pivoting to improve performance 15
  • 16. references • [Bengio et al. 2003] A Neural Probabilistic Language Model • [Xiaochuan et al. 2011] Cross Lingual Text Classification by Mining Multilingual Topics from Wikipedia 16