SlideShare a Scribd company logo
Bilingual Word Representations with
Monolingual Quality in Mind
Minh-Thang Luong, Hieu Pham, Christopher D. Manning
Proceedings of NAACL-HLT 2015 Workshop
AHC-Lab
M1 Hiroyuki Fudaba
1
What are Word Representations?
Vectors representing words
• One-hot word representations
• Distributed word representations [Bengio et al. 2003]
0, 0, 0, … , 0, 1, 0, 0, 0, … , 0
1.1, 0.5, −3.2, 0.5, … , 0.4
2
Distributed Word Representations
• Vectors representing words’ syntactic / semantic features
3
2 different languages in 1 vector space
4
Why do we need bilingual word representations?
• Crosslingual document classification
5
Apple Inc. Google
apple banana
companies
fruits
アップル株式会社
りんご
Which is more appropriate?
How to do 2-in-1
• Mapping
• Learning with Joint model
6
𝑦 = 𝑊𝑥
dog
cat
犬
猫
cat
猫
dog
犬
Problem of previous work
Perform poorly on monolingual tasks
Why?
tradeoff between bilingual tasks’ performance and monolinguals’
7
Paper’s approach
Substitute words to predict surroundings
8
Which one to substitute?
1. No alignment (BiSkip-MonoAlign)
2. Align before substitution (BiSkip-UnsupAlign)
I have a dog .
私は 犬を 飼って います .
9
Which one to substitute?
1. No alignment (BiSkip-MonoAlign)
2. Align before substitution (BiSkip-UnsupAlign)
I have a dog .
私は 犬を 飼って います .
10
Bilingual Skipgram Model
11
犬
is
my
,
Delicious
Try to predict
“is my , Delicious” from “犬”
Evaluation: word similarity
• Measures semantic quality of the word vectors monolingually
e.g.
tiger cat
computer keyboard internet
12
Evaluation: CLDC
Train with language A’s vector, and predict documents with language B
13
Document classifier
(perceptron)
Result
14
Conclusion and future work
What this paper say
• Substituting words make better bilingual word representations
Future work
• Pivoting to improve performance
15
references
• [Bengio et al. 2003] A Neural Probabilistic Language Model
• [Xiaochuan et al. 2011] Cross Lingual Text Classification by Mining
Multilingual Topics from Wikipedia
16

More Related Content

Similar to [Paper Introduction] Bilingual word representations with monolingual quality in mind

Testing speaking
Testing speakingTesting speaking
Testing speaking
M B
 
English for interpersonal interaction 1
English for interpersonal interaction 1English for interpersonal interaction 1
English for interpersonal interaction 1
Syifa Fadhilah
 
What to do with a language assistant
What to do with a language assistantWhat to do with a language assistant
What to do with a language assistant
Montse Irun
 
NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA Data Science Meetup 1/19/2017 - Presentation 2NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA DATASCIENCE
 
Task- Based Language Teaching
Task- Based Language TeachingTask- Based Language Teaching
Task- Based Language Teaching
BIZ University
 
Natural language processing (nlp)
Natural language processing (nlp)Natural language processing (nlp)
Natural language processing (nlp)
Kuppusamy P
 
Natural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptxNatural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptx
SHIBDASDUTTA
 
Chapter 7
Chapter 7Chapter 7
Chapter 7
Sungwoo Kim
 
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
Kristi Reyes
 
Oral Communication in a Context Q2 Week 1-1.pptx
Oral Communication in a Context Q2 Week 1-1.pptxOral Communication in a Context Q2 Week 1-1.pptx
Oral Communication in a Context Q2 Week 1-1.pptx
MAJOANHEMBRA
 
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMU
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMURebo, the Reflection Guidance Chatbot: Research lab meeting CMU
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMU
IrmtraudWolfbauer
 
Teaching and assessing speaking
Teaching and assessing speakingTeaching and assessing speaking
Teaching and assessing speaking
Evaggelia Charalambous
 
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Saurabh Kaushik
 
agung Power point listening
agung Power point listeningagung Power point listening
agung Power point listening
agungmuarief
 
ELT201 Week 9 teaching productive skills (speaking)
ELT201 Week 9 teaching productive skills (speaking)ELT201 Week 9 teaching productive skills (speaking)
ELT201 Week 9 teaching productive skills (speaking)
Dr. Russell Rodrigo
 
Speaking pedagogy workshop
Speaking pedagogy workshopSpeaking pedagogy workshop
Speaking pedagogy workshop
Wendy Tanagho
 
Effective strategies for project work
Effective strategies for project workEffective strategies for project work
Effective strategies for project work
Center for Global Education at Asia Society
 
Effective Strategies for Project Work in the Chinese Classroom
Effective Strategies for Project Work in the Chinese ClassroomEffective Strategies for Project Work in the Chinese Classroom
Effective Strategies for Project Work in the Chinese Classroom
Center for Global Education at Asia Society
 
Euro Exam
Euro Exam Euro Exam
Euro Exam
Trendy English
 
Strategic planning
Strategic planningStrategic planning
Strategic planning
Shona Whyte
 

Similar to [Paper Introduction] Bilingual word representations with monolingual quality in mind (20)

Testing speaking
Testing speakingTesting speaking
Testing speaking
 
English for interpersonal interaction 1
English for interpersonal interaction 1English for interpersonal interaction 1
English for interpersonal interaction 1
 
What to do with a language assistant
What to do with a language assistantWhat to do with a language assistant
What to do with a language assistant
 
NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA Data Science Meetup 1/19/2017 - Presentation 2NOVA Data Science Meetup 1/19/2017 - Presentation 2
NOVA Data Science Meetup 1/19/2017 - Presentation 2
 
Task- Based Language Teaching
Task- Based Language TeachingTask- Based Language Teaching
Task- Based Language Teaching
 
Natural language processing (nlp)
Natural language processing (nlp)Natural language processing (nlp)
Natural language processing (nlp)
 
Natural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptxNatural Language Processing (NLP).pptx
Natural Language Processing (NLP).pptx
 
Chapter 7
Chapter 7Chapter 7
Chapter 7
 
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
Power Up Vocab Teaching - San Diego Regional CATESOL Conference 2015
 
Oral Communication in a Context Q2 Week 1-1.pptx
Oral Communication in a Context Q2 Week 1-1.pptxOral Communication in a Context Q2 Week 1-1.pptx
Oral Communication in a Context Q2 Week 1-1.pptx
 
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMU
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMURebo, the Reflection Guidance Chatbot: Research lab meeting CMU
Rebo, the Reflection Guidance Chatbot: Research lab meeting CMU
 
Teaching and assessing speaking
Teaching and assessing speakingTeaching and assessing speaking
Teaching and assessing speaking
 
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1Engineering Intelligent NLP Applications Using Deep Learning – Part 1
Engineering Intelligent NLP Applications Using Deep Learning – Part 1
 
agung Power point listening
agung Power point listeningagung Power point listening
agung Power point listening
 
ELT201 Week 9 teaching productive skills (speaking)
ELT201 Week 9 teaching productive skills (speaking)ELT201 Week 9 teaching productive skills (speaking)
ELT201 Week 9 teaching productive skills (speaking)
 
Speaking pedagogy workshop
Speaking pedagogy workshopSpeaking pedagogy workshop
Speaking pedagogy workshop
 
Effective strategies for project work
Effective strategies for project workEffective strategies for project work
Effective strategies for project work
 
Effective Strategies for Project Work in the Chinese Classroom
Effective Strategies for Project Work in the Chinese ClassroomEffective Strategies for Project Work in the Chinese Classroom
Effective Strategies for Project Work in the Chinese Classroom
 
Euro Exam
Euro Exam Euro Exam
Euro Exam
 
Strategic planning
Strategic planningStrategic planning
Strategic planning
 

More from NAIST Machine Translation Study Group

[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
NAIST Machine Translation Study Group
 
[Paper Introduction] Distant supervision for relation extraction without labe...
[Paper Introduction] Distant supervision for relation extraction without labe...[Paper Introduction] Distant supervision for relation extraction without labe...
[Paper Introduction] Distant supervision for relation extraction without labe...
NAIST Machine Translation Study Group
 
On using monolingual corpora in neural machine translation
On using monolingual corpora in neural machine translationOn using monolingual corpora in neural machine translation
On using monolingual corpora in neural machine translation
NAIST Machine Translation Study Group
 
RNN-based Translation Models (Japanese)
RNN-based Translation Models (Japanese)RNN-based Translation Models (Japanese)
RNN-based Translation Models (Japanese)
NAIST Machine Translation Study Group
 
[Paper Introduction] Efficient top down btg parsing for machine translation p...
[Paper Introduction] Efficient top down btg parsing for machine translation p...[Paper Introduction] Efficient top down btg parsing for machine translation p...
[Paper Introduction] Efficient top down btg parsing for machine translation p...
NAIST Machine Translation Study Group
 
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
NAIST Machine Translation Study Group
 
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
NAIST Machine Translation Study Group
 
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
NAIST Machine Translation Study Group
 
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
NAIST Machine Translation Study Group
 
[Book Reading] 機械翻訳 - Section 3 No.1
[Book Reading] 機械翻訳 - Section 3 No.1[Book Reading] 機械翻訳 - Section 3 No.1
[Book Reading] 機械翻訳 - Section 3 No.1
NAIST Machine Translation Study Group
 
[Paper Introduction] Training a Natural Language Generator From Unaligned Data
[Paper Introduction] Training a Natural Language Generator From Unaligned Data[Paper Introduction] Training a Natural Language Generator From Unaligned Data
[Paper Introduction] Training a Natural Language Generator From Unaligned Data
NAIST Machine Translation Study Group
 
[Book Reading] 機械翻訳 - Section 5 No.2
[Book Reading] 機械翻訳 - Section 5 No.2[Book Reading] 機械翻訳 - Section 5 No.2
[Book Reading] 機械翻訳 - Section 5 No.2
NAIST Machine Translation Study Group
 
[Book Reading] 機械翻訳 - Section 7 No.1
[Book Reading] 機械翻訳 - Section 7 No.1[Book Reading] 機械翻訳 - Section 7 No.1
[Book Reading] 機械翻訳 - Section 7 No.1
NAIST Machine Translation Study Group
 
[Book Reading] 機械翻訳 - Section 2 No.2
 [Book Reading] 機械翻訳 - Section 2 No.2 [Book Reading] 機械翻訳 - Section 2 No.2
[Book Reading] 機械翻訳 - Section 2 No.2
NAIST Machine Translation Study Group
 

More from NAIST Machine Translation Study Group (14)

[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
[Paper Introduction] Efficient Lattice Rescoring Using Recurrent Neural Netwo...
 
[Paper Introduction] Distant supervision for relation extraction without labe...
[Paper Introduction] Distant supervision for relation extraction without labe...[Paper Introduction] Distant supervision for relation extraction without labe...
[Paper Introduction] Distant supervision for relation extraction without labe...
 
On using monolingual corpora in neural machine translation
On using monolingual corpora in neural machine translationOn using monolingual corpora in neural machine translation
On using monolingual corpora in neural machine translation
 
RNN-based Translation Models (Japanese)
RNN-based Translation Models (Japanese)RNN-based Translation Models (Japanese)
RNN-based Translation Models (Japanese)
 
[Paper Introduction] Efficient top down btg parsing for machine translation p...
[Paper Introduction] Efficient top down btg parsing for machine translation p...[Paper Introduction] Efficient top down btg parsing for machine translation p...
[Paper Introduction] Efficient top down btg parsing for machine translation p...
 
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
[Paper Introduction] Translating into Morphologically Rich Languages with Syn...
 
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
[Paper Introduction] Supervised Phrase Table Triangulation with Neural Word E...
 
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
[Paper Introduction] Evaluating MT Systems with Second Language Proficiency T...
 
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
[Paper Introduction] A Context-Aware Topic Model for Statistical Machine Tran...
 
[Book Reading] 機械翻訳 - Section 3 No.1
[Book Reading] 機械翻訳 - Section 3 No.1[Book Reading] 機械翻訳 - Section 3 No.1
[Book Reading] 機械翻訳 - Section 3 No.1
 
[Paper Introduction] Training a Natural Language Generator From Unaligned Data
[Paper Introduction] Training a Natural Language Generator From Unaligned Data[Paper Introduction] Training a Natural Language Generator From Unaligned Data
[Paper Introduction] Training a Natural Language Generator From Unaligned Data
 
[Book Reading] 機械翻訳 - Section 5 No.2
[Book Reading] 機械翻訳 - Section 5 No.2[Book Reading] 機械翻訳 - Section 5 No.2
[Book Reading] 機械翻訳 - Section 5 No.2
 
[Book Reading] 機械翻訳 - Section 7 No.1
[Book Reading] 機械翻訳 - Section 7 No.1[Book Reading] 機械翻訳 - Section 7 No.1
[Book Reading] 機械翻訳 - Section 7 No.1
 
[Book Reading] 機械翻訳 - Section 2 No.2
 [Book Reading] 機械翻訳 - Section 2 No.2 [Book Reading] 機械翻訳 - Section 2 No.2
[Book Reading] 機械翻訳 - Section 2 No.2
 

Recently uploaded

5G Radio Network Througput Problem Analysis HCIA.pdf
5G Radio Network Througput Problem Analysis HCIA.pdf5G Radio Network Througput Problem Analysis HCIA.pdf
5G Radio Network Througput Problem Analysis HCIA.pdf
AlvianRamadhani5
 
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
upoux
 
P5 Working Drawings.pdf floor plan, civil
P5 Working Drawings.pdf floor plan, civilP5 Working Drawings.pdf floor plan, civil
P5 Working Drawings.pdf floor plan, civil
AnasAhmadNoor
 
This study Examines the Effectiveness of Talent Procurement through the Imple...
This study Examines the Effectiveness of Talent Procurement through the Imple...This study Examines the Effectiveness of Talent Procurement through the Imple...
This study Examines the Effectiveness of Talent Procurement through the Imple...
DharmaBanothu
 
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Transcat
 
SCALING OF MOS CIRCUITS m .pptx
SCALING OF MOS CIRCUITS m                 .pptxSCALING OF MOS CIRCUITS m                 .pptx
SCALING OF MOS CIRCUITS m .pptx
harshapolam10
 
Open Channel Flow: fluid flow with a free surface
Open Channel Flow: fluid flow with a free surfaceOpen Channel Flow: fluid flow with a free surface
Open Channel Flow: fluid flow with a free surface
Indrajeet sahu
 
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICSUNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
vmspraneeth
 
DESIGN AND MANUFACTURE OF CEILING BOARD USING SAWDUST AND WASTE CARTON MATERI...
DESIGN AND MANUFACTURE OF CEILING BOARD USING SAWDUST AND WASTE CARTON MATERI...DESIGN AND MANUFACTURE OF CEILING BOARD USING SAWDUST AND WASTE CARTON MATERI...
DESIGN AND MANUFACTURE OF CEILING BOARD USING SAWDUST AND WASTE CARTON MATERI...
OKORIE1
 
ITSM Integration with MuleSoft.pptx
ITSM  Integration with MuleSoft.pptxITSM  Integration with MuleSoft.pptx
ITSM Integration with MuleSoft.pptx
VANDANAMOHANGOUDA
 
An Introduction to the Compiler Designss
An Introduction to the Compiler DesignssAn Introduction to the Compiler Designss
An Introduction to the Compiler Designss
ElakkiaU
 
Introduction to Computer Networks & OSI MODEL.ppt
Introduction to Computer Networks & OSI MODEL.pptIntroduction to Computer Networks & OSI MODEL.ppt
Introduction to Computer Networks & OSI MODEL.ppt
Dwarkadas J Sanghvi College of Engineering
 
Impartiality as per ISO /IEC 17025:2017 Standard
Impartiality as per ISO /IEC 17025:2017 StandardImpartiality as per ISO /IEC 17025:2017 Standard
Impartiality as per ISO /IEC 17025:2017 Standard
MuhammadJazib15
 
Determination of Equivalent Circuit parameters and performance characteristic...
Determination of Equivalent Circuit parameters and performance characteristic...Determination of Equivalent Circuit parameters and performance characteristic...
Determination of Equivalent Circuit parameters and performance characteristic...
pvpriya2
 
Null Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAMNull Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAM
Divyanshu
 
Transformers design and coooling methods
Transformers design and coooling methodsTransformers design and coooling methods
Transformers design and coooling methods
Roger Rozario
 
openshift technical overview - Flow of openshift containerisatoin
openshift technical overview - Flow of openshift containerisatoinopenshift technical overview - Flow of openshift containerisatoin
openshift technical overview - Flow of openshift containerisatoin
snaprevwdev
 
Accident detection system project report.pdf
Accident detection system project report.pdfAccident detection system project report.pdf
Accident detection system project report.pdf
Kamal Acharya
 
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
nedcocy
 
Blood finder application project report (1).pdf
Blood finder application project report (1).pdfBlood finder application project report (1).pdf
Blood finder application project report (1).pdf
Kamal Acharya
 

Recently uploaded (20)

5G Radio Network Througput Problem Analysis HCIA.pdf
5G Radio Network Througput Problem Analysis HCIA.pdf5G Radio Network Througput Problem Analysis HCIA.pdf
5G Radio Network Througput Problem Analysis HCIA.pdf
 
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
一比一原版(osu毕业证书)美国俄勒冈州立大学毕业证如何办理
 
P5 Working Drawings.pdf floor plan, civil
P5 Working Drawings.pdf floor plan, civilP5 Working Drawings.pdf floor plan, civil
P5 Working Drawings.pdf floor plan, civil
 
This study Examines the Effectiveness of Talent Procurement through the Imple...
This study Examines the Effectiveness of Talent Procurement through the Imple...This study Examines the Effectiveness of Talent Procurement through the Imple...
This study Examines the Effectiveness of Talent Procurement through the Imple...
 
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
Tools & Techniques for Commissioning and Maintaining PV Systems W-Animations ...
 
SCALING OF MOS CIRCUITS m .pptx
SCALING OF MOS CIRCUITS m                 .pptxSCALING OF MOS CIRCUITS m                 .pptx
SCALING OF MOS CIRCUITS m .pptx
 
Open Channel Flow: fluid flow with a free surface
Open Channel Flow: fluid flow with a free surfaceOpen Channel Flow: fluid flow with a free surface
Open Channel Flow: fluid flow with a free surface
 
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICSUNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
UNIT 4 LINEAR INTEGRATED CIRCUITS-DIGITAL ICS
 
DESIGN AND MANUFACTURE OF CEILING BOARD USING SAWDUST AND WASTE CARTON MATERI...
DESIGN AND MANUFACTURE OF CEILING BOARD USING SAWDUST AND WASTE CARTON MATERI...DESIGN AND MANUFACTURE OF CEILING BOARD USING SAWDUST AND WASTE CARTON MATERI...
DESIGN AND MANUFACTURE OF CEILING BOARD USING SAWDUST AND WASTE CARTON MATERI...
 
ITSM Integration with MuleSoft.pptx
ITSM  Integration with MuleSoft.pptxITSM  Integration with MuleSoft.pptx
ITSM Integration with MuleSoft.pptx
 
An Introduction to the Compiler Designss
An Introduction to the Compiler DesignssAn Introduction to the Compiler Designss
An Introduction to the Compiler Designss
 
Introduction to Computer Networks & OSI MODEL.ppt
Introduction to Computer Networks & OSI MODEL.pptIntroduction to Computer Networks & OSI MODEL.ppt
Introduction to Computer Networks & OSI MODEL.ppt
 
Impartiality as per ISO /IEC 17025:2017 Standard
Impartiality as per ISO /IEC 17025:2017 StandardImpartiality as per ISO /IEC 17025:2017 Standard
Impartiality as per ISO /IEC 17025:2017 Standard
 
Determination of Equivalent Circuit parameters and performance characteristic...
Determination of Equivalent Circuit parameters and performance characteristic...Determination of Equivalent Circuit parameters and performance characteristic...
Determination of Equivalent Circuit parameters and performance characteristic...
 
Null Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAMNull Bangalore | Pentesters Approach to AWS IAM
Null Bangalore | Pentesters Approach to AWS IAM
 
Transformers design and coooling methods
Transformers design and coooling methodsTransformers design and coooling methods
Transformers design and coooling methods
 
openshift technical overview - Flow of openshift containerisatoin
openshift technical overview - Flow of openshift containerisatoinopenshift technical overview - Flow of openshift containerisatoin
openshift technical overview - Flow of openshift containerisatoin
 
Accident detection system project report.pdf
Accident detection system project report.pdfAccident detection system project report.pdf
Accident detection system project report.pdf
 
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
一比一原版(爱大毕业证书)爱荷华大学毕业证如何办理
 
Blood finder application project report (1).pdf
Blood finder application project report (1).pdfBlood finder application project report (1).pdf
Blood finder application project report (1).pdf
 

[Paper Introduction] Bilingual word representations with monolingual quality in mind

  • 1. Bilingual Word Representations with Monolingual Quality in Mind Minh-Thang Luong, Hieu Pham, Christopher D. Manning Proceedings of NAACL-HLT 2015 Workshop AHC-Lab M1 Hiroyuki Fudaba 1
  • 2. What are Word Representations? Vectors representing words • One-hot word representations • Distributed word representations [Bengio et al. 2003] 0, 0, 0, … , 0, 1, 0, 0, 0, … , 0 1.1, 0.5, −3.2, 0.5, … , 0.4 2
  • 3. Distributed Word Representations • Vectors representing words’ syntactic / semantic features 3
  • 4. 2 different languages in 1 vector space 4
  • 5. Why do we need bilingual word representations? • Crosslingual document classification 5 Apple Inc. Google apple banana companies fruits アップル株式会社 りんご Which is more appropriate?
  • 6. How to do 2-in-1 • Mapping • Learning with Joint model 6 𝑦 = 𝑊𝑥 dog cat 犬 猫 cat 猫 dog 犬
  • 7. Problem of previous work Perform poorly on monolingual tasks Why? tradeoff between bilingual tasks’ performance and monolinguals’ 7
  • 8. Paper’s approach Substitute words to predict surroundings 8
  • 9. Which one to substitute? 1. No alignment (BiSkip-MonoAlign) 2. Align before substitution (BiSkip-UnsupAlign) I have a dog . 私は 犬を 飼って います . 9
  • 10. Which one to substitute? 1. No alignment (BiSkip-MonoAlign) 2. Align before substitution (BiSkip-UnsupAlign) I have a dog . 私は 犬を 飼って います . 10
  • 11. Bilingual Skipgram Model 11 犬 is my , Delicious Try to predict “is my , Delicious” from “犬”
  • 12. Evaluation: word similarity • Measures semantic quality of the word vectors monolingually e.g. tiger cat computer keyboard internet 12
  • 13. Evaluation: CLDC Train with language A’s vector, and predict documents with language B 13 Document classifier (perceptron)
  • 15. Conclusion and future work What this paper say • Substituting words make better bilingual word representations Future work • Pivoting to improve performance 15
  • 16. references • [Bengio et al. 2003] A Neural Probabilistic Language Model • [Xiaochuan et al. 2011] Cross Lingual Text Classification by Mining Multilingual Topics from Wikipedia 16