16. 分布仮説 (Distributional Hypothesis)
l 同じ⽂文脈で出現する単語は同じ意味を持つとい
うこと
l データから単語の意味を学習する話は、少なか
らずこの仮説が元になっている
16
The Distributional Hypothesis is that words
that occur in the same contexts tend to have
similar meanings (Harris, 1954). (ACL wikiより)
97. 参考⽂文献
l [Evert10] Stefan Evert.
Distributional Semantic Models. NAACL 2010 Tutorial.
l [Mikolov+13a] Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey
Dean.
Efficient Estimation of Word Representations in Vector Space.
CoRR, 2013.
l [Morin+05] Frederic Morin, Yoshua Bengio.
Hierarchical Probabilistic Neural Network Language Model.
AISTATS, 2005.
l [Mikolov+13c] Tomas Mikolov, Ilya Sutskever, Kai Chen, Gregory
S. Corrado, Jeffrey Dean.
Distributed Representations of Words and Phrases and their
Compositionality. NIPS, 2013.
97
98. 参考⽂文献
l [Kim+13] Joo-Kyung Kim, Marie-Catherine de Marneffe.
Deriving adjectival scales from continuous space word
representations. EMNLP, 2013.
l [Mikolov+13d] Tomas Mikolov, Quoc V. Le, Ilya Sutskever.
Exploiting Similarities among Languages for Machine
Translation. CoRR, 2013.
l [Neelakantan+14] Arvind Neelakantan, Jeevan Shankar,
Alexandre Passos, Andrew McCallum.
Efficient Non-parametric Estimation of Multiple Embeddings
per Word in Vector Space. EMNLP, 2014.
l [Le+14] Quoc Le, Tomas Mikolov.
Distributed Representations of Sentences and Documents.
ICML, 2014.
98
99. 参考⽂文献
l [Hochreiter+97] Sepp Hochreiter, Jurgen Schmidhunber.
Long Short-Term Memory. Neural Computation 9(8), 1997.
l [Mikolov+10] Tomas Mikolov, Martin Karafiat, Lukas Burget, Jan
Honza Cernocky, Sanjeev Khudanpur.
Recurrent neural network based language model.
Interspeech, 2010.
l [Graves13] Alex Graves.
Generating Sequences With Recurrent Neural Networks. arXiv:
1308.0850, 2013.
l [Vinyal+15a] Oriol Vinyals, Alexander Toshev, Samy Bengio,
Dumitru Erhan.
Show and tell: A neural image caption generator. CVPR, 2015.
99
100. 参考⽂文献
l [Sutskever+14] Ilya Sutskever, Oriol Vinyals, Quoc V. Le.
Sequence to Sequence Learning with Neural Networks.
NIPS 2014.
l [Vinyals+15b] Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav
Petrov, Ilya Sutskever, Geoffrey Hinton.
Grammar as a foreign language.
ICLR 2015.
l [Vinyals+15c] Oriol Vinyals, Quoc Le.
A Neural Conversational Model. ICML 2015.
100
101. 参考⽂文献
l [Socher+11] Richard Socher, Cliff Lin, Andrew Y. Ng, Christopher D.
Manning.
Parsing Natural Scenes and Natural Language with Recursive Neural
Networks. ICML 2011
l [Socher+12] Richard Socher, Brody Huval, Christopher D. Manning,
Andrew Y. Ng.
Semantic Compositionality through Recursive Matrix-Vector Spaces.
EMNLP2012.
l [Socher+13] Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang,
Chris Manning, Andrew Ng, Chris Potts.
Recursive Deep Models for Semantic Compositionality Over a
Sentiment Treebank. EMNLP 2013.
l [Tai+15] Kai Sheng Tai, Richard Socher, Christopher D. Manning.
Improved Semantic Representations From Tree-Structured Long
Short-Term Memory Networks. ACL 2015.
101
102. 参考⽂文献
l [Bordes+11] A. Bordes, J. Weston, R. Collobert, Y. Bengio.
Learning structured embeddings of knowledge bases. AAAI2011.
l [Bordes+13] A. Bordes, N. Usunier, A. Garcia-Duran, J. Weston, O.
Yakhnenko.
Translating Embeddings for Modeling Multi-relational Data. NIPS
2013.
l [Fan+14] M. Fan, Q. Shou, E. Chang, T. F. Zheng.
Transition-based Knowledge Graph Embedding with Relational
Mapping Properties. PACLIC 2014.
l [Wang+14] Z. Wang, J. Zhang, J. Feng, Z. Chen.
Knowledge Graph Embedding by Translating on Hyperplanes. AAAI
2014.
l [Bordes&Weston14] A. Bordes, J. Weston.
Embedding Methods for Natural Language Processing. EMNLP2014
tutorial.
102
103. 参考⽂文献
l [Peng+15a] Baolin Peng, Kaisheng Yao.
Recurrent Neural Networks with External Memory for Language
Understanding. arXiv:1506.00195, 2015.
l [Weston+15] J. Weston, S. Chopra, A. Bordes.
Memory Networks. ICLR 2015.
l [Sukhbaatar+15] Sainbayar Sukhbaatar, Arthur Szlam, Jason Weston, Rob
Fergus.
End-To-End Memory Networks. arXiv:1503.08895, 2015.
l [Kumar+15] Ankit Kumar, Ozan Irsoy, Jonathan Su, James Bradbury, Robert
English, Brian Pierce, Peter Ondruska, Ishaan Gulrajani, Richard Socher.
Ask Me Anything: Dynamic Memory Networks for Natural Language
Processing. arXiv:1506.07285, 2015.
l [Peng+15b] Baolin Peng, Zhengdong Lu, Hang Li, Kam-Fai Wong.
Towards Neural Network-based Reasoning. arXiv:1508.05508, 2015.
l [Kiros+15] Ryan Kiros, Yukun Zhu, Ruslan Salakhutdinov, Richard S. Zemel,
Antonio Torralba, Raquel Urtasun, Sanja Fidler.
Skip-Thought Vectors. arXiv:1506.06726, 2015.
103