Pycon2016- Applying Deep Learning in Information Retrieval System
•
10 likes•2,794 views
This document summarizes how deep learning can be applied in information retrieval systems. It discusses using neural networks like convolutional neural networks (CNNs) to learn feature representations of queries and documents. CNNs and other neural networks can be used as trainable feature extractors to encode sentences and images into fixed-length vectors. These vectors can then be compared using similarity functions to retrieve relevant results from a database. Specific examples discussed include using CNNs and recurrent neural networks to classify and retrieve sentences, and training CNNs on large datasets to learn image feature vectors for image retrieval.
Recommended
Recommended
More Related Content
Viewers also liked
Viewers also liked (11)
Similar to Pycon2016- Applying Deep Learning in Information Retrieval System
Similar to Pycon2016- Applying Deep Learning in Information Retrieval System (16)
Recently uploaded
Recently uploaded (20)
Pycon2016- Applying Deep Learning in Information Retrieval System
- 1. Applying Deep Learning in Information Retrieval System
- 3. My project http://twchrislin.com/project/ Tools: python numpy pandas scikit-learn xgboost theano lasagne id LifeTime dira FlightDistance FlightDistanceError … signal 18453471 0.00157832377590239050.999999344348907614.0333347320556640.6814009547233582 … 0 5364094 0.00098804314620792870.99970477819442765.536156654357910.3023407459259033 … 0 11130990 0.00087667378829792140.99998414516448966.1173019409179690.27646270394325256… 0 15173787 0.0008541746647097170.99990290403366085.2280673980712890.22073912620544436… 0 1102544 0.00112933525815606140.999995291233062939.069534301757811.8981972932815552 … 0 https://youtu.be/_SjgCoR7e1s
- 5. Outline 1. Information Retrieval 2. Neural Networks and Convolutional Neural Networks 3. Feature Vector Representation 4. Sentence Retrieval 5. Image Retrieval 6. Conclusion
- 7. Neural Networks http://speech.ee.ntu.edu.tw/~tlkagk/courses/MLDS_2015_2/Lecture/DNN%20(v4).pdf Single Neuron, Logistic Regression Sigmoid funciton 28 pixels 28 pixels 784 pixels
- 9. Neural Networks http://speech.ee.ntu.edu.tw/~tlkagk/courses/MLDS_2015_2/Lecture/DNN%20(v4).pdf MLP Multiclass Z1xL1=X1x785 W785xL1 + f(Z1xL1) Sigmoid funciton Z1xL2=X1xL1 WL1xL2 + f(Z1xL2)
- 10. Convolutional Neural Networks http://www.slideshare.net/tw_dsconf/ss-62245351 translation invariance subsampling Conv + Pooling Conv + Pooling MLP
- 11. Feature Vector Representation Bengio, 2014, Representation Learning: A Review and New Perspectives FeatureVector Representation Learning http://deeplearning4j.org/convolutionalnets NN feature extractor
- 12. Feature Vector Representation Input Neural Networks including MLP, CNN, RNN Feature Vector [ 0.25, 0.33…, 0.78 ] Encoding Matching Feature Vector 1 Feature Vector 2 Similarity Function Score • cosine similarity • v1 TM v2
- 13. Outline 1. Information Retrieval 2. Neural Networks and Convolutional Neural Networks 3. Feature Vector Representation 4. Sentence Retrieval 5. Image Retrieval 6. Conclusion
- 14. <Pad> <Pad> <Pad> <Pad> Sentence Retrieval http://emnlp2014.org/papers/pdf/EMNLP2014181.pdf Yoon Kim, 2014, Convolutional Neural Networks for Sentence Classification Neural Net Architecture NN feature extractor 1 of K (one-hot) + word embedding Conv + MaxPooling
- 15. Sentence Retrieval Results: • Training acc: 99% up • Validation acc: 99% up Experiment: • 5 classes sentences • Training Set 80% • Validation Set 20% Feature vector PCA tSNE
- 16. Sentence Retrieval Recurrent Neural Network Kyunghyun Cho et. al., 2014, Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation https://devblogs.nvidia.com/parallelforall/ introduction-neural-machine-translation-gpus-part-2/
- 17. Skip-Thought Vectors Skip-Thought Vectors http://arxiv.org/pdf/1506.06726v1.pdf Encode a sentence into a thought vector hi by predicting its neighbor sentence. Ryan Kiros et. al., 2015, Skip-Thought Vectors … I just got back home. I could see the cat on the steps. This was strange. …
- 18. Channel A http://www.cherry123.com/data/book/14299/book14299_2.html Skip-Thought Vectors ? ? ? ? ? ? ? ! ? ! ? , ,
- 19. Semantic Similarity Skip-Thought Vectors query sen skt result skt score … … 0.934 0.918 0.901 0.898 0.889 0.861 0.836 0.831 0.812 0.777 0.765
- 20. Reference Relationship Skip-Thought Vectors query sen skt result skt score 0.898 0.916 0.836 … … 0.755 0.757 0.763
- 21. Limitations Skip-Thought Vectors • Requiring huge size of corpus • … I just got back home. I could see the cat on the steps. This was strange. … • … I got back to office. I could see the cat on the steps. This was cool. … 11,051 novels with 17,515,150 sentences • Scenario dependency query sen skt result skt score 0.697 … … 0.697 0.697
- 22. Something Interesting Skip-Thought Vectors query sen skt result skt score 0.889 0.697 skip-thought knows: = I’m ? skip-thought knows: = ?
- 23. Image Retrieval “A cat” Image Input Expected Outcome: Feature Vector Calculation Matching Database
- 24. Image Retrieval Input Image pre- processing (2) (1) Hashing Vector Layer FC + Softmax Layer . . . . . . CNN network 1st: various targets Step1 Training copy CNN network Step2 Training Kevin Lin et. al, 2015, Deep Learning of Binary Hash Codes for Fast Image Retrieval . . . . . . 2nd: task specific targets encoding application Hashing Vector Layer copy CNN network copy Feature Vector AlexNet VGG
- 25. Image Retrieval Kevin Lin et. al, 2015, Deep Learning of Binary Hash Codes for Fast Image Retrieval Demo http://tweddielin.pythonanywhere.com/ https://github.com/tweddielin/flask-imsearch Github
- 26. Conclusion 1. Deep Learning => Trainable Feature Extractor 2. Type of Neural Nets: MLP, CNN, RNN 3. • http://www.slideshare.net/tw_dsconf/ss-62245351 • http://speech.ee.ntu.edu.tw/~tlkagk/courses_MLSD15_2.html
- 27. Backup
- 28. Convolutional Neural Networks http://cs231n.github.io/convolutional-networks/ More Information 1 1 2 4 5 6 7 8 3 2 1 -2 1 2 3 4 ReLU max(0,x) Convolved Feature MaxPooling