basic Deep learning method in NLP

1. History of Deep
Learning in NLP
Wen Li, Fangya Tan

2. Overview
• What is NLP for: Computer science discipline focuses on the analysis of human
languages
• NLP Application & Challenges
• Traditional Approaches: CNN, RNN
• Innovative Approaches: Video
• Q & A

3. What Is NLP
• Natural language processing (NLP) deals with building computational
algorithms to automatically analyze and represent human language.
• Aim: produce a condensed prestation of an input text that captures the core
meaning of the original text. Extractive vs Abstractive
• 7 Problems: Text Classification, Language Modeling, Speech Recognition, Caption Generation,
Machine Translation, Document Summarization, Question Answering
• Challenge: curse of dimensionality, multiple documentation, evaluation

4. Deep Learning in NLP
Application
Why should we care?
NLP in Daily Life:
• Google Search Engine
• Voice Technology: Amazon Alexa, Apple Siri
• Youdao voice translation
• Auto Email detection
• Marketing (Father’s day)
NLP in Future:
• Social Media (including emoji)
• Sentimental analysis: spam vs Non-spama

5. Model 1:
Convolutional Neural Network:
represents a feature function that is
applied to constituting words or n-
grams to extract higher-level features
sentiment analysis, machine translation, and
question answering, among other tasks.
Basic Steps:
1. tokenize: Sentence into words,
matrix of d dimension
2. Filter for feature map
3. Produce Final sentence
Challenge: Long distance dependency

6. Model 2:
Recurrent Neural Network
• The main strength of an RNN is
the capacity to memorize the results
of previous computations
• Inputs of arbitrary length so as to
create a proper composition of the
input.
• Tasks:
Machine Translation, Image
Captioning, Language Modeling
RNN is effective at processing
sequential information

7. Innovative Approaches Basic Traditional Approaches:
• RNN: recursive Neural Network
• Reinforcement Learning
• Unsupervised Learning
• Deep Generative Models
• Memory Augmented Network
Mutli-Modal Methods
Computer teaches themselves to
Recognize Cats
• Watch video of a generic cat or a specific
cat
• The feeling of petting a cat’s soft fur,
meow
• The letters ‘c’, ‘a’ and ‘t’
• Sometimes-selfish and largely
independent creatures

8. Reference:
• https://medium.com/dair-ai/deep-learning-for-nlp-an-overview-of-recent-
trends-d0d8f40a776d
• https://en.wikipedia.org/wiki/Curse_of_dimensionality
• https://arxiv.org/abs/1708.02709
• http://ruder.io/4-biggest-open-problems-in-nlp/
Thanks!