This is the presentation from our first AI Meet held on Nov 19, 2016. You can join Artifacia AI Meet Bangalore Group: https://www.meetup.com/Artifacia-AI-Meet/

1. Introduction to Recurrent Neural Networks
with Application to Sentiment Analysis
by Rajarshee Mitra, Research Engineer (NLP) , Artifacia
(@rajarshee_mitra)
November 19, 2016

2. AI Meet|
Agenda
1. What is AI ?
2. What is NLU ? Why is it hard ?
3. Introduction to Neural Networks.
4. Introduction to Recurrent Neural Networks (RNN).
5. Application of RNN models.
6. Variants of RNN.
7. Sequence to Sequence Learning.
8. Sentiment Analysis - an application.
9. Food for Thought.

3. AI Meet|
What is AI
AI is the ability of software to mimic human brain and
perform human-like abilities such as understanding emotions
and meanings from text, handling ambiguities, recognizing
objects etc.

4. AI Meet|
What is NLU? Why is it hard ?
Natural Language Understanding is the ability to process, understand
and generate human languages (to create some action or intent).
● Language contains ambiguities.
“I am looking at the elephant in white pyajamas”
● Modification of a single word (insertion, deletion) changes the
meaning of the whole sentence.
● Context plays a serious role in language understanding.

5. AI Meet|
Neural Network - An introduction

6. AI Meet|
Neural Network - An introduction
INPUT OUTPUT TARGET
am He, running I, going
looked I, am I, at
sofa The, is The, is
Neural Language Modelling - skip gram model

7. AI Meet|
Neural Network - An introduction
Loss Functions:
1. Absolute Difference
2. Root Mean Square
3. Cross Entropy or Log Loss

8. AI Meet|
Recurrent Neural Nets

9. AI Meet|
Application of RNN Models
1. Sentiment Analysis
2. Language Modelling
3. Translation
4. Conversational Agents
5. Language Generation
6. Image Captioning
7. Text Summarization

10. AI Meet|
Some Interesting Variants of Neural
Network
1. Long Short Term Memory Networks
2. Gated Recurrent Unit.
3. End to End networks - Sequence to Sequence Learning
4. Memory Networks.

11. AI Meet|
Sequence to Sequence Learning

12. AI Meet|
Sentiment Analysis - An Application
The process of computationally identifying and categorizing
opinions expressed in a piece of text, especially in order to
determine whether the writer's attitude towards a particular topic,
product, etc. is positive, negative, or neutral.

13. AI Meet|
Sentiment Analysis - An Application
1. We can treat the last output vector as our predicted sentiment.
2. We calculate loss between our output and target vector which
contains the actual sentiment.
3. We update our model accordingly to minimize the loss.
4. A successfully learnt model will automatically predict
sentiments of unseen sentences.

14. AI Meet|
Food for Thought
As we are more approaching towards linking concepts of
neuroscience with mathematical concepts of Deep Learning, I
imagine a system which might have following components :
1. A processor or generator - An RNN that process sentence
word by word or generates sentence in the same way.
2. A memory that will facilitate read / write operations.
3. A some variant of feed forward neural network that will act
between the processor and the memory and will determine
which neurons to activate, what to read and write.

15. AI Meet|
Further Reading
1. http://karpathy.github.io/2015/05/21/rnn-effectiveness/
2. http://colah.github.io/posts/2015-08-Understanding-LSTMs/
3. http://papers.nips.cc/paper/5346-sequence-to-sequence-learni
ng-with-neural-networks.pdf
From Our Blog:
- http://research.artifacia.com/learn-deep-learning-the-hard-way

16. THANK YOU
Join
meetup.com/Artifacia-AI-Meet/