Sentiment analysis using imdb 50 k data

1. Sentiment Analysis
(Deep Learning)
Sarthak Dasgupta

2. Using IMDB-
50k dataset to
do sentiment
analysis

3. Outcome:-
Based on public review, we will
perform sentiment Analysis and
will try to generalize the
emotion of Public toward the
movie.

4. First 5 public review and their sentiments :-

5. Bar graph representing the count of each class
As we can see, half the
reviews has positive
sentiment and other half
has negative sentiment

6. Preprocessing the “Sentiment” column
We have 2 main values in
sentiment - ‘positive’ &
‘negative’. So assigning 1s
and 0s to them

7. Data Cleaning
Clearing data
● process of clearing punctuation marks in data.
● cleaning unnecessary marks in data.
● capitalization to lowercase.
● cleaning extra spaces.
● removal of stopwords in sentences.

8. Tokenization:-
Word
1. thesis
2. Behind
1. rise
Token
1. 13091
2. 383
3. 2007

9. Splitting the Dataset for testing and training
We splitted the data set in 80-20 ratio
randomly for training and testing our
models after removing stopword from
lines.
TRAIN size: 40000
TEST size: 10000
from sklearn.model_selection
import train_test_split
x_train, x_test, y_train,
y_test =
train_test_split(data,sentim
ent,test_size = 0.2,
random_state = 42)

10. Words and their Equivalent Tokens:-
thesis behind rise evil
seems br br hitler bad man
bad man hated jews case miss
going fact every scene film
br br
13091 383 2007 337 85
1 1 2009 16 44 16
44
1631 4271 296 557 70
90 73 47 3 1 1

11. Sequential Model Summary

12. Model Training & Evaluating
Training the model with necessary informations
history = model.fit(x_train_pad, y_train, validation_split=0.3,
epochs=5, batch_size=1000, shuffle=True, verbose = 1)
Then we evaluate the model on test data
result = model.evaluate(x_test_pad, y_test)

13. Data visualization : Accuracy
plt.figure()
plt.plot(history.history["accuracy"
], label = "Train")
plt.plot(history.history["val_accur
acy"], label = "Test")
plt.title("Accuracy")
plt.ylabel("Acc")
plt.xlabel("epochs")
plt.legend()
plt.show()

14. Data visualization : Loss
plt.figure()
plt.plot(history.history["loss"],
label = "Train")
plt.plot(history.history["val_loss"
], label = "Test")
plt.title("Loss")
plt.ylabel("Acc")
plt.xlabel("epochs")
plt.legend()
plt.show()

15. Creating a second model
We remove
the drop-out
layer and
create a new
model

16. Fitting the second model and evaluating
accuracy

17. Thank You