Real-time Sentiment Analysis in a Facebook Messenger Bot
1. Use of Sentiment Analysis via
trained set, in message sent to
Facebook messenger bot
created via AWS
lambda/Stamplay.
2. Introduction
We have created a Facebook messenger bot. Bot doesn’t know how to
respond according to situations but we have to find the general idea.
So we are introducing the Sentiment Analysis on the message and
figure out the proper response according to the user message and
record the overall response or attitude towards the bot.
3. Why bots ?
~90% of our time on mobile is spent on email and
messaging platforms
4. 3rd generation platform
• Bots: New Apps
• Voice: User Interface
• AI: Protocol (linguistic analysis, image recognition, concept inference,
sentiment analysis, text enrichments and many other techniques)
• Messaging Apps: Browsers (Slack, Skype or Facebook Messenger)
5.
6. Basic Idea/Features
The basic idea is to create a bot that can respond to human sentiments
and also provide the feedback to us if the bot is receiving positive or
negative response. Bot could send message to user according to their
response.
We are using “Sentiwordnet” and java to figure out the sentiment.
Sentiwordnet is a huge training set, that contain the words , stemmed
words and phrases with their corresponding positive and/or negative
value.
7. Dataset/Source
There are 3 sources for data.
1) SentiWordText
We can consider the sentiwordtext as the training set data, We have altered
the sentiwordtext in several places for to make more suitable for out purpose.
It is stored in Amazon S3, to make it accessible from AWS lambda.
2) User Message to bot
The test set here is the message the user sends to the messenger. We compare
the phrase and word again set in sentiwordtext to get the sentiment value of
the message and process it.
3) AWS DynamoDB
The messages and the calculated sentiment is saved in the dynamodb along
with userid, so later it can be scanned easily to get the particular user’s data.
8. Data Source Attribute
• Sentiwordnet: Sentiwordnet had 6 attributes namely, POS, ID,
PosScore, NegScore, SynsetTerms, Gloss. We used java to isolate the
word from Sysset Terms and stored the PosScore and NegScore for
the word. They are all textual file in tab separated format
• User Message: User message consisted of many attribute in JSON
format, We are interested in message and facebook userId.
• AWS DynamoDB: dynamodb is NOSql, so there are no fix schema,
how ever we created object with attributes. ID which is primary key
and unique(set it as timestamp), facebookID (secondary key, used for
searching),message and sentiment value.
9. Tools
There are lots of tools involved in this project
1) Facebook Messenger bot: The tool which is used to interact with user and get the
input data to calculate the sentiment.
2) Stamplay: A online tool used to get the message from bot and perform the AWS
lambda function. After the lambda function return result, carryout corresponding
reply.
3) AWS lambda: Functionality provided by Amazon web services, by the means of which
we can perform java code in Amazon server without having to get much server time.
4) Sentiwordnet: The trained set text file that contains the trained set with the stemmed
words and their corresponding sentiment value.
5) AWS DynamoDb: The NOSql database by Amazon, where we store message, its value
and user id
6) AWS S3: The Amazon storeage facility, where the sentiwordnet is stored. Lambda uses
this service to get sentiwordnet.txt
11. Analysis/Algorithm
For the purpose of analyzing the words, we are using Bag of Words
Approach, where most common words are assigned certain value. The
sentiwordnet training set created a hash of the words with vector
about the stemmed word and the value of sentiment it represents. We
the use the data and compare it against the hash and we find the
sentiment value for a given phrase.
The value is then measured against the values set for positive, negative,
and neutral response. The messages are all saved with their
corresponding sentiment value.
Certain trigger word currently “recommend” will get the data from the
database, and calculate the average for the messages and return
proper phrase to send to user. Stamplay sends the message to user
12. SentiWordnet
WordNet is lexical database which groups the synonyms of words
namely, synsets. SENTIWORDNET 3.0 glosses from the Princeton
WordNet Gloss Corpus, according to which a gloss is actually a
sequence of WORDNET synsets, thus based on “bag of synsets” model,
unlike the “bag of words” approach in previous version.
Basically how it populates its data is, it gets positive and negative
synsets for a synset within certain k distance from itself. The glosses
are used to classify the positive,negative or neutral rating for the
synset.
http://sentiwordnet.isti.cnr.it/
13. Result
We hope the result would be a smarter bot which will be able to
respond according to users current sentiment. Finding sentiment would
help in better sales pitch, use of softer language, word of appreciation
towards user etc.
The way user interacts with the bot could help us know the feedback
and improve the bot in future according to the feedback.