This document proposes a new method to measure the credibility of sentiment polarity in tweets based on the author's background knowledge and expertise. It consists of two main components: a sentiment analyzer to classify tweet polarity and an opinion credibility calculator. The credibility calculator considers features like the author's bio, lists, topic-related tweets, retweets, friends/followers to assign an expert score. It then combines the expert score with the sentiment analysis result to provide an overall credibility measurement for the tweet polarity. An experiment on Hong Kong protest tweets aims to evaluate the accuracy of the proposed credibility calculation approach.

1. S
Measuring Opinion Credibility in
Twitter
Mya Thandar and Sasiporn Usanavasin
School of Information and Communication Technology (ICT), Sirindhorn International Institute of Technology (SIIT),
Thammasat University, Thailand

2. Content
S Introduction
S Objective
S Research Problem
S Proposed Method
S Experiment
S Conclusion
2

3. Introduction
S In Hong Kong Protesting, many people such as Hong Kong citizen,
government staff, journalists and news channels express information and their
opinions over social media.
S In that event, we don’t know whose opinions are strong or credential.
S take the information from opinion content, the identifier of an author may help
to determine credibility.
S For that reason, we propose a new method to define the credibility of
sentiment polarity based on their expertise or background knowledge and
apply on Twitter: social media
3

4. Objective
S To calculate the credibility of sentiment polarity based on
author’s background knowledge in Twitter.
4

5. Research Problem
5
S In this case, these two tweets shows their opinion for HongKong Revolution.
S Whose opinion is reliable or not?

6. Research Problem (Cont..)
S According to this point:
S we added the accuracy weight of the author’s background knowledge for a
given topic based on the following information:
▪ Who is mention about this tweet?
▪ How much does this user know about this topic?
▪ What is their background knowledge?
6

7. Proposed Method
S assume when we classify user tweet’s polarity we didn’t think
this is not the fully percent of positive or negative.
S added the accuracy weight of the tweet polarity.
1) Find the author’s sentiment polarity for a given topics
2) Calculate the accuracy of author’s Sentiment Polarity for author tweet in a given
specific topic
7

8. Proposed Method
S consists of two main components:
S sentiment analyzer
S opinion credibility calculator
8

9. Overview system of credibility of tweet polarity
9

10. Sentiment Analyzer
S is to achieve classification of tweet’s polarity result.
S used Linear SVM to classify sentiment polarity of our tweet dataset.
S SVM is a supervised learning method used for classification.
S w represent to find maximum hyperplane and separates document from one class or
other
S ci corresponds positive and negative (1,-1) with class of document di .
S αi is to solve dual optimization problem.
10

11. Overview system of credibility of tweet polarity
11

12. Opinion Credibility Calculator
S calculate credibility of opinion based on an author’s expert knowledge for a given
topic.
S In twitter, we consider user bio, user list and user tweets to identify the accuracy
weight of user’s background knowledge.
S User bio: contains important information that indicates the expertise of the user, such as his/herself
summarized interests, career information and links to his/her personal web page.
S User Lists: allow users to organize people they are following into labeled groups. It contains self-
reported expertise indicator. i.e, follower’s judgment about one’s expertise and provide
straightforward cues about this judgment to other users.
S User Tweets: They tweeted almost everything: such as:
• their daily activities;
• comment on news;
• promotion of their company, etc.,
12

13. Expert Score
S we compute expert score for given topic by adding weight of author’s bio, list features and author’s
tweets behavior.
wBL = weight of author’s background knowledge (using Bio & List)
wTR = author’s topic related ratio (using Author’s Tweets)
wRT = ratio of author’s tweet retweet by other users for a given topic (using Author’s Tweets)
wF1 = ratio of author’s friends who are related with given topic (using Author’s Tweets)
wF2 = ratio of author’s follower who are related with given topic (using Author’s Tweets)
wOP= author’s opinions ratio for given topic(using Author’s Tweets)
13

14. wBL (using bio & list)
S apply N-gram approach (using unigram and bigram) to segment them
S extract Noun and Adjective using NLP toolkit1.
S used ontology concept to filter conceptual keyword for a given topic
S calculate the ratio of number of related keywords to total number of raw keywords.
S This strategy produces wBL(using bio and list) to define the author’s expert
knowledge.
14

15. Author’s tweets
S the next weight of author based on an author tweets behavior and his/
her network activities.
S focus on the following features that reflected the impact of user
background knowledge (expert knowledge score) for a given topic.
• author topic related ratio (wTR)
• ratio of author’s tweet retweet by other users (wRT) for a given topic
• ratio of author’s friends and followers who are related with given topic
(wF1, wF2)
• author’s opinions ratio for given topic (wOP)
15

16. wTR (Topic related ration)
S For author topic related ratio, we make the ratio of number of author’s tweet
related for specific topic to the number of his all tweet.
16

17. wRT (Author’s tweet retweet by other
users)
S In twitter, retweet is a re-posting someone else’s Tweet.
S Using retweet features, we compute how much times author’s tweet
has been retweeted by others for a particular topic.
17

18. wF1 and wF2 (Author’s friends and followers
who are related with given topic)
S wF1 and wF2 indicate the ratio of author’s friends and followers who
are related with given topic.
18

19. wOP (Author’s opinions ratio)
S For author’s opinions ratio;
S we assume how much times author expresses this opinions based
on his/her past all opinions for given topic.
S e.g., given tweet opinion is negative, we calculate number of
author’s negative tweet based on his/his all past tweet opinion.
19

20. Opinion credibility
S combine expert scores with the result of polarity from sentiment
analyzer
S calculate the credibility of tweet polarity result (Cop).
20

21. Preliminary Experiment
S use Twitter API to crawl HK revolution data (Sep 30 2014- Nov
30 2014) as training and testing dataset.
S use 1000 sample data for training dataset to learn sentiment
classification using RapidMiner Tool.
S labeled this dataset with tweet’s polarity(positive and negative).
S use many set of SVM parameters for finding the best result and
apply 10 fold cross validation.
21

22. Experiment (Sentiment Classification)
22

23. Experiment (Credibility Calculation)
23

24. Experiment (Credibility Calculation)
24
the range for credibility value: {highest>=70%, lowest<=30%, middle}

25. Experiment (Credibility Calculation)
25

26. Conclusion
S proposed a system to measure credibility of sentiment polarity based
on author’s expert knowledge.
S To find credibility opinion, our system performed 2 steps:
S Sentiment classification
S Opinion Credibility calculation
S used dataset Hong Kong Revolution dataset.
S will evaluate the accuracy of opinion credibility result.
26

27. Acknowledgements
S Center of Excellence in Intelligent Informatics, Speech and Language
Technology and Service Innovation (CILS), Thammasat University,
S Intelligent Informatics, and Service Innovation (IISI), SIIT,
Thammasat University
S NRU grant at SIIT, Thammasat University
27

28. S
Thank You! J
Any Questions???
28