University of Sheffield, NLP
Diana Maynard
Mark Greenwood
University of Sheffield, UK
Who cares about sarcastic tweets?
Twitter is full of mindless drivel
●
OMMMFG!!! JUST HEARD EMINEM'S “RAPGOD”. SMFH!!!
these other dudes might as well stop ...
What are people reading about?
●
Of the top 10 Twitter accounts
with the highest number of
followers:
●
7 pop stars
●
2 so...
Even the mindless drivel could be useful
●
OMMMFG!!! JUST HEARD EMINEM'S “RAPGOD”. SMFH!!!
these other dudes might as well...
Sarcasm is a part of British culture
●
The BBC has its own webpage on sarcasm designed to teach
non-native English speaker...
How do you know when someone is being
sarcastic?
• Use of hashtags in tweets such as #sarcasm, #irony, #whoknew etc.
It's ...
My friend Barry likes Apple products
Or does he?
Understanding sarcasm is hard
Sarcastic or not?
How about now?
It often requires world knowledge
Capitalisation indicates sarcasm
But not always
What does sarcasm do to polarity?
●
Sarcasm often indicated by hashtags in tweets such as #sarcasm,
#irony, #whoknew etc.
...
Getting the scope of hashtags right
Eating breakfast food for lunch. Living the dream.
#toast #rebel #sarcasm
Getting the scope of hashtags right
Eating breakfast food for lunch. Living the dream.
#toast #rebel #sarcasm
Getting the scope of hashtags right
Eating breakfast food for lunch. Living the dream.
#toast #rebel #sarcasm
Getting the scope of hashtags right
Eating breakfast food for lunch. Living the dream.
#toast #rebel #sarcasm
Getting the scope of hashtags right
Eating breakfast food for lunch. Living the dream.
#toast #rebel #sarcasm
Analysing Hashtags
What's in a hashtag?
●
Hashtags often contain smushed words
●
#SteveJobs
●
#CombineAFoodAndABand
●
#southamerica
●
For NER...
How to analyse hashtags?
●
Camelcasing makes it relatively easy to separate the words, using an
adapted tokeniser, but man...
Identifying the scope of sarcasm
I am not happy that I woke up at 5:15 this morning.
#greatstart #sarcasm
You are really m...
Rules for identifying scope
I am not happy that I woke up at 5:15 this morning. #greatstart
#sarcasm
●
negative sentiment ...
Experiments with sarcastic hashtags

Collected a corpus of 134 tweets containing #sarcasm

Manually annotated sentences ...
Conclusions
●
Unlike most work on sarcasm detection, we don't try to
identify sarcasm where it's not explicitly indicated
...
Questions?
?
Who cares about sarcastic tweets? Investigating the impact of sarcasm on sentiment analysis.
Upcoming SlideShare
Loading in...5
×

Who cares about sarcastic tweets? Investigating the impact of sarcasm on sentiment analysis.

923

Published on

Presentation given at LREC 2014 on detecting the scope of sarcasm in tweets.

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
923
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
14
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Who cares about sarcastic tweets? Investigating the impact of sarcasm on sentiment analysis.

  1. 1. University of Sheffield, NLP Diana Maynard Mark Greenwood University of Sheffield, UK Who cares about sarcastic tweets?
  2. 2. Twitter is full of mindless drivel ● OMMMFG!!! JUST HEARD EMINEM'S “RAPGOD”. SMFH!!! these other dudes might as well stop rapping if they not on this level ● i've got dressed but only because I need biscuits ● I used to be so bad at naming any k idol group members pmsl I would get so confused and now I'm pro ;))) ● im gonna learn to be a lifegaurd hopfully so while everyone else is working in a shop actually doing stuff il be sitting on a pool side.yay
  3. 3. What are people reading about? ● Of the top 10 Twitter accounts with the highest number of followers: ● 7 pop stars ● 2 social media sites ● and Barack Obama ● Why on earth do we care about this stuff?
  4. 4. Even the mindless drivel could be useful ● OMMMFG!!! JUST HEARD EMINEM'S “RAPGOD”. SMFH!!! these other dudes might as well stop rapping if they not on this level ● i've got dressed but only because I need biscuits ● I used to be so bad at naming any k idol group members pmsl I would get so confused and now I'm pro ;))) ● im gonna learn to be a lifegaurd hopfully so while everyone else is working in a shop actually doing stuff il be sitting on a pool side.yay ➔ English people like biscuits. A lot. ➔ What do young people think about their future careers? ➔ People who like K Idol and RapGod also like Apple products
  5. 5. Sarcasm is a part of British culture ● The BBC has its own webpage on sarcasm designed to teach non-native English speakers how to be sarcastic successfully in conversation
  6. 6. How do you know when someone is being sarcastic? • Use of hashtags in tweets such as #sarcasm, #irony, #whoknew etc. It's not like I wanted to eat breakfast anyway #sarcasm • Large collections of tweets based on hashtags can be used to make a training set for machine learning • But you still have to know what to do with sarcasm once you've found it • Sarcasm generally entails saying the opposite of what you mean – But it doesn't necessarily just invert the polarity of an opinion – “It's not like I wanted to eat breakfast anyway” is negative when uttered sarcastically, but non-opinionated when uttered neutrally.
  7. 7. My friend Barry likes Apple products
  8. 8. Or does he?
  9. 9. Understanding sarcasm is hard Sarcastic or not? How about now?
  10. 10. It often requires world knowledge Capitalisation indicates sarcasm But not always
  11. 11. What does sarcasm do to polarity? ● Sarcasm often indicated by hashtags in tweets such as #sarcasm, #irony, #whoknew etc. ● It's very hard to identify sarcasm outside these parameters ● In general, when someone is being sarcastic, they're saying the opposite of what they mean ● So as long as you know which bit of the utterance is the sarcastic bit, you can simply reverse the polarity Eating breakfast food for lunch. Living the dream. #toast #rebel #sarcasm ● If there is no polarity on the original statement, the sarcastic version is probably negative It's not like I wanted to eat breakfast anyway #sarcasm ● If there's more than one hashtag, you need to look at the combination, and any sentiments they express
  12. 12. Getting the scope of hashtags right Eating breakfast food for lunch. Living the dream. #toast #rebel #sarcasm
  13. 13. Getting the scope of hashtags right Eating breakfast food for lunch. Living the dream. #toast #rebel #sarcasm
  14. 14. Getting the scope of hashtags right Eating breakfast food for lunch. Living the dream. #toast #rebel #sarcasm
  15. 15. Getting the scope of hashtags right Eating breakfast food for lunch. Living the dream. #toast #rebel #sarcasm
  16. 16. Getting the scope of hashtags right Eating breakfast food for lunch. Living the dream. #toast #rebel #sarcasm
  17. 17. Analysing Hashtags
  18. 18. What's in a hashtag? ● Hashtags often contain smushed words ● #SteveJobs ● #CombineAFoodAndABand ● #southamerica ● For NER we want the individual tokens so we can link them to the right entity ● For opinion mining, individual words in the hashtags often indicate sentiment, sarcasm etc. ● #greatidea ● #worstdayever ● We need to retokenise hashtags so that we can use the content in our application
  19. 19. How to analyse hashtags? ● Camelcasing makes it relatively easy to separate the words, using an adapted tokeniser, but many people don't bother ● We use a simple approach based on dictionary matching the longest consecutive strings, working L to R ● We use a combination of dictionaries (Linux dictionary, slang dictionary, plus gazetteers of Named Entities, modified manually) ● #lifeisgreat -> #-life-is-great ● #lovinglife -> #-loving-life ● It's not foolproof, however ● #greatstart -> #-greats-tart ● In an experiment with 2010 English hashtags (4538 tokens): P=98.12%, R=96.41% , F1= 97.25%. ● We could use a language modelling approach based on bigrams and trigrams, but since hashtags are often novel, it might not help much
  20. 20. Identifying the scope of sarcasm I am not happy that I woke up at 5:15 this morning. #greatstart #sarcasm You are really mature. #lying #sarcasm
  21. 21. Rules for identifying scope I am not happy that I woke up at 5:15 this morning. #greatstart #sarcasm ● negative sentiment + positive hashtag + sarcasm hashtag ● The positive hashtag becomes negative with sarcasm You are really mature. #lying #sarcasm ● positive sentiment + sarcasm hashtag + sarcasm hashtag ● The positive sentiment is turned negative by both sarcasm hashtags ● When in doubt, it's usually safe to assume that a sarcastic statement carries negative sentiment
  22. 22. Experiments with sarcastic hashtags  Collected a corpus of 134 tweets containing #sarcasm  Manually annotated sentences with sentiment  266 sentences, of which 68 opinionated (25%)  62 negative, 6 positive (yes, this is biased...)  Adding sarcasm detection improved accuracy of polarity detection from 27.27% to 77.28%  Even though we know these sentences are sarcastic, we don't always get polarity right  After implementing rules for sarcasm scope, 91% accuracy
  23. 23. Conclusions ● Unlike most work on sarcasm detection, we don't try to identify sarcasm where it's not explicitly indicated ● We instead examine the effect that known sarcasm has on the sentiment expressed in tweets ● We retokenise hashtags so that we can make use of information within them in order to identify sarcasm scope ● We develop a set of rules for determining sarcasm scope, and improve polarity detection as a result ● Lots more work could be done on this topic, but it's a #greatstart #really
  24. 24. Questions? ?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×