Каковы лингвистические паттерны, которым следуют пользователи социальных сетей, чтобы высказывать иронию в совсем коротких фразах? Лингвистические средства - такие как неоднозначность, непоследовательность, неожиданность эмоциональный контекст, гораздо более широкий, чем просто негативная или позитивная тональность - играют очень важную роль триггеров иронии. В иронических текстах буквальный смысл сообщения как правило отрицается, но формальные маркеры отрицания отсутствуют. Это делает задачу определения иронии очень сложной. В своем выступлении я опишу как ирония выражается в социальных сетях (Twitter, Amazon, Facebook и др.) и каково современное положение дел в автоматическом определении иронии. Определение иронии очень важно для таких задач анализа текста как определение тональности сообщения, извлечение мнений, или анализ репутаций, и существует определенный интерес исследовательского сообщества к этой теме. На конференции SemEval 2015 будет организована задача-соревнование по определению тональности фигуративного языка в Твиттере (Sentiment Analysis of Figurative Language in Twitter, http://alt.qcri.org/semeval2015/task11/). В конце я коснусь еще более сложной проблемы различения иронии, сатиры и сарказма, например: Если вам тяжело смеяться над собой, я буду счастлив сделать это за вас.

1. On Irony Detection in Social Media
Paolo Rosso
Natural Language Engineering Lab – PRHLT Research Center
Technical University of Valencia
http://www.dsic.upv.es/~prosso/
Artificial Intelligence & Natural Language (AINL)
Moscow, 12th September 2014

2. Outline
•Figurative language: humour, irony,…
•Irony: linguistic device for polarity negation
•Verbal vs. situational irony
•Irony in social media
•Benchmark activities and projects on irony detection
•Recent works on irony detection: 2013 & 2014

3. Figurative language processing
•Figurative vs. natural language: figurative vs. literal meaning
•Humour, irony, metaphor etc.
•No facial expression or voice pitch
•Irony and opinion mining: implicit negation of polarity in sentiment analysis
•Opposition (lack of an explicit negation marker), incongruity, intentionality, ambiguity, unexpectedness, etc.
•Verbal vs. situational irony: e.g. A vegetarian having a heart attack outside Mc Donald’s / Burger King…

4. Visual / situational irony (incongruity)

5. Intentionality (most of the times…)
Picture taken at Kurskiy railway station in Moscow, one month ago

6. Verbal + visual irony (unexpectedness)

7. Irony in Russian

8. Irony in Russian

9. Irony in Russian (incongruity)

10. Irony and reputation in social media

Toyota's new slogan; moving forward (even if u don't want to);
hahahaha :)

'Toyota; moving forward.' Yeah because you have faulty brakes
and jammed accelerators. :P

My car broke down! Nooooooooooo! I bought a Toyota so that
it wouldn't brake down.:(

CERN recruiting engineers from Toyota for further
improvements to their particle accelerator :P IamconCERNed
#Toyota tweets

11. Irony and hashtags (the wisdom of crowds)

12. Irony and virality: viral effect / viral marketing

13. Irony, sarcasm or satire

If you find it hard to laugh at yourself,
I would be happy to do it for you
My mother never saw the irony
in calling me a son-of-a-bitch

14. Humour and irony: one-liners
Jesus saves, and at today's prices, that's a miracle!
Love is blind, but marriage is a real eye-opener.
Drugs may lead to nowhere, but at least it's a scenic route.
Become a computer programmer and never see the world again.
My software never has bugs; it just develops random features.
Sex is one of the nine reasons for reincarnation; the other eight are unimportant.
I've got the body of a god ...unfortunately is Buddha.

15. Humour and irony: one-liners: some pattern
Jesus saves, and at today's prices, that's a miracle! [ambiguity]
Love is blind, but marriage is a real eye-opener. [antonymy]
Drugs may lead to nowhere, but at least it's a scenic route. [human weakness]
Become a computer programmer and never see the world again. [common topic / community]
My software never has bugs; it just develops random features. [??]
Sex is one of the nine reasons for reincarnation; the other eight are unimportant. [language]
I've got the body of a god ...unfortunately is Buddha. [irony]

16. Humour and irony: more examples
I’m on a thirty day diet. So far, I have lost 15 days
Change is inevitable, except from a vending machine
Children in the back seats of cars cause accidents, but accidents in the back seats of cars cause children.
Don’t worry about what people think. They don’t do it very often.
I feel so miserable without you, it’s almost like having you here.
 Sometimes I need what only you can provide: your absence.

17. Humour and irony: more patterns
I’m on a thirty day diet. So far, I have lost 15 days.
Change is inevitable, except from a vending machine.
Children in the back seats of cars cause accidents, but accidents in the back seats of cars cause children.
Don’t worry about what people think. They don’t do it very often.
I feel so miserable without you, it’s almost like having you here.
 Sometimes I need what only you can provide: your absence.

18. Irony and humour: more patterns
I’m on a thirty day diet. So far, I have lost 15 days. [incongruity]
Change is inevitable, except from a vending machine. [ambiguity]
Children in the back seats of cars cause accidents, but accidents in the back seats of cars cause children. [syntactic ambiguity]
Don’t worry about what people think. They don’t do it very often. [irony]
I feel so miserable without you, it’s almost like having you here. [irony]
 Sometimes I need what only you can provide: your absence. [irony]

19. State-of-the-art
Humour recognition & generation: Phonological, incongruity, semantics (Binsted, Mihalcea, Strapparava).
Irony, sarcasm, satire detection: Similes, onomatopoeic expressions, headlines (Veale, Hao, Carvalho, Tsur)

20. Irony and humour: some features

N-grams: frequent sequences of words
Descriptors: tuned up sequences of words
POS n-grams: POS templates
Polarity: polarity of words
Affectiveness: emotional content (WordNet Affect)
Pleasantness: degree of pleasure (Whissel’s dictionary)
Funniness: relationship between humor and irony (humour domains and lexical ambiguity)
Tested on Amazon viral effect corpus: (Reyes and Rosso, 2013)

21. Irony detection: more ambitious features
•
Signatures: Pointedness (typographical marks: punctuation or emoticons); Counter- factuality (discursive marks: adverbs implying negation: nevertheless); Temporal compression: opposition in time (adverbs of time: suddenly, now).
•
Unexpectedness: Temporal imbalance (opposition in a same document); Contextual imbalance (inconsistencies within a context – semantic relatedness).
•
Style: Character n-grams (c-grams); Skip n-grams (s-grams); Polarity s-grams (ps-sgrams).
•
Emotional contexts: Activation (degree of response that humans have under an emotional state); Imagery (how difficult is to form a mental picture of a given word); Pleasantness (degree of pleasure produced by words).

22. Examples
•
Activation:
My male(1.55) ego(2.00) so eager(2.25) to let(1.70) it be stated(2.00) that I’m THE MAN(1.8750) but won’t allow(1.00) my pride(1.90) to admit(1.66) that being egotistical(0) is a weakness(1.75) ...
•
Imagery:
Yesterday(1.6) was the official(1.4) first(1.6) day(2.6) of spring(2.8)... and there was over a foot(2.8) of snow(3.0) on the ground(2.4).
•
Pleasantness :
The guy(1.9000) who(1.8889) called(2.0000) me Ricky(0) Martin(0) has(1.7778) a blind(1.0000) lunch(2.1667) date(2.33).

23. Results (Twitter)
Tested on Twitter corpus (Reyes et al., 2013)

24. Some references

Reyes A., Rosso P., Buscaldi D. (2012). From Humor Recognition to Irony Detection: The Figurative Language of Social Media. In: Data & Knowledge Engineering, 74:1-12
Reyes A., Rosso P. (2013). Making Objective Decisions from Subjective Data: Detecting Irony in Customers Reviews. In: Journal on Decision Support Systems, 53(4):754–760
Reyes A., Rosso P., Veale T. (2013). A Multidimensional Approach for Detecting Irony in Twitter. In: Language Resources and Evaluation, 47(1):239-268
Reyes A., Rosso P. (2014). On the Difficulty of Automatically Detecting Irony: Beyond a Simple Case of Negation. In: Knowledge and Information Systems, 40(3): 595-614
http://www. dsic.upv.es/~prosso/

25. Benchmark activities on irony detection
•
Pilot task @ Sentipolc: Evalita 2014
http://www.evalita.it/2014/tasks/sentipolc
Organisers: Viviana Patti (Università di Torino), Andrea Bolioli (CELI),
Malvina Nissim (Università di Bologna), Valerio Basile (University of Groningen),
Paolo Rosso (Universitat Politècnica de València)
•
Sentiment Analysis of Figurative Language in Twitter: Task 11 @ SemEval 2015
http://alt.qcri.org/semeval2015/task11
Organisers: Tony Veale (University College Dublin), John Barnden (University of Birmingham), Antonio Reyes (ISIT), Ekaterina Shutova (UC Berkeley),
Paolo Rosso (Universitat Politècnica de València)

26. Projects on irony/sarcasm detection (in US)

Army Research Office (ARO)
Sociolinguistically Informed Natural Language Processing:
Automating Irony Detection
http://www.reddit.com/r/irony
Secret Service seeks Twitter sarcasm detector
http://www.bbc.com/news/technology-27711109
http://www.washingtonpost.com/blogs/the-fix/wp/2014/06/03/the-secret-service-wants- software-that-detects-social-media-sarcasm-yeah-sure-it-will-work/

27. The tweet should be detected as ironic…

28. J. M. Whalen, P. M. Pexman, A. J. Gill & S. Nowson
Behavior & Information Technology (32)6: 560-569, 2013.
Verbal irony use in personal blogs

29. 
71 regular bloggers (24 male and 47 female) from North America, UK, Australia and New Zeeland.
The utterance was only counted as ironic if it was clear that a literal interpretation was not intended.
Hyperbole was the ironic form most frequently used by bloggers (for instance wrt sarcasm)
Inter-annotator agreement for identifying that an utterance was ironic: 89.57% (on the 25% of the blogs, selected randomly)
Inter-annotator agreement on the category: 98.36%

30. #Irony or #Sarcasm A quantitative and qualitative study based on Twitter
Po-Ya Angela Wang
Proc. 27th Pacific Asia Conference on Language, Information, and
Computation (PACLIC 27), 2013

31. Irony & Sarcasm
Identify similarities and distinctions
Quantitative Sentiment Analysis
Qualitative content analysis
Special way of language creativity
Interaction between cognition
and language
Speaker intention plays an important role
Irony is an umbrella term that covers Sarcasm

32. 
Corpus: 500 tweets #irony & 500 tweets #sarcasm

Tagging: crowdsourcing (participants are asked to judge how good the example is to be ironic/sarcastic).

They used a lexicon of 2600 positive words and 4783 negative words: difference between positive and negative words in a tweet is the sentiment score of the tweet.

Interest to understand how speakers use sentiment words in these types of language creativity.

Sarcastic tweets use more positive words but ironic tweets use more neutral

The positive words used in tweets seems to represent the aggressive intention

33. Sarcasm as contrast between a positive sentiment and a negative situation
E. Riloff, A. Qadir, P. Surve, L. De Silva, N. Gilbert & R. Huang
Proc. Conference on Empirical Methods in Natural Language
Processing (EMNLP), 2013

34. 
Sarcastic tweets often express a positive sentiment in reference to a negative situation

The goal is to identify sarcasm that arises from the contrast between a positive sentiment referring to a negative situation

Identify stereotypically negative “situations” (unenjoyable or undesirable)

#sarcasm reveals the intended sarcasm, but we do not always have the benefit of an explicit sarcasm label

35. Positive sentiment word with a negative activity or state
Oh how I love being ignored #sarcasm
Absolutely adore it when my bus is late #sarcasm

36. Authors
Focus on positive sentiments that are expressed as a verb phrase or as a predicative expression and negative activities or states that can be complement to a verb phrase.
Assume sarcasm probably arises from positive/negative contrast and exploit syntactic structure to extract phrases that are likely to have contrasting polarity
Harvest the n-grams that follow the word “love” as negative situation candidates, then selected the best of them using a scoring metric and add them to a list of negative situation phrases.

37. 
Collected 1,600 tweets with a sarcasm hashtag (#sarcasm or #sarcastic) and 1,600 without this hashtags.
Created a gold standard data set of manually annotated tweets (sarcasm hashtags were removed)
They perform a set of experiments, one of which consist in label a tweet as sarcastic if contains a positive sentiment phrase in close proximity to a negative situation phrase, both extracted from their bootstrapping algorithm. Achieves a precision of 70%
Contrasting a positive sentiment with a negative situation seems to be a key element of sarcasm.

38. Modelling irony in Twitter
F. Barbieri & H. Saggion
Proc. of the Student Research Workshop at the 14th conference of the European Chapter of the Association for Computational Linguistics (EACL), 2014.

39. Irony
Model uses seven groups of features to represent each tweet:
*Frequency: gap between rare and common words
*Written-spoken: written-spoken style uses
*Intensity: intensity of adverbs and adjectives
Structure: length, punctuation, emoticons
Sentiments: gap between positive and negative terms
Synonyms: common vs. rare synonyms use
Ambiguity: measure of possible ambiguity
Dataset used: (Reyes et al., 2013)
Decision tree
Education
Humor
Irony
Politics
* not used before for irony detection

40. Frequency
ANC: American National Corpus Frequency Data to measure the frequency of word usage
Written-Spoken
Intensity
Intensity of Potts1 adjectives and adverbs scale based on star ratings on service and product reviews
1 http://www.stanford.edu/~cgpotts/data/wordnetscales/
Synonyms
WordNet & ANC
WordNet
Structure
Ambiguity
Sentiments
SentiWordNet

41. Model
Education
Humour
Politics
P
R
F1
P
R
F1
P
R
F1
Reyes et al.
0.76
0.66
0.70
0.78
0.74
0.76
0.75
0.71
0.73
Authors
0.73
0.73
0.73
0.75
0.75
0.75
0.75
0.75
0.75

42. Modelling sarcasm in Twitter, a novel approach
F. Barbieri & H. Saggion
Proc. of the 5th Computational Approaches to Subjectivity, Sentiment & Social Media, WASSA 2014.

43. Experiments
Sarcasm vs
Education
Humor
Irony
Newspaper
Politics
The best results are obtained when distinguished Sarcasm from Newspaper tweets (F1: 0.97)
Difficulty in distinguishing sarcastic tweets from ironic ones (F1 : 0.62)
Relevant features to detect sarcasm against irony are two:
Use of adverbs: sarcasm uses less adverbs but more intense
Sentiment scores: sarcastic tweet are denoted by more positive sentiments than irony

44. An impact analysis of features in a classification approach to irony detection in product reviews
K. Buschmeier, P. Cimiano & R. Klinger
Proc. of the 5th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis Association for Computational Linguistics, WASSA 2014

45. 
Aim: to contribute to a deeper understanding of the linguistic properties of irony and sarcasm as linguistic phenomena and their corpus based evaluation and verification.

Authors analyze the impact of a number of features which have been proposed in previous research on irony detection

Automatic classification of a product review corpus from Amazon, by Filatova (Irony and sarcasm: Corpus generation and analysis using crowdsourcing, LREC 2012)
Irony detection as a supervised classification problem

46. Features
Imbalance between the overall polarity of words in the review and the star-rating
Hyperbole indicates the occurrence of a sequence of three positive or negative words in a row
Quotes indicates that up to two consecutive adjectives or nouns in quotation marks have a positive or negative polarity
Pos/Neg & Punctuation span of up to four words contains at least one positive(negative) but no negative (positive) word and ends with at least two exclamation marks
Pos/Neg & Ellipsis indicates that such a positive or negative span ends with an ellipsis (“…”)
Emoticon indicates the occurrence of an emoticon
Punctuation conveys the presence of an ellipsis as well as multiple question or exclamation marks or a combination of the latter two
Interjection indicates the occurrence of terms like “wow” and “huh”
Laughter measures onomatopoeia as well as acronyms of grin or laughter
Bag of words

47. 
Classifiers: SMV, Naïve Bayes, Logistic Regression, Decision Tree and Random Forest Classifier

Corpus: 1254 Amazon Reviews, 437 ironic utterances.

Baselines: Star-rating relies only on the number of stars assigned in the review as feature. Bag-of-words exploits only the unigrams in the text as features, sentiment word count, All (all features)

Performed experiments using different feature set combinations for the different classifiers.

The best result is achieved by using the star-rating together with bag-of-words and all features with a logistic regression approach (F1: 0.74)

48. L. Alba-Juez & S. Attardo
Evaluation in Context (Chapter 5)
John Benjamins Publishing Company, 2014
The evaluative palette of verbal irony

49. Irony
Negative
Most frequent and common type of verbal irony
Typical examples of sarcasm where an apparently positive comment expresses a negative criticism or judgment of a person, a thing or a situation.
Positive
Positive evaluation of a given person, thing or situation.
Frequently found in family discourse
Neutral
No intention of criticizing or praising any participant, thing, or situation
The utterance may include some kind or overt evaluation (very distant from either a positive or a critical negative position).

50. Irony
Negative
After Peter betrays his friend Tom, Tom says to Peter:
You’re certainly my best friend ever!
Tom is using negative irony in order to express his very negative evaluation of the way in which Peter has behaved.
Positive
Daniel comes back home from school and shows his father his report-card, which is full of As, to which his father reacts in the following manner:
Father: Daniel, I’m really worried; your grades are terrible! (with blank face)
Daniel: (giggles) Thank you, Dad
The father is trying to express his pride for his son’s success, an ironic act that is clearly understood by Daniel, as can be deduced from hi answer and reaction.
Neutral
From Blaise Pascal Letter XVI.
The letter is longer than usual because
I didn’t have the time to make it shorter
Seems to be not intention of criticizing or praising any participant, thing, or situation. Pascal was using fine irony in order to show wittiness, and therefore be funny.

51. 
Purpose: to see if the native speakers of each of the 2 languages distinguished between the ironic and non-ironic utterances, as well as to verify whether there was any significant difference in the identification of irony’s polarity.

Designed a questionnaire (both in English and Spanish) based on 20 situations. Ten of these situations contained some ironic utterances that could be related to a positive, a negative or a neutral evaluative stance, and the other ten were used as distractors.

38 native speakers of English and 56 of Spanish

Participant would have to classify each of the 20 situations according to the labels ironic/sarcastic, polite/impolite, aggressive/not aggressive, humorous/non-humorous

52. 
Conclude that speakers can identify reliably ironical and non ironical utterances

Results reveal that there seems to be no difference between the identification of negative irony and that positive and/or neutral irony, which not only supports authors’ hypothesis in favor of the existence of different “evaluative values” in ironic speech acts, but in fact supports a much stronger claim namely that positive and neutral irony are not significantly harder to identify than negative irony

53. Getting reliable annotations for sarcasm in online dialogues
R. Swanson, S. Lukin, L. Eisenberg, T. Chase Corcoran & M. A. Walker
Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC) 2014

54. 
Report the first study of the issues involved with achieving high reliability labels for sarcasm in online dialogue
Authors used Internet Argument Corpus (IAC), a large corpus of online social and political dialogues. The initial IAC annotation involved 10,003 Quote-Response (Q-R) pairs where Mechanical Turkers were shown seven Q-R pairs and asked to judge whether the response was sarcastic or not.
Turkers were not given additional definitions of the meaning of sarcasm
A subset of 25 new annotations was used to compare the different reliability measures on gold standard data in terms of accuracy as a function of the number of Turker annotations.

55. Who cares about sarcastic tweets? Investigating the impact of sarcasm on sentiment analysis
Diana Maynard and Mark A. Greenwood
Proceedings of the Ninth International Conference on
Language Resources and Evaluation, LREC 2014

56. 
Consider in particular the effect of sentiment and sarcasm contained in hashtags, and have developed a hashtag tokenizer for GATE, so that sentiment and sarcasm found within hashtags can be detected more easily
Tweets labeled with the hashtag #irony typically do not refer to verbal irony, but to situational irony. Collected a corpus of 257 tweets containing the hashtag #irony, and found that only 2 tweets contained clear instances of verbal irony, about 25% involved clear situational irony, while about 75% referred to extra-contextual information, so that the meaning was not clear.

57. Sarcasm Detection on Czech and English Twitter
Tomáš Ptácek, Ivan Habernal and Jun Hong
Proc. 25th Int. Conf. on Computational Linguistics
COLING-2014

58. Chinese Irony Corpus Construction and Ironic Structure Analysis
Y. Tang and H. Chen
Proc. 25th Int. Conf. on Computational Linguistics
COLING-2014

59. Emotions and Irony per Gender in Facebook
F. Rangel, I. Hernández, P. Rosso & A. Reyes
Proc. Workshop on Emotion, Social Signals, Sentiment
& Linked Open Data (ES³LOD), LREC-2014

60. Emotions & irony per gender in FB
Anger
Fear
Disgust
Surprise
Joy
Sadness
+
+
Ekman 6 basic emotions + no-emotion

61. Statistics wrt irony
ironic/non-ironic comments (2/3 annotators)
ironic comments per topic and gender (2/3 annotators)
ironic comments per emotion (2/3 annotators)
ironic comments per annotator

62. Inter-annotator agreement: irony
‣
Fleiss Kappa: It allows multiple annotators (three in our case) and binary variables (ironic / non-ironic)
‣
We obtained a value of 0.0989 -> very low index of agreement
‣
Irony is quite subjective and depends on annotators, their moods, linguistic and cultural context: we did not provide a common definition for irony
‣
Contextual information was not provided, only individual comments
‣
Males tended to be more ironic than females (in this corpus)
‣
The category politics is the one with more negative emotions and irony (in Spain? Difficult to believe it… #irony)
‣
EmIroGeFB Facebook corpus tagged with Emotions, Irony and Gender:

63. Inter-annotator agreement: irony & emotional comments
‣
Kappa Diaz-Sidorov (it allows to calculate concordance for more than two annotators, in our case three, with multiple not mutually exclusive categories, the six basic emotions, in the subset of comments identified as ironic
‣
We obtained a negative value of -0.0660: there is no agreement among annotators

64. Spasibo! Questions?
Irony (and its detection) is fun!
Enjoy it! Enjoy task 11 @ SemEval-2015
http://alt.qcri.org/semeval2015/task11/
Paolo Rosso: prosso@dsic.upv.es
Artificial Intelligence & Natural Language (AINL)
Moscow, 12th September 2014