SlideShare a Scribd company logo
1 of 40
Download to read offline
10th Author Profiling task at PAN
Profiling Irony and Stereotype
Spreaders on Twitter
PAN-AP-2022 CLEF 2022
Bologna, 5-8 September
Reynier Ortega Bueno - Universitat Politècnica de València
Berta Chulvi - Universitat Politècnica de València
Francisco Rangel - Symanto Research
Paolo Rosso - Universitat Politècnica de València
Elisabetta Fersini - Università Degli Studi di Milano-Bicocca
Introduction
Author profiling aims at identifying
personal traits such as age, gender,
personality traits, native language,
language variety… from writings?
This is crucial for:
- Marketing.
- Security.
- Forensics.
2
Author
Profiling
PAN’22
Last years goal
Profiling Harmful Information Spreaders:
2019 - Profiling Bots
2020 - Profiling Fake News Spreaders
2021 - Profiling Hate Speech Spreaders
2022 - Profiling Irony and Stereotypes
Spreaders
3
Author
Profiling
PAN’22
Task goal
Given a Twitter feed, determine whether
its author is keen to spread irony or not,
with special focus on users who use irony
towards stereotypes.
4
Author
Profiling
Language:
English
PAN’22
Corpus
5
Author
Profiling
PAN’22
Methodology
1. Defining Taxonomy and Stereotypes Categories:
a. We examine the "target minority" field of the SBIC [Sap et al., 2020]
b. We identify 600 unique labels that could be considered a social category in SBIC, and
classify them into 17 categories:
(1) national majority groups; (2) illness/health groups; (3) age and role family groups; (4)
victims; (5) political groups; (6) ethnic/racial minorities; (7) immigration/national
minorities; (8) professional and class groups; (9) sexual orientation groups; (10) women;
(11) physical appearance groups; (12) religious groups; (13) style of life groups; (14)
non-normative behaviour groups; (15) man/male groups; (16) minorities expressed in
generic terms; (17) white people
2. Tweet Retrieval
a. We retrieve tweets containing at least one of the labels included in categories 5 to 14
of the taxonomy (with and without the hashtags irony and sarcasm).
b. We select users with more tweets accomplishing previous conditions.
Corpus
6
Author
Profiling
PAN’22
Methodology
3. Annotation Process
a. Irony: annotators were asked to mark the tweets where the user "express the opposite
of what was saying as a disguised mockery". If the user had more than 5 ironic tweets,
it was labelled as ironic.
b. Stereotypes: annotators were asked to check if the social categories in the tweets
were used to refer to a social group by associating them with a homogenising image of
the category (e.g., as if all gays or Muslims were the same and could be described with
that word). If the user had more than 5 tweets with stereotypes, it was labelled
accordingly.
4. Corpus Construction
a. Two independent annotators labelled the data (IAA 0.7093).
b. A third annotator sorted out disagreements.
c. For each user, we provide 200 tweets.
Corpus
7
Author
Profiling
PAN’22
IRONIC NON-IRONIC
Total
Stereotypes Non-stereotypes Total Stereotypes Non-stereotypes Total
Training 140 70 210 140 70 210 420
Test 60 30 90 60 30 90 180
Total 200 100 300 200 100 300 600
Statistics
Number of users per class and set. Each user contains 200 tweets.
Evaluation measures
8
Author
Profiling
PAN’22
Since the dataset is completely balanced for the two
target classes, ironic vs. non-ironic, we
have used the accuracy measure and ranked the
performance of the systems by that metric.
Baselines
9
Author
Profiling
PAN’22
CHAR 2-GRAMS + RF Bigrams with Random Forest
WORD 1-GRAMS + LR Bag of words with Logistic Regression
BERT + LSTM We represent each tweet in the profile utilising pretrained Bert-base model.
Later, we fed an LSTM with these vectors as input.
Symanto (LDSE) This method represents documents on the basis of the probability distribution of
occurrence of their words in the different classes. The key concept of LDSE is a
weight, representing the probability of a term to belong to one of the different
categories: irony spreader / non-spreader. The distribution of weights for a given
document should be closer to the weights of its corresponding category. LDSE
takes advantage of the whole vocabulary
65 participants
32 working notes
12 countries
10
Author
Profiling
PAN’22
Participation
https://mapchart.net/world.html
Approaches
11
Author
Profiling
PAN’22
Approaches - Preprocessing
12
Author
Profiling
Twitter elements (RT, VIA, FAV) Giglou et al.; Cao et al.; Lin et al.; Jang; Xu et al.; Wang & Ning; Zhang & NIng;
Sagar & Varma; Hazrati et al.
Emojis and other non-alphanumeric
chars
Wang & Ning; Zhang & Ning; Hazrati et al.; Butt et al.; Jian & Huang
Lemmatisation Haolong & Sun
Punctuation signs Giglou et al.; Lin et al.; Xu & Ning; Wang & Ning; Hazrati et al.; Dong et al.
Numbers Xu & Ning; Hazrati et al.; Butt et al.
Lowercase Giglou et al.; Xu & Ning; Butt et al.; Dong et al.; Siino & Cascia; Mangione & Garbo;
Zengyao & Zhongyuan; Croce et al.; Herold & Castro
Stopwords Giglou et al.; Hazrati et al.; Butt et al.
Infrequent terms Giglou et al.; Wang & Ning; Zengyao & Zhongyuan
X2
with PMI and TF/IDF Ikae
I/NI labels to the end of the tweets Jian & Huang
GloVe to filter out features Haolong & Sun
PAN’22
Approaches - Features
13
Author
Profiling
Stylistic features:
- Vocabulary size
- Number of tokens
- Tweet length
- Number of hashtags, mentions, URLS
- Number of emojis
- ...
Nikolova et al.
N-gram models Sagar & Varma; Butt et al.; Herold & Castro; Ikae; Nikolova et al.
Emotional and personality features Butt et al.
TextVectorizer Croce et al.
Embeddings Dong et al.; Yang et al.
Transformers
...BERT Cao et al.; Lin et al.; Xu & Ning; 71; Hazrati et al.; Jian & Huang;
Zengyao & Zhongyuan; Wentao & Kolossa; Rodriguez & Barroso
...SBERT Tahaei et al.
...BERTTweet Wang & Ning
PAN’22
Approaches - Features
14
Author
Profiling
Transformers + others
...BERT & Twitter RoBERTa + LM HateXPlain Mathew et al.
SBERT + emojis Tahaei et al.
SBERT + psychometrics, emotions and irony Tavan et al.
BERT + TF-IDF n-grams Das et al.; Gómez & Parres
SBERT + graph-based & one-hot embeddings Giglou et al.
+ Ironic-, contextual-, psychometric-related
features fine-tuned with datasets annotated
with sentiment/emotions from Kaggle
Tavan et al.
Sentence Transformers + n-grams + stylistic Jang
BERT & RoBERTa + FastText + stylistic Díaz et al.
PAN’22
Approaches - Features
15
Author
Profiling
PAN’22
CNN Díaz et al.
CNN + TF-IDF 1-grams + BiGRU Haolong & Sun
Sequence probabilities + n-grams + GPT2 & DistilGPT2 Huang
Irony identification at tweet level by combining:
1) structural features
2) sentiment words
3) fine-grained emotions by means of several lexicons
Hernández & Montes
Approaches - Methods
16
Author
Profiling
Logistic regression Butt et al.; Das et al.
Random Forest Sagar & Varma; Ikae; Nikolova et al.; Hernández & Montes
Bayes Huang
Multilayer Perceptron Hazrati et al.
Gradient Booster Classifier Tavan et al.
k-Nearest Neighbours Rodriguez & Barroso
Ensembles (trad. classifiers) Zengyao & Zhongyuan; Herold & Castro; Tavan et al.
Ensembles (trad. Class + DL) Siino & Cascia; Croce et al.
PAN’22
Approaches - Methods
17
Author
Profiling
Linear Feed Forward Networks Tahaei et al.
CNN Dong et al.; Mangione & Garbo; Wentao & Kolossa
GCNN Giglou et al.
bi-LSTM + CNN Yang et al.
Fully-connected networks Haolong & Sun; Labadie & Castro; Díaz et al.
AutoKeras; AutoGluon Wang & Ning; Xu & Ning; Zhang & Ning
BERT + Decision Rules Jian & Huang
BERT + SVM, MLP, Gaussian NB
& RF
Rodriguez & Barroso
BERT + CNN, LSTM, att. layer Gómez & Parres
BERT & DistilBERT + RF & SVM Jang
BERT + voting classifier Cao et al.; Lin et al.
PAN’22
Global ranking
18
Author
Profiling
PAN’22
Team Acc
Best 0.9944
LDSE 0.9389
Char 2-grams + RF 0.8610
BoW + LR 0.8490
BERT + LSTM 0.6940
Worst 0.5333
Confusion matrix
19
Author
Profiling
PAN’22
20
Author
Profiling
PAN’22
Corpus analysis
Does these high results mean that the corpus is biased?
We have conducted a deep analysis of the corpus from five different angles:
- Topics used by ironic and non-ironic users.
- Twitter elements usage.
- Language style: categorical vs. narrative.
- Emotionality: activation, imaginary, pleasantness.
- Personality type and communication style of the users.
21
Author
Profiling
PAN’22
Topic-based analysis
Two-fold analysis:
- Determining the set of words that are highly polarized according to the indexes introduced in [Poletto et al., 2021]: Polarized
Wiredness Index (PWI) which takes into account how polarized the words are in each class in the corpus (irony and non-irony).
- Determining the set of unique words in each class and analysing how this vocabulary impacts the learning process.
- We identified 1,379 words
which only appear in one
class.
- In the irony class, we found
334 unique terms, whereas,
in the class non-irony, we
identified 1,045 unique terms.
- We trained an SVM and RF classifiers considering as features the words in this vocabulary, obtaining respectively an
accuracy of 0.8817 and 0.8763.
Related to ethnic Related to politics and religion... and emojis
Highlights: there seems that there is a topic bias, but since there are also stylistic elements that vary between classes, this
bias may be inherent in the type of users per class.
22
Author
Profiling
PAN’22
Twitter elements analysis
Ironic people write
shorter tweets, use
more hashtags, more
mentions, and fewer
URLs than non-ironic
ones.
For all of them, the Mann-Whitney test is
significant in both sets (training and test);
p<.001
23
Author
Profiling
PAN’22
Language style analysis
POS tagging with Freeling, and calculation of the
Categorical (+) vs. Narrative (-) style inspired in
[Nisbett et al., 2001] and composed as:
NOUN + ADJ + PREP - VERB - ADV - PERSPRON
- Categorical style is used to express ideas and
concepts, whereas narrative is used to tell stories.
Highlights:
- Non-ironic users use significantly more a
categorical style, while ironic users utilise more a
narrative style (Mdn=-0.99; U=15,834; p<.001).
- Language style is topic agnostic: regardless of the
topic, there are significant differences in the way
ironic and non-ironic users employ language.
24
Author
Profiling
PAN’22
Emotions analysis
The new Dictionary of Affect in Language
(DAL) [Whissell, 1989]:
- List of 8,742 words.
- Annotated by 200 naïve volunteers.
- Three dimensions: activation (active vs.
passive), imaginary (easy vs. difficult to
imagine), and pleasantness (unpleasant vs.
pleasant).
Highlights:
- Non-ironic users present higher scores in
the three emotional dimensions than ironic
ones.
For all of them, the Mann-Whitney test is significant in both sets (training
and test); p<.001
25
Author
Profiling
PAN’22
Communication styles
*https://rapidapi.com/collection/symanto-symanto-default-apis
Symanto API* to obtain the users’ personality type
and communication styles [Stajner et al., 101]:
- The personality type refers to the way the person
behaves in a specific interaction from the
emotional vs rational point of view.
- Action-seeking, defined as direct or indirect
requests, suggestions, and recommendations that
expect action from other people.
- Fact-oriented, where the user utilises factual and
objective statements.
- Self-revealing, when the users share personal
information or experiences.
- Information-seeking, defined as direct or indirect
questions searching for information.
Highlights:
- Ironic and non-ironic users do not differ in
action-seeking but the do in the other four styles.
- Ironic users use less the fact-oriented
style and more a self-revealing and
information-seeking style.
- Non-ironic users are more emotional and less
rational than ironic ones.
For all of them, the Mann-Whitney test is significant p<.001
Subtask
The aim of the Profiling Stereotype
stance on Ironic Authors subtask is to
detect whether ironic users employed
stereotypes to hurt the target or to
somehow defend it.
26
Author
Profiling
Language:
English
PAN’22
2022 -> Profiling
Irony and
Stereotypes
spreaders
27
Author
Profiling
PAN’2
Examples
Corpus
28
Author
Profiling
PAN’22
IN FAVOUR AGAINST Total
Training 46 94 140
Test 12 48 60
Total 58 142 200
Methodology
1. We selected those authors that were annotated as ironic and spreaders of stereotypes.
2. For each author, only the tweets marked as ironic and using stereotypes were annotated
with their stance.
3. We asked the annotators to rely on their own perspectives on whether the tweets are in
favour or against the mentioned social category, with no other guidance.
4. The overall stance of an author corresponds to the majority class at tweet level.
5. The IAA between the first two annotators was 0.645.
6. A third annotator sorted out disagreements.
For each user, we provided 200 tweets
Evaluation measures
29
Author
Profiling
PAN’22
The performance of the systems is evaluated using the
macro averaged F1 measure (F_Macro).
We also analyse the F1-measure per class to study more
in depth the behaviour of the systems.
Baselines
30
Author
Profiling
PAN’22
CHAR 3-GRAMS + RF Trigrams with Random Forest
WORD 2-GRAMS + SVM Bigrams with Support Vector Machine
Symanto (LDSE) This method represents documents on the basis of the probability
distribution of occurrence of their words in the different classes. The key
concept of LDSE is a weight, representing the probability of a term to
belong to one of the different categories: irony spreader / non-spreader. The
distribution of weights for a given document should be closer to the weights
of its corresponding category. LDSE takes advantage of the whole
vocabulary
Results
31
Author
Profiling
PAN’22
Methods:
- dirazuherfa: Emotion-based approach
combining structural-, sentiment-, and
emotion-based features.
- toshevska: Deep graph convolutional
neural network.
- JoseAGD: UMUTextStats + FastText +
BERT + RoBERTa & a fully-connected
network.
- tamayo: RoBERTa + KNN to prototype
creation.
- AmitDasRup: BERT + TFIDF & LR.
- taunk: TFIDF BoW + trad. ML methods.
- fernanda: n-grams combination + voting.
Results
32
Author
Profiling
PAN’22
Highlights:
- Low performance in the "in-favour" class,
whereas high performance in the "against"
class.
- Three main difficulties:
i) The inherent complexity of profiling the
stance of ironic authors that employ
stereotypes.
ii) the short size of the corpus.
iii) the imbalance between “in-favour” and
“against” classes which made challenging
the learning process.
Subtask take away
33
Author
Profiling
PAN’22
This task opens a new way to study
ironic language to perpetuate stereotypes
and constitutes a starting point for
profiling authors who frame aggressiveness, toxicity and
messages of hatred towards social categories
such as immigrants, women and the LGTB+ community,
using an implicit way to convey
hate speech employing stereotypes.
Conclusions
● Several approaches to tackle the task:
○ Transformers (BERT-based), also combined with traditional representations and methods,
obtained the highest results.
● Results:
○ Over 89% on average.
○ Best (99.44%): Yu et al. - BERT + CNN
● Error analysis:
○ False positives (non-irony spreaders as spreaders): 15.45%
○ False negatives (irony spreaders as non-spreaders): 9.18%
34
Author
Profiling
PAN’22
Conclusions
One of the main challenges of this task was to contemplate the use of stereotypes in a broad
sense, that is, not focusing on a target group but considering those users who explain what
happens in their environment by intensively using social categories.
Behind this theoretical approach there is the idea that prejudice is fundamentally a vision of the world
that homogenizes people on the basis of their groups of origin or affiliation. A vision of the world that
considers that these group affiliations are the main cause of the people’s behaviours and could explain
social or economic problems.
It is evident that to embrace stereotyping towards many social groups may have introduced a topic bias,
although certainly when we analyse stereotypes towards a single group, the type of discourse changes
if what is held is a stereotypical view of a group (certain social categories are brought up in order to
present certain arguments). For example, gays are brought up in a moral discourse and immigrants are
evoked in an economic or legal discussion, then it is important to take into account this association
between target groups and topics.
● Corpus analysis illustrates that Ironic and non-Ironic users significantly differ not only in the use of
Twitter elements but also in the indices used to characterise language, use of emotions, and
communication styles, what could explain the high scores of the classifiers, and it opens the door
to future research in order to characterise better the use of irony.
35
Author
Profiling
PAN’22
Conclusions
Looking at the results, the corpus analysis and the error analysis, we can conclude:
● It is feasible to automatically discriminate between Irony and non-Irony Spreaders with high
precision
○ ...even when only textual features are used.
● Not only are the topics addressed by both types of users significantly different but also other
elements such as the number of emojis they use, the number of users they mention, the number of
hashtags they use, the number of URLs they share, their writing style, the emotions they convey or
even their personality and communication style.
● We have to bear in mind false positives since they are almost double than false negatives, and
misclassification might lead to ethical or legal implications.
36
Author
Profiling
PAN’22
37
Author
Profiling
PAN’22
Industry at PAN (Author Profiling)
38
Author
Profiling
Organisation
Sponsors
PAN’22
This year, the winners of the task are:
● Wentao Yu, Benedikt Boenninghoff, and
Dorothea Kolossa, Institute of Communication
Acoustics, Ruhr University Bochum, Germany
39
Author
Profiling
PAN’22
PAN'23: Profile cryptocurrency influencers in
social media from a low-resource perspective:
● Low-resource influencer profiling.
● Low-resource influencer interest identification.
● Low-resource influencer intent identification.
40
Author
Profiling
On behalf of the author profiling task organisers:
Thank you very much for participating
and hope to see you next year!!
PAN’22

More Related Content

Similar to Profiling Irony and Stereotype Spreaders on Twitter (IROSTEREO)

Tags as tools for social classification
Tags as tools for social classificationTags as tools for social classification
Tags as tools for social classificationIsabella Peters
 
Do characters abuse more than words?
Do characters abuse more than words?Do characters abuse more than words?
Do characters abuse more than words?Tharushi Ruwandika
 
A Hybrid Method of Long Short-Term Memory and AutoEncoder Architectures for S...
A Hybrid Method of Long Short-Term Memory and AutoEncoder Architectures for S...A Hybrid Method of Long Short-Term Memory and AutoEncoder Architectures for S...
A Hybrid Method of Long Short-Term Memory and AutoEncoder Architectures for S...AhmedAdilNafea
 
Application Of Linguistic Cues In The Analysis Of Language Of Hate Groups
Application Of Linguistic Cues In The Analysis Of Language Of Hate GroupsApplication Of Linguistic Cues In The Analysis Of Language Of Hate Groups
Application Of Linguistic Cues In The Analysis Of Language Of Hate GroupsLeonard Goudy
 
An Analytical Survey on Hate Speech Recognition through NLP and Deep Learning
An Analytical Survey on Hate Speech Recognition through NLP and Deep LearningAn Analytical Survey on Hate Speech Recognition through NLP and Deep Learning
An Analytical Survey on Hate Speech Recognition through NLP and Deep LearningIRJET Journal
 
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...kevig
 
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...ijnlc
 
Hate speech detection on Indonesian text using word embedding method-global v...
Hate speech detection on Indonesian text using word embedding method-global v...Hate speech detection on Indonesian text using word embedding method-global v...
Hate speech detection on Indonesian text using word embedding method-global v...IAESIJAI
 
Themes identification techniques in qualitative research
Themes identification techniques in qualitative researchThemes identification techniques in qualitative research
Themes identification techniques in qualitative researchGhulam Qambar
 
Linguistic Cues to Deception: Identifying Political Trolls on Social Media
Linguistic Cues to Deception: Identifying Political Trolls on Social MediaLinguistic Cues to Deception: Identifying Political Trolls on Social Media
Linguistic Cues to Deception: Identifying Political Trolls on Social MediaAseel Addawood
 
Discourse Analysis - Project Instructions
Discourse Analysis - Project InstructionsDiscourse Analysis - Project Instructions
Discourse Analysis - Project InstructionsAshwag Al Hamid
 
Metaphic or the art of looking another way.
Metaphic or the art of looking another way.Metaphic or the art of looking another way.
Metaphic or the art of looking another way.Suresh Manian
 
Proposal defense
Proposal defenseProposal defense
Proposal defensexiaojuzheng
 
Dr.saleem gul assignment summary
Dr.saleem gul assignment summaryDr.saleem gul assignment summary
Dr.saleem gul assignment summaryJaved Riza
 
From TeacherTo assist you with preparing the Week 7 assignment.docx
From TeacherTo assist you with preparing the Week 7 assignment.docxFrom TeacherTo assist you with preparing the Week 7 assignment.docx
From TeacherTo assist you with preparing the Week 7 assignment.docxhanneloremccaffery
 
Reflective Plan Examples
Reflective Plan ExamplesReflective Plan Examples
Reflective Plan ExamplesMonica Turner
 
Data Mining of Informational Stream in Social Networks
Data Mining of Informational Stream in Social Networks   Data Mining of Informational Stream in Social Networks
Data Mining of Informational Stream in Social Networks Bohdan Pavlyshenko
 

Similar to Profiling Irony and Stereotype Spreaders on Twitter (IROSTEREO) (20)

Tags as tools for social classification
Tags as tools for social classificationTags as tools for social classification
Tags as tools for social classification
 
Do characters abuse more than words?
Do characters abuse more than words?Do characters abuse more than words?
Do characters abuse more than words?
 
A Hybrid Method of Long Short-Term Memory and AutoEncoder Architectures for S...
A Hybrid Method of Long Short-Term Memory and AutoEncoder Architectures for S...A Hybrid Method of Long Short-Term Memory and AutoEncoder Architectures for S...
A Hybrid Method of Long Short-Term Memory and AutoEncoder Architectures for S...
 
Application Of Linguistic Cues In The Analysis Of Language Of Hate Groups
Application Of Linguistic Cues In The Analysis Of Language Of Hate GroupsApplication Of Linguistic Cues In The Analysis Of Language Of Hate Groups
Application Of Linguistic Cues In The Analysis Of Language Of Hate Groups
 
An Analytical Survey on Hate Speech Recognition through NLP and Deep Learning
An Analytical Survey on Hate Speech Recognition through NLP and Deep LearningAn Analytical Survey on Hate Speech Recognition through NLP and Deep Learning
An Analytical Survey on Hate Speech Recognition through NLP and Deep Learning
 
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...
 
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...
SARCASM AS A CONTRADICTION BETWEEN A TWEET AND ITS TEMPORAL FACTS: A PATTERNB...
 
Hate speech detection on Indonesian text using word embedding method-global v...
Hate speech detection on Indonesian text using word embedding method-global v...Hate speech detection on Indonesian text using word embedding method-global v...
Hate speech detection on Indonesian text using word embedding method-global v...
 
Themes identification techniques in qualitative research
Themes identification techniques in qualitative researchThemes identification techniques in qualitative research
Themes identification techniques in qualitative research
 
Linguistic Cues to Deception: Identifying Political Trolls on Social Media
Linguistic Cues to Deception: Identifying Political Trolls on Social MediaLinguistic Cues to Deception: Identifying Political Trolls on Social Media
Linguistic Cues to Deception: Identifying Political Trolls on Social Media
 
Discourse Analysis - Project Instructions
Discourse Analysis - Project InstructionsDiscourse Analysis - Project Instructions
Discourse Analysis - Project Instructions
 
Metaphic or the art of looking another way.
Metaphic or the art of looking another way.Metaphic or the art of looking another way.
Metaphic or the art of looking another way.
 
Ny3424442448
Ny3424442448Ny3424442448
Ny3424442448
 
Proposal defense
Proposal defenseProposal defense
Proposal defense
 
A Primer on High-Quality Identifier Naming
A Primer on High-Quality Identifier NamingA Primer on High-Quality Identifier Naming
A Primer on High-Quality Identifier Naming
 
Dr.saleem gul assignment summary
Dr.saleem gul assignment summaryDr.saleem gul assignment summary
Dr.saleem gul assignment summary
 
From TeacherTo assist you with preparing the Week 7 assignment.docx
From TeacherTo assist you with preparing the Week 7 assignment.docxFrom TeacherTo assist you with preparing the Week 7 assignment.docx
From TeacherTo assist you with preparing the Week 7 assignment.docx
 
Reflective Plan Examples
Reflective Plan ExamplesReflective Plan Examples
Reflective Plan Examples
 
1808.10245v1 (1).pdf
1808.10245v1 (1).pdf1808.10245v1 (1).pdf
1808.10245v1 (1).pdf
 
Data Mining of Informational Stream in Social Networks
Data Mining of Informational Stream in Social Networks   Data Mining of Informational Stream in Social Networks
Data Mining of Informational Stream in Social Networks
 

More from Francisco Manuel Rangel Pardo

Overview of the 8th Author Profiling task at PAN: Profiling Fake News Spreade...
Overview of the 8th Author Profiling task at PAN: Profiling Fake News Spreade...Overview of the 8th Author Profiling task at PAN: Profiling Fake News Spreade...
Overview of the 8th Author Profiling task at PAN: Profiling Fake News Spreade...Francisco Manuel Rangel Pardo
 
Overview of the 7th Author Profiling task at PAN: Bots and Gender Profiling ...
Overview of the 7th Author Profiling task at PAN: Bots and Gender Profiling  ...Overview of the 7th Author Profiling task at PAN: Bots and Gender Profiling  ...
Overview of the 7th Author Profiling task at PAN: Bots and Gender Profiling ...Francisco Manuel Rangel Pardo
 
AL4Trust - Artificial Intelligence for Building Trust 2019
AL4Trust - Artificial Intelligence for Building Trust 2019AL4Trust - Artificial Intelligence for Building Trust 2019
AL4Trust - Artificial Intelligence for Building Trust 2019Francisco Manuel Rangel Pardo
 
Author Profiling en Social Media. En la Academia... y en la Industria.
Author Profiling en Social Media. En la Academia... y en la Industria.Author Profiling en Social Media. En la Academia... y en la Industria.
Author Profiling en Social Media. En la Academia... y en la Industria.Francisco Manuel Rangel Pardo
 
Multimodal Stance Detection in Tweets on Catalan #1Oct Referendum @Ibereval 2...
Multimodal Stance Detection in Tweets on Catalan #1Oct Referendum @Ibereval 2...Multimodal Stance Detection in Tweets on Catalan #1Oct Referendum @Ibereval 2...
Multimodal Stance Detection in Tweets on Catalan #1Oct Referendum @Ibereval 2...Francisco Manuel Rangel Pardo
 
Overview of the 6th Author Profiling task at PAN: Multimodal Gender Identific...
Overview of the 6th Author Profiling task at PAN: Multimodal Gender Identific...Overview of the 6th Author Profiling task at PAN: Multimodal Gender Identific...
Overview of the 6th Author Profiling task at PAN: Multimodal Gender Identific...Francisco Manuel Rangel Pardo
 
RusProfiling Gender Identification in Russian Texts PAN@FIRE
RusProfiling Gender Identification in Russian Texts PAN@FIRERusProfiling Gender Identification in Russian Texts PAN@FIRE
RusProfiling Gender Identification in Russian Texts PAN@FIREFrancisco Manuel Rangel Pardo
 
Stance and Gender Detection in Tweets on Catalan Independence. Ibereval@SEPLN...
Stance and Gender Detection in Tweets on Catalan Independence. Ibereval@SEPLN...Stance and Gender Detection in Tweets on Catalan Independence. Ibereval@SEPLN...
Stance and Gender Detection in Tweets on Catalan Independence. Ibereval@SEPLN...Francisco Manuel Rangel Pardo
 
Gender and Language Variety Identification in Twitter. Overview of the 5th. A...
Gender and Language Variety Identification in Twitter. Overview of the 5th. A...Gender and Language Variety Identification in Twitter. Overview of the 5th. A...
Gender and Language Variety Identification in Twitter. Overview of the 5th. A...Francisco Manuel Rangel Pardo
 
Overview of the 4th. Author Profiling task at PAN-CLEF 2016
Overview of the 4th. Author Profiling task at PAN-CLEF 2016Overview of the 4th. Author Profiling task at PAN-CLEF 2016
Overview of the 4th. Author Profiling task at PAN-CLEF 2016Francisco Manuel Rangel Pardo
 
AL4Trust - Artificial Intelligence for Building Trust
AL4Trust - Artificial Intelligence for Building TrustAL4Trust - Artificial Intelligence for Building Trust
AL4Trust - Artificial Intelligence for Building TrustFrancisco Manuel Rangel Pardo
 
PR-SOCO Personality Recognition in SOurce COde (PAN@FIRE 2016)
PR-SOCO Personality Recognition in SOurce COde (PAN@FIRE 2016)PR-SOCO Personality Recognition in SOurce COde (PAN@FIRE 2016)
PR-SOCO Personality Recognition in SOurce COde (PAN@FIRE 2016)Francisco Manuel Rangel Pardo
 
Overview of PAN'16 - New challenges for Authorship Analysis: Cross-genre prof...
Overview of PAN'16 - New challenges for Authorship Analysis: Cross-genre prof...Overview of PAN'16 - New challenges for Authorship Analysis: Cross-genre prof...
Overview of PAN'16 - New challenges for Authorship Analysis: Cross-genre prof...Francisco Manuel Rangel Pardo
 
El Futuro de las Comunicaciones Personales a Través de los Dispositivos Móvil...
El Futuro de las Comunicaciones Personales a Través de los Dispositivos Móvil...El Futuro de las Comunicaciones Personales a Través de los Dispositivos Móvil...
El Futuro de las Comunicaciones Personales a Través de los Dispositivos Móvil...Francisco Manuel Rangel Pardo
 
A Low Dimensionality Representation for Language Variety Identification (CICL...
A Low Dimensionality Representation for Language Variety Identification (CICL...A Low Dimensionality Representation for Language Variety Identification (CICL...
A Low Dimensionality Representation for Language Variety Identification (CICL...Francisco Manuel Rangel Pardo
 
Language Variety Identification using Distributed Representations of Words an...
Language Variety Identification using Distributed Representations of Words an...Language Variety Identification using Distributed Representations of Words an...
Language Variety Identification using Distributed Representations of Words an...Francisco Manuel Rangel Pardo
 

More from Francisco Manuel Rangel Pardo (20)

Overview of the 8th Author Profiling task at PAN: Profiling Fake News Spreade...
Overview of the 8th Author Profiling task at PAN: Profiling Fake News Spreade...Overview of the 8th Author Profiling task at PAN: Profiling Fake News Spreade...
Overview of the 8th Author Profiling task at PAN: Profiling Fake News Spreade...
 
Overview of the 7th Author Profiling task at PAN: Bots and Gender Profiling ...
Overview of the 7th Author Profiling task at PAN: Bots and Gender Profiling  ...Overview of the 7th Author Profiling task at PAN: Bots and Gender Profiling  ...
Overview of the 7th Author Profiling task at PAN: Bots and Gender Profiling ...
 
AL4Trust - Artificial Intelligence for Building Trust 2019
AL4Trust - Artificial Intelligence for Building Trust 2019AL4Trust - Artificial Intelligence for Building Trust 2019
AL4Trust - Artificial Intelligence for Building Trust 2019
 
Author Profiling en Social Media. En la Academia... y en la Industria.
Author Profiling en Social Media. En la Academia... y en la Industria.Author Profiling en Social Media. En la Academia... y en la Industria.
Author Profiling en Social Media. En la Academia... y en la Industria.
 
Multimodal Stance Detection in Tweets on Catalan #1Oct Referendum @Ibereval 2...
Multimodal Stance Detection in Tweets on Catalan #1Oct Referendum @Ibereval 2...Multimodal Stance Detection in Tweets on Catalan #1Oct Referendum @Ibereval 2...
Multimodal Stance Detection in Tweets on Catalan #1Oct Referendum @Ibereval 2...
 
Overview of the 6th Author Profiling task at PAN: Multimodal Gender Identific...
Overview of the 6th Author Profiling task at PAN: Multimodal Gender Identific...Overview of the 6th Author Profiling task at PAN: Multimodal Gender Identific...
Overview of the 6th Author Profiling task at PAN: Multimodal Gender Identific...
 
RusProfiling Gender Identification in Russian Texts PAN@FIRE
RusProfiling Gender Identification in Russian Texts PAN@FIRERusProfiling Gender Identification in Russian Texts PAN@FIRE
RusProfiling Gender Identification in Russian Texts PAN@FIRE
 
Stance and Gender Detection in Tweets on Catalan Independence. Ibereval@SEPLN...
Stance and Gender Detection in Tweets on Catalan Independence. Ibereval@SEPLN...Stance and Gender Detection in Tweets on Catalan Independence. Ibereval@SEPLN...
Stance and Gender Detection in Tweets on Catalan Independence. Ibereval@SEPLN...
 
Gender and Language Variety Identification in Twitter. Overview of the 5th. A...
Gender and Language Variety Identification in Twitter. Overview of the 5th. A...Gender and Language Variety Identification in Twitter. Overview of the 5th. A...
Gender and Language Variety Identification in Twitter. Overview of the 5th. A...
 
Overview of the 4th. Author Profiling task at PAN-CLEF 2016
Overview of the 4th. Author Profiling task at PAN-CLEF 2016Overview of the 4th. Author Profiling task at PAN-CLEF 2016
Overview of the 4th. Author Profiling task at PAN-CLEF 2016
 
Redes sociales y preadolescentes
Redes sociales y preadolescentesRedes sociales y preadolescentes
Redes sociales y preadolescentes
 
AL4Trust - Artificial Intelligence for Building Trust
AL4Trust - Artificial Intelligence for Building TrustAL4Trust - Artificial Intelligence for Building Trust
AL4Trust - Artificial Intelligence for Building Trust
 
PR-SOCO Personality Recognition in SOurce COde (PAN@FIRE 2016)
PR-SOCO Personality Recognition in SOurce COde (PAN@FIRE 2016)PR-SOCO Personality Recognition in SOurce COde (PAN@FIRE 2016)
PR-SOCO Personality Recognition in SOurce COde (PAN@FIRE 2016)
 
Overview of PAN'16 - New challenges for Authorship Analysis: Cross-genre prof...
Overview of PAN'16 - New challenges for Authorship Analysis: Cross-genre prof...Overview of PAN'16 - New challenges for Authorship Analysis: Cross-genre prof...
Overview of PAN'16 - New challenges for Authorship Analysis: Cross-genre prof...
 
El Futuro de las Comunicaciones Personales a Través de los Dispositivos Móvil...
El Futuro de las Comunicaciones Personales a Través de los Dispositivos Móvil...El Futuro de las Comunicaciones Personales a Través de los Dispositivos Móvil...
El Futuro de las Comunicaciones Personales a Través de los Dispositivos Móvil...
 
Smart Listening - MUIinf
Smart Listening - MUIinfSmart Listening - MUIinf
Smart Listening - MUIinf
 
IA + Big Data = problema + oportunidad
IA + Big Data = problema + oportunidadIA + Big Data = problema + oportunidad
IA + Big Data = problema + oportunidad
 
A Low Dimensionality Representation for Language Variety Identification (CICL...
A Low Dimensionality Representation for Language Variety Identification (CICL...A Low Dimensionality Representation for Language Variety Identification (CICL...
A Low Dimensionality Representation for Language Variety Identification (CICL...
 
Language Variety Identification using Distributed Representations of Words an...
Language Variety Identification using Distributed Representations of Words an...Language Variety Identification using Distributed Representations of Words an...
Language Variety Identification using Distributed Representations of Words an...
 
Author Profiling task at PAN Lab at CLEF 2015
Author Profiling task at PAN Lab at CLEF 2015Author Profiling task at PAN Lab at CLEF 2015
Author Profiling task at PAN Lab at CLEF 2015
 

Recently uploaded

INTERNSHIP ON PURBASHA COMPOSITE TEX LTD
INTERNSHIP ON PURBASHA COMPOSITE TEX LTDINTERNSHIP ON PURBASHA COMPOSITE TEX LTD
INTERNSHIP ON PURBASHA COMPOSITE TEX LTDRafezzaman
 
GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]📊 Markus Baersch
 
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degreeyuu sss
 
Business Analytics using Microsoft Excel
Business Analytics using Microsoft ExcelBusiness Analytics using Microsoft Excel
Business Analytics using Microsoft Excelysmaelreyes
 
Heart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis ProjectHeart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis ProjectBoston Institute of Analytics
 
ASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel CanterASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel Cantervoginip
 
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024thyngster
 
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...Boston Institute of Analytics
 
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一F La
 
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfPredicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfBoston Institute of Analytics
 
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Thomas Poetter
 
IMA MSN - Medical Students Network (2).pptx
IMA MSN - Medical Students Network (2).pptxIMA MSN - Medical Students Network (2).pptx
IMA MSN - Medical Students Network (2).pptxdolaknnilon
 
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxaleedritatuxx
 
毕业文凭制作#回国入职#diploma#degree美国加州州立大学北岭分校毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#de...
毕业文凭制作#回国入职#diploma#degree美国加州州立大学北岭分校毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#de...毕业文凭制作#回国入职#diploma#degree美国加州州立大学北岭分校毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#de...
毕业文凭制作#回国入职#diploma#degree美国加州州立大学北岭分校毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#de...ttt fff
 
1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样vhwb25kk
 
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改yuu sss
 
DBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfDBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfJohn Sterrett
 
原版1:1定制南十字星大学毕业证(SCU毕业证)#文凭成绩单#真实留信学历认证永久存档
原版1:1定制南十字星大学毕业证(SCU毕业证)#文凭成绩单#真实留信学历认证永久存档原版1:1定制南十字星大学毕业证(SCU毕业证)#文凭成绩单#真实留信学历认证永久存档
原版1:1定制南十字星大学毕业证(SCU毕业证)#文凭成绩单#真实留信学历认证永久存档208367051
 
Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Colleen Farrelly
 
MK KOMUNIKASI DATA (TI)komdat komdat.docx
MK KOMUNIKASI DATA (TI)komdat komdat.docxMK KOMUNIKASI DATA (TI)komdat komdat.docx
MK KOMUNIKASI DATA (TI)komdat komdat.docxUnduhUnggah1
 

Recently uploaded (20)

INTERNSHIP ON PURBASHA COMPOSITE TEX LTD
INTERNSHIP ON PURBASHA COMPOSITE TEX LTDINTERNSHIP ON PURBASHA COMPOSITE TEX LTD
INTERNSHIP ON PURBASHA COMPOSITE TEX LTD
 
GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]GA4 Without Cookies [Measure Camp AMS]
GA4 Without Cookies [Measure Camp AMS]
 
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
毕业文凭制作#回国入职#diploma#degree澳洲中央昆士兰大学毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#degree
 
Business Analytics using Microsoft Excel
Business Analytics using Microsoft ExcelBusiness Analytics using Microsoft Excel
Business Analytics using Microsoft Excel
 
Heart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis ProjectHeart Disease Classification Report: A Data Analysis Project
Heart Disease Classification Report: A Data Analysis Project
 
ASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel CanterASML's Taxonomy Adventure by Daniel Canter
ASML's Taxonomy Adventure by Daniel Canter
 
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
Consent & Privacy Signals on Google *Pixels* - MeasureCamp Amsterdam 2024
 
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
NLP Data Science Project Presentation:Predicting Heart Disease with NLP Data ...
 
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
办理(Vancouver毕业证书)加拿大温哥华岛大学毕业证成绩单原版一比一
 
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdfPredicting Salary Using Data Science: A Comprehensive Analysis.pdf
Predicting Salary Using Data Science: A Comprehensive Analysis.pdf
 
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
Minimizing AI Hallucinations/Confabulations and the Path towards AGI with Exa...
 
IMA MSN - Medical Students Network (2).pptx
IMA MSN - Medical Students Network (2).pptxIMA MSN - Medical Students Network (2).pptx
IMA MSN - Medical Students Network (2).pptx
 
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptxmodul pembelajaran robotic Workshop _ by Slidesgo.pptx
modul pembelajaran robotic Workshop _ by Slidesgo.pptx
 
毕业文凭制作#回国入职#diploma#degree美国加州州立大学北岭分校毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#de...
毕业文凭制作#回国入职#diploma#degree美国加州州立大学北岭分校毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#de...毕业文凭制作#回国入职#diploma#degree美国加州州立大学北岭分校毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#de...
毕业文凭制作#回国入职#diploma#degree美国加州州立大学北岭分校毕业证成绩单pdf电子版制作修改#毕业文凭制作#回国入职#diploma#de...
 
1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
1:1定制(UQ毕业证)昆士兰大学毕业证成绩单修改留信学历认证原版一模一样
 
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改
专业一比一美国俄亥俄大学毕业证成绩单pdf电子版制作修改
 
DBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdfDBA Basics: Getting Started with Performance Tuning.pdf
DBA Basics: Getting Started with Performance Tuning.pdf
 
原版1:1定制南十字星大学毕业证(SCU毕业证)#文凭成绩单#真实留信学历认证永久存档
原版1:1定制南十字星大学毕业证(SCU毕业证)#文凭成绩单#真实留信学历认证永久存档原版1:1定制南十字星大学毕业证(SCU毕业证)#文凭成绩单#真实留信学历认证永久存档
原版1:1定制南十字星大学毕业证(SCU毕业证)#文凭成绩单#真实留信学历认证永久存档
 
Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024Generative AI for Social Good at Open Data Science East 2024
Generative AI for Social Good at Open Data Science East 2024
 
MK KOMUNIKASI DATA (TI)komdat komdat.docx
MK KOMUNIKASI DATA (TI)komdat komdat.docxMK KOMUNIKASI DATA (TI)komdat komdat.docx
MK KOMUNIKASI DATA (TI)komdat komdat.docx
 

Profiling Irony and Stereotype Spreaders on Twitter (IROSTEREO)

  • 1. 10th Author Profiling task at PAN Profiling Irony and Stereotype Spreaders on Twitter PAN-AP-2022 CLEF 2022 Bologna, 5-8 September Reynier Ortega Bueno - Universitat Politècnica de València Berta Chulvi - Universitat Politècnica de València Francisco Rangel - Symanto Research Paolo Rosso - Universitat Politècnica de València Elisabetta Fersini - Università Degli Studi di Milano-Bicocca
  • 2. Introduction Author profiling aims at identifying personal traits such as age, gender, personality traits, native language, language variety… from writings? This is crucial for: - Marketing. - Security. - Forensics. 2 Author Profiling PAN’22
  • 3. Last years goal Profiling Harmful Information Spreaders: 2019 - Profiling Bots 2020 - Profiling Fake News Spreaders 2021 - Profiling Hate Speech Spreaders 2022 - Profiling Irony and Stereotypes Spreaders 3 Author Profiling PAN’22
  • 4. Task goal Given a Twitter feed, determine whether its author is keen to spread irony or not, with special focus on users who use irony towards stereotypes. 4 Author Profiling Language: English PAN’22
  • 5. Corpus 5 Author Profiling PAN’22 Methodology 1. Defining Taxonomy and Stereotypes Categories: a. We examine the "target minority" field of the SBIC [Sap et al., 2020] b. We identify 600 unique labels that could be considered a social category in SBIC, and classify them into 17 categories: (1) national majority groups; (2) illness/health groups; (3) age and role family groups; (4) victims; (5) political groups; (6) ethnic/racial minorities; (7) immigration/national minorities; (8) professional and class groups; (9) sexual orientation groups; (10) women; (11) physical appearance groups; (12) religious groups; (13) style of life groups; (14) non-normative behaviour groups; (15) man/male groups; (16) minorities expressed in generic terms; (17) white people 2. Tweet Retrieval a. We retrieve tweets containing at least one of the labels included in categories 5 to 14 of the taxonomy (with and without the hashtags irony and sarcasm). b. We select users with more tweets accomplishing previous conditions.
  • 6. Corpus 6 Author Profiling PAN’22 Methodology 3. Annotation Process a. Irony: annotators were asked to mark the tweets where the user "express the opposite of what was saying as a disguised mockery". If the user had more than 5 ironic tweets, it was labelled as ironic. b. Stereotypes: annotators were asked to check if the social categories in the tweets were used to refer to a social group by associating them with a homogenising image of the category (e.g., as if all gays or Muslims were the same and could be described with that word). If the user had more than 5 tweets with stereotypes, it was labelled accordingly. 4. Corpus Construction a. Two independent annotators labelled the data (IAA 0.7093). b. A third annotator sorted out disagreements. c. For each user, we provide 200 tweets.
  • 7. Corpus 7 Author Profiling PAN’22 IRONIC NON-IRONIC Total Stereotypes Non-stereotypes Total Stereotypes Non-stereotypes Total Training 140 70 210 140 70 210 420 Test 60 30 90 60 30 90 180 Total 200 100 300 200 100 300 600 Statistics Number of users per class and set. Each user contains 200 tweets.
  • 8. Evaluation measures 8 Author Profiling PAN’22 Since the dataset is completely balanced for the two target classes, ironic vs. non-ironic, we have used the accuracy measure and ranked the performance of the systems by that metric.
  • 9. Baselines 9 Author Profiling PAN’22 CHAR 2-GRAMS + RF Bigrams with Random Forest WORD 1-GRAMS + LR Bag of words with Logistic Regression BERT + LSTM We represent each tweet in the profile utilising pretrained Bert-base model. Later, we fed an LSTM with these vectors as input. Symanto (LDSE) This method represents documents on the basis of the probability distribution of occurrence of their words in the different classes. The key concept of LDSE is a weight, representing the probability of a term to belong to one of the different categories: irony spreader / non-spreader. The distribution of weights for a given document should be closer to the weights of its corresponding category. LDSE takes advantage of the whole vocabulary
  • 10. 65 participants 32 working notes 12 countries 10 Author Profiling PAN’22 Participation https://mapchart.net/world.html
  • 12. Approaches - Preprocessing 12 Author Profiling Twitter elements (RT, VIA, FAV) Giglou et al.; Cao et al.; Lin et al.; Jang; Xu et al.; Wang & Ning; Zhang & NIng; Sagar & Varma; Hazrati et al. Emojis and other non-alphanumeric chars Wang & Ning; Zhang & Ning; Hazrati et al.; Butt et al.; Jian & Huang Lemmatisation Haolong & Sun Punctuation signs Giglou et al.; Lin et al.; Xu & Ning; Wang & Ning; Hazrati et al.; Dong et al. Numbers Xu & Ning; Hazrati et al.; Butt et al. Lowercase Giglou et al.; Xu & Ning; Butt et al.; Dong et al.; Siino & Cascia; Mangione & Garbo; Zengyao & Zhongyuan; Croce et al.; Herold & Castro Stopwords Giglou et al.; Hazrati et al.; Butt et al. Infrequent terms Giglou et al.; Wang & Ning; Zengyao & Zhongyuan X2 with PMI and TF/IDF Ikae I/NI labels to the end of the tweets Jian & Huang GloVe to filter out features Haolong & Sun PAN’22
  • 13. Approaches - Features 13 Author Profiling Stylistic features: - Vocabulary size - Number of tokens - Tweet length - Number of hashtags, mentions, URLS - Number of emojis - ... Nikolova et al. N-gram models Sagar & Varma; Butt et al.; Herold & Castro; Ikae; Nikolova et al. Emotional and personality features Butt et al. TextVectorizer Croce et al. Embeddings Dong et al.; Yang et al. Transformers ...BERT Cao et al.; Lin et al.; Xu & Ning; 71; Hazrati et al.; Jian & Huang; Zengyao & Zhongyuan; Wentao & Kolossa; Rodriguez & Barroso ...SBERT Tahaei et al. ...BERTTweet Wang & Ning PAN’22
  • 14. Approaches - Features 14 Author Profiling Transformers + others ...BERT & Twitter RoBERTa + LM HateXPlain Mathew et al. SBERT + emojis Tahaei et al. SBERT + psychometrics, emotions and irony Tavan et al. BERT + TF-IDF n-grams Das et al.; Gómez & Parres SBERT + graph-based & one-hot embeddings Giglou et al. + Ironic-, contextual-, psychometric-related features fine-tuned with datasets annotated with sentiment/emotions from Kaggle Tavan et al. Sentence Transformers + n-grams + stylistic Jang BERT & RoBERTa + FastText + stylistic Díaz et al. PAN’22
  • 15. Approaches - Features 15 Author Profiling PAN’22 CNN Díaz et al. CNN + TF-IDF 1-grams + BiGRU Haolong & Sun Sequence probabilities + n-grams + GPT2 & DistilGPT2 Huang Irony identification at tweet level by combining: 1) structural features 2) sentiment words 3) fine-grained emotions by means of several lexicons Hernández & Montes
  • 16. Approaches - Methods 16 Author Profiling Logistic regression Butt et al.; Das et al. Random Forest Sagar & Varma; Ikae; Nikolova et al.; Hernández & Montes Bayes Huang Multilayer Perceptron Hazrati et al. Gradient Booster Classifier Tavan et al. k-Nearest Neighbours Rodriguez & Barroso Ensembles (trad. classifiers) Zengyao & Zhongyuan; Herold & Castro; Tavan et al. Ensembles (trad. Class + DL) Siino & Cascia; Croce et al. PAN’22
  • 17. Approaches - Methods 17 Author Profiling Linear Feed Forward Networks Tahaei et al. CNN Dong et al.; Mangione & Garbo; Wentao & Kolossa GCNN Giglou et al. bi-LSTM + CNN Yang et al. Fully-connected networks Haolong & Sun; Labadie & Castro; Díaz et al. AutoKeras; AutoGluon Wang & Ning; Xu & Ning; Zhang & Ning BERT + Decision Rules Jian & Huang BERT + SVM, MLP, Gaussian NB & RF Rodriguez & Barroso BERT + CNN, LSTM, att. layer Gómez & Parres BERT & DistilBERT + RF & SVM Jang BERT + voting classifier Cao et al.; Lin et al. PAN’22
  • 18. Global ranking 18 Author Profiling PAN’22 Team Acc Best 0.9944 LDSE 0.9389 Char 2-grams + RF 0.8610 BoW + LR 0.8490 BERT + LSTM 0.6940 Worst 0.5333
  • 20. 20 Author Profiling PAN’22 Corpus analysis Does these high results mean that the corpus is biased? We have conducted a deep analysis of the corpus from five different angles: - Topics used by ironic and non-ironic users. - Twitter elements usage. - Language style: categorical vs. narrative. - Emotionality: activation, imaginary, pleasantness. - Personality type and communication style of the users.
  • 21. 21 Author Profiling PAN’22 Topic-based analysis Two-fold analysis: - Determining the set of words that are highly polarized according to the indexes introduced in [Poletto et al., 2021]: Polarized Wiredness Index (PWI) which takes into account how polarized the words are in each class in the corpus (irony and non-irony). - Determining the set of unique words in each class and analysing how this vocabulary impacts the learning process. - We identified 1,379 words which only appear in one class. - In the irony class, we found 334 unique terms, whereas, in the class non-irony, we identified 1,045 unique terms. - We trained an SVM and RF classifiers considering as features the words in this vocabulary, obtaining respectively an accuracy of 0.8817 and 0.8763. Related to ethnic Related to politics and religion... and emojis Highlights: there seems that there is a topic bias, but since there are also stylistic elements that vary between classes, this bias may be inherent in the type of users per class.
  • 22. 22 Author Profiling PAN’22 Twitter elements analysis Ironic people write shorter tweets, use more hashtags, more mentions, and fewer URLs than non-ironic ones. For all of them, the Mann-Whitney test is significant in both sets (training and test); p<.001
  • 23. 23 Author Profiling PAN’22 Language style analysis POS tagging with Freeling, and calculation of the Categorical (+) vs. Narrative (-) style inspired in [Nisbett et al., 2001] and composed as: NOUN + ADJ + PREP - VERB - ADV - PERSPRON - Categorical style is used to express ideas and concepts, whereas narrative is used to tell stories. Highlights: - Non-ironic users use significantly more a categorical style, while ironic users utilise more a narrative style (Mdn=-0.99; U=15,834; p<.001). - Language style is topic agnostic: regardless of the topic, there are significant differences in the way ironic and non-ironic users employ language.
  • 24. 24 Author Profiling PAN’22 Emotions analysis The new Dictionary of Affect in Language (DAL) [Whissell, 1989]: - List of 8,742 words. - Annotated by 200 naïve volunteers. - Three dimensions: activation (active vs. passive), imaginary (easy vs. difficult to imagine), and pleasantness (unpleasant vs. pleasant). Highlights: - Non-ironic users present higher scores in the three emotional dimensions than ironic ones. For all of them, the Mann-Whitney test is significant in both sets (training and test); p<.001
  • 25. 25 Author Profiling PAN’22 Communication styles *https://rapidapi.com/collection/symanto-symanto-default-apis Symanto API* to obtain the users’ personality type and communication styles [Stajner et al., 101]: - The personality type refers to the way the person behaves in a specific interaction from the emotional vs rational point of view. - Action-seeking, defined as direct or indirect requests, suggestions, and recommendations that expect action from other people. - Fact-oriented, where the user utilises factual and objective statements. - Self-revealing, when the users share personal information or experiences. - Information-seeking, defined as direct or indirect questions searching for information. Highlights: - Ironic and non-ironic users do not differ in action-seeking but the do in the other four styles. - Ironic users use less the fact-oriented style and more a self-revealing and information-seeking style. - Non-ironic users are more emotional and less rational than ironic ones. For all of them, the Mann-Whitney test is significant p<.001
  • 26. Subtask The aim of the Profiling Stereotype stance on Ironic Authors subtask is to detect whether ironic users employed stereotypes to hurt the target or to somehow defend it. 26 Author Profiling Language: English PAN’22
  • 27. 2022 -> Profiling Irony and Stereotypes spreaders 27 Author Profiling PAN’2 Examples
  • 28. Corpus 28 Author Profiling PAN’22 IN FAVOUR AGAINST Total Training 46 94 140 Test 12 48 60 Total 58 142 200 Methodology 1. We selected those authors that were annotated as ironic and spreaders of stereotypes. 2. For each author, only the tweets marked as ironic and using stereotypes were annotated with their stance. 3. We asked the annotators to rely on their own perspectives on whether the tweets are in favour or against the mentioned social category, with no other guidance. 4. The overall stance of an author corresponds to the majority class at tweet level. 5. The IAA between the first two annotators was 0.645. 6. A third annotator sorted out disagreements. For each user, we provided 200 tweets
  • 29. Evaluation measures 29 Author Profiling PAN’22 The performance of the systems is evaluated using the macro averaged F1 measure (F_Macro). We also analyse the F1-measure per class to study more in depth the behaviour of the systems.
  • 30. Baselines 30 Author Profiling PAN’22 CHAR 3-GRAMS + RF Trigrams with Random Forest WORD 2-GRAMS + SVM Bigrams with Support Vector Machine Symanto (LDSE) This method represents documents on the basis of the probability distribution of occurrence of their words in the different classes. The key concept of LDSE is a weight, representing the probability of a term to belong to one of the different categories: irony spreader / non-spreader. The distribution of weights for a given document should be closer to the weights of its corresponding category. LDSE takes advantage of the whole vocabulary
  • 31. Results 31 Author Profiling PAN’22 Methods: - dirazuherfa: Emotion-based approach combining structural-, sentiment-, and emotion-based features. - toshevska: Deep graph convolutional neural network. - JoseAGD: UMUTextStats + FastText + BERT + RoBERTa & a fully-connected network. - tamayo: RoBERTa + KNN to prototype creation. - AmitDasRup: BERT + TFIDF & LR. - taunk: TFIDF BoW + trad. ML methods. - fernanda: n-grams combination + voting.
  • 32. Results 32 Author Profiling PAN’22 Highlights: - Low performance in the "in-favour" class, whereas high performance in the "against" class. - Three main difficulties: i) The inherent complexity of profiling the stance of ironic authors that employ stereotypes. ii) the short size of the corpus. iii) the imbalance between “in-favour” and “against” classes which made challenging the learning process.
  • 33. Subtask take away 33 Author Profiling PAN’22 This task opens a new way to study ironic language to perpetuate stereotypes and constitutes a starting point for profiling authors who frame aggressiveness, toxicity and messages of hatred towards social categories such as immigrants, women and the LGTB+ community, using an implicit way to convey hate speech employing stereotypes.
  • 34. Conclusions ● Several approaches to tackle the task: ○ Transformers (BERT-based), also combined with traditional representations and methods, obtained the highest results. ● Results: ○ Over 89% on average. ○ Best (99.44%): Yu et al. - BERT + CNN ● Error analysis: ○ False positives (non-irony spreaders as spreaders): 15.45% ○ False negatives (irony spreaders as non-spreaders): 9.18% 34 Author Profiling PAN’22
  • 35. Conclusions One of the main challenges of this task was to contemplate the use of stereotypes in a broad sense, that is, not focusing on a target group but considering those users who explain what happens in their environment by intensively using social categories. Behind this theoretical approach there is the idea that prejudice is fundamentally a vision of the world that homogenizes people on the basis of their groups of origin or affiliation. A vision of the world that considers that these group affiliations are the main cause of the people’s behaviours and could explain social or economic problems. It is evident that to embrace stereotyping towards many social groups may have introduced a topic bias, although certainly when we analyse stereotypes towards a single group, the type of discourse changes if what is held is a stereotypical view of a group (certain social categories are brought up in order to present certain arguments). For example, gays are brought up in a moral discourse and immigrants are evoked in an economic or legal discussion, then it is important to take into account this association between target groups and topics. ● Corpus analysis illustrates that Ironic and non-Ironic users significantly differ not only in the use of Twitter elements but also in the indices used to characterise language, use of emotions, and communication styles, what could explain the high scores of the classifiers, and it opens the door to future research in order to characterise better the use of irony. 35 Author Profiling PAN’22
  • 36. Conclusions Looking at the results, the corpus analysis and the error analysis, we can conclude: ● It is feasible to automatically discriminate between Irony and non-Irony Spreaders with high precision ○ ...even when only textual features are used. ● Not only are the topics addressed by both types of users significantly different but also other elements such as the number of emojis they use, the number of users they mention, the number of hashtags they use, the number of URLs they share, their writing style, the emotions they convey or even their personality and communication style. ● We have to bear in mind false positives since they are almost double than false negatives, and misclassification might lead to ethical or legal implications. 36 Author Profiling PAN’22
  • 38. Industry at PAN (Author Profiling) 38 Author Profiling Organisation Sponsors PAN’22 This year, the winners of the task are: ● Wentao Yu, Benedikt Boenninghoff, and Dorothea Kolossa, Institute of Communication Acoustics, Ruhr University Bochum, Germany
  • 39. 39 Author Profiling PAN’22 PAN'23: Profile cryptocurrency influencers in social media from a low-resource perspective: ● Low-resource influencer profiling. ● Low-resource influencer interest identification. ● Low-resource influencer intent identification.
  • 40. 40 Author Profiling On behalf of the author profiling task organisers: Thank you very much for participating and hope to see you next year!! PAN’22