All authors contributed equally.An Analysis of Categoric

* All authors contributed equally.
An Analysis of Categorical Biases in Word
Embeddings
Abstract— This work focuses on discovering forms of bias in
word embeddings through the use of the Word Embedding
Association Test (WEAT). A thorough categorical analysis is
performed to distinguish among different forms of biases, to
discover the extent and importance of it in different domains
and also uncover biases that are commonly not expected.
Moreover, a model specifically designed and trained to mitigate
bias in genders is also tested to see how much it can mitigate
the
effect of embedded bias in training corpora. Finally, to discover
how those biases came to be, we perform an analysis on a
historic dataset which can illustrate how biases evolved and
whether there is a consistency in their current estimated form.
I. INTRODUCTION

Word Embeddings trained on Natural Language Processing
(NLP) and machine learning models are vulnerable to
rampant biases. These biases maybe desirable or undesirable.
Significant work has been done to develop de-biasing models
for word embeddings. But, making a model absolutely bias
free can be difficult. These biases are mostly generated due
lack of representational data, or as a direct influence of our in
bred societal biases that get incorporated in the data sets that
are used to train these models on.
With the advent of Artificial Intelligence and technologies
built on it, word embeddings find major real-world
application. For example; recommendation systems on e-
commerce websites find use of word embeddings to suggest
search-keywords or merchandises to its users. Websites for
professional networking can generate biased output too. If a
recruiter looks for suitable candidates for a position, biased
output can be generated based on the candidate’s gender,
location etc. If the recruiter enters the search keyword

“programmer”, the algorithm may tend to return resumes of
male candidates with higher priority, as dictated by the word
embeddings which associates the profession, “programmer”
more with the male gender. This is an undesirable bias which
should be eliminated from the model to make the application
fair. Again, when the results returned tend to be biased
towards candidates with residence in a particular location, to
some extent they might be desirable biases. For example;
when a company which is searching for candidates to fill a
position in London, if the results returned give low priority to
candidates from Tokyo, then it may not be completely
undesirable. This indicates, on top of eliminating undesirable
biases, algorithms must also be trained to identify and retain
the desirable biases.
Previous work done in this area have targeted biases like
gender bias, ethnic bias, temporal bias in historical data, etc.
The most used method is WEAT (Word Embedding
Association Test) to quantify and understand the extent of

biases in different word embedding models. Publicly
available datasets, like those from the areas of Journalism
(Google News data corpus), Social Media (Twitter data) as
well as Wikipedia data have been used for these analyses.
This paper will present a study on biases in word embedding
models by identifying sub-categories in areas like gender,
society, religion, racial and non-people categories. The biases
are measured against attribute sets like man vs woman, rich
vs poor, slow vs quick, etc. Four different word embedding
models trained have been used to measure the bias in each of
those target and attribute sets. The models used are: (a) the
micro-blogging site, Twitter’s data corpus trained on the
GloVe algorithm; (b) Google News data corpus trained on the
word2vec algorithm; (c) Wikipedia data corpus trained on the
GloVe algorithm and (d) Historical Data corpus from 1820 to
1990. Plots to visualize the biases will be constructed based
on the data from those derivations. The results of a de-biasing
algorithm run on the same dataset will be discussed along

with the causes of presence of such biases. To conclude, a
summary of future work required will be listed.
II. BACKGROUND AND RELATED WORK
This section provides a brief overview on word embedding,
their uses and applications. It also touches upon the previous
work done in this area and weighs in upon what are identified
as major problem areas. It concludes with a summary of w -
hat was aimed to be achieved in this paper and methodology
used.
At a higher level, word embedding can be described as vector
representation of words in a dictionary which is capable of
capturing the geometric distances between the words in it.
These distances can then be used to group together similar
words based on their proximity to one another. With the
Artificial Intelligence (AI) driven decision making tools
picking up pace in the real world, the biases in word
embedding models become more prominent. Prior work done

on biases in word embeddings show a wide variety of biases
pertaining to different categories. These categories may range
from gender, race and location to political concepts. These
works also reveal how the extent of bias in a category towards
an attribute can vary with the data corpus the word
embedding model has been trained on. For example; gender
bias in twitter data is lower than that observed in a word
embedding model trained on the Google News data corpus.
Such changes can also be temporal. This study [1] on models
trained on 100 years of historical data of the American society
reveals that, ethnic biases in word embeddings have changed
with respect to time. This may be attributed to changes in
societal constructs and diaspora of the country. We w ill talk
more about the causes of change in the analysis section of this
paper.
Significant work has been done to study and analyze gender
biases in word embedding models. In [2], the author studies
the prejudices of one gender against another in different

categories based on word embedding models trained on
different type of data sources. The data sets used are GAP,
Google News, Twitter and PubMed; and the categories
studied being Career vs Family, Science vs Arts, Math vs
Arts, Intelligence vs Appearance and Strength vs Weakness.
Different datasets showed different range of bias in those
categories- the Google News corpus showed bias in all the
categories unlike PubMed which showed relatively less bias
in all of the categories studied. Further, the author tries to
auto detect categories in word embeddings which display
gender biases based on the proximity of male- and female-
pronouns to other words in the dictionary. In [3], it is asserted
that, the use of unsupervised algorithms may lead to the
generation of bias when run on user generated or historical
data. It can lead to bias generation mimicking the bias in the
original documents later on, or, by identifying the biased
patterns in the original data as key concepts and them being
fundamental to the area. For example; as illustrated in [3], the

word Prime Minister was only associated with the male
gender prior to 1960. Word embeddings run on data till 1960
only associate the male gender with the Prime Minister as no
data is available for a woman holding the position till then. It
argues that the presence of genderless nouns in English can
be used to expose the correlation between he and she
stereotypes in data.
Reference [3] attempts to de-bias gender-neutral words by
eliminating the gender associations with them as a post
processing step. It is proposed two ways – (i) Hard de-
biasing, which involves manual intervention; (ii) Soft
Debiasing, where computer programs are responsible for de-
biasing sans any human intervention. According to the study,
hard de-biasing methods work better at this point in time as
they could not find an efficient enough debiasing algorithm.
However, they do not consider the possibility of valid gender
associations with non-gender-definitional (or gender neutral)
words, while de-biasing - like the association of beard with

men. To overcome this problem, [4]proposes a model that
retains the desirable gender biases whilst doing away with the
undesirable ones. It identifies four sets of words in a
dictionary – (i) masculine, (ii) feminine, (iii) neutral and (iv)
stereotypical. Based on this classification, it models an
interaction that can retain the masculinity of a target, retain
the femininity target, protect the gender neutrality in the
target and remove the gender biases.
Though the elimination of desirable biases in [3] can be
overcome by implementing the method laid out in [4], the
models are not widely trained and it is unknown how the
model might run on other categories like race, location, etc.
Both the models try to categorically identify the biases with
respect to the attributes (in these cases, gender). But there
may be other correlated unidentified attributes which may be
biased towards these categories and may lead to the
production of biased results.
This paper focuses on the analysis of categorical biases in

word embeddings. Instead of focusing on a particular
category or attribute set, tests have been run on a diverse set
of sub-categories from broader areas and the biases they are
subjected to. The biases generated when these targets are
pitted against different set of attributes have been studied. For
example; we have studied the social (category) bias
pertaining to one’s ethnicity (sub category) using the WEAT
algorithm. We checked the bias for Germans vs Italians
(target) for the attribute set of lazy vs hardworking, which are
preconceived notions popularly associated with our target set.
We also study the variations in biases depending on the
training data sets for the word embeddings- we have used the
Google News data, Twitter data, Wikipedia data and
historical data from the 19th and 20th century. We will try to
reason the possible source of these biases at the discussion
section of this paper.
III. METHODOLOGY
a. Biased Models

The approach followed, attempts to answer the points
discussed in the introduction. We follow three main
directions and depending on the availability of data and code
packages, we perform a thorough analysis and plot the
results. The points we tackle can be summarized as follows:
• Biased models trained on multiple text corpora.
• Debiased model trained on text corpora to mitigate
bias.
• Historical development of biases through historic
corpora.
Figure 1: Three-part analysis followed
To recognize bias in word embedding models, first a selection
of available word embedding models on large corpora needs
to be done. For this purpose, three datasets are down-selected
to form the basis of our analysis.
The first dataset is Google News [5]. It is based on articles
from the same name website and consists of large size data.

It contains 100 billion words and totally 3 million different
trained words. The size of the word embeddings, meaning the
representation size of each word, is 300.
The second model used is that of Twitter which has 1.2
million different words and contains embedding sizes of 25,
50,100,200. [6]
The third model is that of Wikipedia which contains 400
thousand different words and embedding dimensions of
50,100,200 and 300.
Google News is trained based on the Word2Vec method
while the others on the Glove method.
Word2vec [7] is a two-layer neural net that takes as input text
and vectorizes it into decimal numbers. Specifically, it
vectorizes words. It is similar to an autoencoder in a sense
that it is trained against other words that neighbor them in the
input corpus. This can be done with two ways. One way is
through Continuous Bag of Words which uses context to
predict a target word.

The other way is through Skip-Gram which given the word
predicts the neighbor words. When the feature vector or else
embedding assigned to word does not predict accurately the
context, the vector is adjusted. An accurate model with proper
training will place similar words close to each other.
Figure 2: CBOW and Skip-Gram as methods for embedding
learning
GloVe [8] is a model trained on non-zero entries of word to
word co-occurences which recognizes how often words co-
occur in a corpus. It is essentially a log-bilinear model for
unsupervised learning with a weighted least squares
objective. Semantic similarity is learned between words in
this way.
The characteristics of the models are shown in table 1. The
trained models that were used, are of dimension 50 for twitter
and Wikipedia and dimension 300 for Google News, while
they can be found at the gensim library where they are open
sourced [9].

b. De-biased Models
In order to uncover whether a model specifically trained to be
debiased, can perform better than those who are not, a popular
model for tackling gender bias is presented.
Reference [3] addresses the problem of bias by defining a
subspace which identifies the direction of the embedding that
captures the bias.
Then the algorithm has two options, being to neutralize or
soften. By neutralizing, it ensures that gender neutral words
are zero in the gender subspace. On the other hand, softening,
perfectly equalizes words outside the subspace and makes
sure that the any neutral word is equidistant to all words in
each equality set.
As an example, given the words grandmother and grandfather
and the words guy and gal, namely two equality sets, after
softening, the word babysit would be equidistant to
grandmother and grandfather and also the same to gal and
guy. A parameter also can control how much similarity is

maintained to the original embedding since this can be useful
in some applications.
For the purposes of this analysis, a trained model is used
which can be found here [10]. This model has been trained
using a combination of words to tackle gender specifi c
embedding bias on the aforementioned Google News dataset.
Datasets used Characteristics
Google News about 100 billion words,
300-dimensional vectors for
3 million words and phrases
Twitter 2B tweets, 27B tokens,
1.2M vocab, uncased, 25d,
50d, 100d, & 200d vectors
Wikipedia 6B tokens, 400K vocab,
uncased, 50d, 100d, 200d, &
300d vectors
c. Historical Models

In order to understand the language evolution and analyze
single and mutual biases in word usage throughout a certain
period, the lexical data about frequencies of word
appearances in the corpora for different years should be used.
Nowadays, this kind of time-wise analysis has become
possible due to development and dramatic expansion of
Google Books N-gram corpora, which comprises large
collections of books and similar materials printed between
1500 and 2008 in 8 different languages (English, Chinese,
French, German, Hebrew, Italian, Russian, and Spanish),
with a total of over 5 million books. The dataset includes
information about frequency of usage of n-grams (where n is
2 or higher). Such extensive text corpora allow researchers
for solving a wide range of natural language processing, and
exploring word embedding bias analysis is not an exception.
The authors of [11] provide an extensive statistical analysis
of words to examine two proposed statistical laws: the law of
conformity and the law of innovation. In this work,

researchers use 6 historical datasets to analyze historical
change, which are essentially subsets of Google Books N-
gram corpora. The datasets then have been used to obtain
different groups of word embeddings by applying several
techniques, one of which is word2vec based one. Given the
sparsity of data between 1500 and 1800, it is recommended
to exploit lexical data from the materials published after
1800.
In our work, we perform the analysis over various groups of
words from diverse semantic groups which might have
represented an expression of bias. Hence, it makes sense to
use a set of word embeddings with a sufficiently multivariate
set of words. For this reason, we decided to use the words
from “All English” dataset, which includes data from Google
books of all genres published between 1800 and 1999, with a
total of 8.5*1011 tokens.
We focused our attention on word2vec embeddings
pretrained on this dataset (SGNS) kindly provided by the

authors of the work on the HistWords project GitHub page
[12]), which contains multiple tools and word embeddings.
SGNS dataset represents a group of files in a special format,
which is not compatible with gensim library, which is utilized
in our research. Thus, the following steps should be taken in
order to produce WEAT scores:
1. Word embedding files are to be converted into .txt
gensim-compatible format
2. Extract embeddings for words from our categories
and calculate mutual biases
3. Plot and discuss the results
d. Word Embedding Association Test
The Word Embedding Association Test is a statistical test
which aids in understanding the relation between words in
embedded in text corpora.
Considering that we have two sets of target words (X and Y)
and two sets of attribute words (A and B), we want to discover
if there is a difference between the former in terms of their

relative similarity to the latter.
Specifically, since this is a statistical test, we have the null
hypothesis that there is no relative difference between the
target words and the attribute words. We measure the
(un)likelihood of the null hypothesis by computing the
probability that a random permutation of attribute words
would produce the observed or greater difference in sample
mean.
The statistic of the test is derived by:
�(�, �, �, �) = ∑ �(�, �, �)
�∈ �
− ∑ �(�, �, �)
�∈ �
with:
�(�, �, �) = �� ∈ � cos(�, �) − �� ∈ � cos(�,
�)
where cos the cosine distance, s(w,A,B) measuring the
association of w with the attribute and s(X,Y,A,B) measuring

the differential association of target words and attributes.
The test being performed is a permutation test and if {(�� ,
�� )}
denotes all the partitions of � ∪ � into two sets of equal size,
the one-sided p-value of the permutation test is:
�� [�(�� , �� , �, �) > �(�, �, �, �)]
effect size =
��∈ ��(�,�,�)− ��∈ � s(�,�,�)
��−��∈ �∪ ��(�,�,�)
with the effect size being a normalized measure of the
separation of the two distributions of associations between
target and attribute.
By calculating the effect size of this statistical test, we follow
basically what is called “Cohen’s d” which calculates the
standard mean difference between two groups. An effect size
of 1 indicates that the groups differ by 1 standard deviation
and 2 indicates 2 standard deviations. Cohan has suggested
that d=0.2 is considered a “small” size, “0.5” medium and
above “0.8” large [13].
For performing the test, our code development was based on

a template to perform such analysis which can be found here
[14].
e. Analysis Procedure
The analysis performed aims primarily to measure and
compare the size of bias in word embedding in the
aforementioned models. In order to do that we focus on a
thorough categorical analysis when this is possible and try to
recognize in which cases there is evident bias.
The procedure for the analysis is illustrated in figure 3 and
with the main goals of:
• Quantify bias through the use of the WEAT.
• Compare between training corpuses.
• Compare between attributes of the same
subcategory.
• Compare between inter-category attributes.
• Compare between intra-category attributes.
• Discover whether the de-biased model achieves

better results.
• Discover the historical path of some formerly
discovered biases.
Figure 3: Processing pipeline for analysis
The categorical analysis focuses in the case of biased models
in specific categories which are outlined in table. The most
common suspected categories for bias are described and
include gender, race, religion, social and objects. Those
categories are split into subcategories.
Category Subcategory
gender work/education
character
sexual orientation
religious -
racial color
ethnicity
citizenship
social age group

prof./economic group
political
non-people -
Table 1: Categorical analysis
For each word set, a list of words are used which are
synonyms or close to its context. Below are some examples
for the two cases of comparison. For each case, we define a
set X, a set Y and their relation in comparison to A and B is
put to be tested through WEAT. Sets like these form the basis
of our comparison and multiple cases have been tested.
X: career career
profession
work
successful
Y : family family
life
kids
wife

A: man man
male
he
himself
B: woman woman
female
she
herself
Table 2: Example of comparison set for gender bias
X: communism communism
communist
Y: socialism socialism
socialist
A: popular popular
rational
rational
B: unpopular unpopular
irrational

Table 3: Example of comparison set for social bias
IV. EXPERIMENTS AND DISCUSSION
a. Biased models
i. Gender bias
In investigating gender bias, an analysis was performed
based on three different subcategories, being
work/education, character and sexual orientation.
For work/education, there is a comparison between
engineering and humanities, doctor vs nurse, career vs
family, law vs medicine, author vs architect and cook vs
teacher. For the first four cases, there is a positive bias
which indicates man is more associated with the first
attribute. Specifically, engineering, doctor and career, all
of them with one exception show very strong bias as the
values are much larger than 1.
Interestingly, law is more associated with men than
medicine, perhaps owing this fact to medical professions
in general such as nurse, caretaker and others. A less

significant and not similar trend for all datasets trend is
shown in author vs architect which was also expected
since there is not a clear distinction also in reality. Cook
however is much more associated with women and
teacher with men. The cases where there is significant
bias averages absolute 1.35 (high), while the non-biased
case averages absolute 0.4 (small to medium).
Figure 4: Representation of gender bias
ii. Racial bias
For the racial bias category, the first six comparisons
have to do with skin color such as Europe vs Africa with
educated vs illiterate and African vs Asian with safe vs
dangerous. The words chosen are not color such as white
and black as these would contain noise from the actual
colors. When comparing Europe and Africa, we can see
that in most cases, bias is strong for safe and educated
towards the former. For Caucasian and Asian there is a

strong bias only in Wikipedia concerning rich vs poor.
African-American vs latino for lawful vs outlaw shows
small bias while African vs Asian for safe vs dangerous
shows negative strong or medium positive. Interestingly,
the word latino is found to be more associated to superior
than African- American in all datasets with strong
negative bias around -1.
The next sub-category, which is country, first compares
common stereotypes such as German vs Italian for
hardworking vs lazy. It is interesting again that the only
dataset that seems to be biased towards the stereotype is
neither the news nor social media but Wikipedia. The
same is true for the comparison of American vs Russian
and friend vs enemy. Very weak biases are found in
commonly rich countries when comparing between
them, such as Polish, Danish, Japanese, Korean, with
effect sizes less than 0.5.
Finally, citizenship shows illegal residents to be

significantly be associated with dangerous. There is no
clear however bias in immigrants and refugees though as
they are both moderately to low considered good and
bad.
Figure 5: Representation of racial bias
iii. Religious bias
When looking at religious bias, an effort was made again
to discover bias and compare known stereotypes with
combinations that probably also are not biased. The
biased results seem to come when comparing
Christianity vs Islam with peace vs violence and church
with mosque. A strong bias is found in news and
Wikipedia while twitter remains at non to small. In the
case of Sikhism vs Hinduism, only Wikipedia shows no
bias while the rest indicate that bad is associated with
Hinduism. Between protestant and orthodox another
comparison was made to compare with a non-biased as
expected comparison between rich and poor.

Figure 6: Representation of religious bias
iv. Social bias
In social bias, a split in comparison is first done for
different age groups. Old people are clearly strongly to
medium associated with slow and impolite.
Next, for social groups, quite unexpected, aristocrats
were not strongly associated with rich. Perhaps, there
was not much reference of some common adjectives.
Then, bankers and doctors were found to be rich and
educated correspondingly.
Finally, for political systems, democracy was connected
to happy, capitalism to unfair and libertarian popular in
Wikipedia.
Figure 7: Representation of social bias
iv. Non-people bias
Looking at non-people bias, interestingly we find that Books
are strongly associated with amusing. That can be explained

by the fact that perhaps when they are mentioned people
usually talk positively about them. For football and basketball
results were towards small with the exception of news that
favored football.
Figure 8: Representation of non-people bias
v. Inter-category comparison
Looking back at the overall results per category, it can be seen
that in the gender category, there are many comparisons that
are biased. That of course depends on the selected words but
when there is a bias it is very strong with values at around 1.5
or more. The other categories also show bias for specific
comparisons, however their strong values remain at a bit
lower levels of around 1 to 1.5 with fewer exceptions going
above 1.5.
It is also evident that many common stereotypes are
confirmed in most cases such as those about gender related
occupations, racial stereotypes such as white vs black and
illegals as well as social stereotypes about rich, poor and

happy. Those that had small effect size validate our expected
results and the validity of the method in finding biases, one
example being protestant vs orthodox.
As far as the datasets are compared, Twitter contains less
biases and Google News and Wikipedia seem to be higher.
This conclusion, however, takes into account the overall
picture since there are many observations that show otherwise
and only specific words are tested here, which of course
cannot generalize to the overall datasets.
b. De-biased model
When comparing the gender specific de-biased model with
the simple model some improvements were found. In some
cases, such as doctor vs nurse and author vs architect the
effect is reversed.
To split them down however, there was a reduction of 50%
or more in engineering vs humanities, doctor vs nurse, law vs
medicine, cook vs teacher and beautiful vs ugly.
In the other, it remained almost same or increased.

Totally, it decreased in 10 out of 13 cases.
Figure 9: Comparison of biased and de-biased models
c. Historical models
For the purpose of analyzing evolution of biases within our
sub-categories, pairs with high and prominent WEAT metric
values have been chosen, which are interesting in terms of
analysis in historical perspective.
i. Gender bias
In this sub-category, we focused our attention on two groups
of words. For the first one (Career vs Family / Man vs
Woman), we can clearly observe that the idea of men being
more inclined towards career aspirations rather than
dedicating most of their effort and time to family compared
to women has been held in the literature throughout the whole
period of time, as the comparative value fell below 1.0 only
in 1850, with small fluctuations, but general tendency to
increase slowly over time.
The second pair (Straight vs Gay / Right vs Wrong) shows an

unexpected general rapid growth from 1820 to 1920, and the
values stay high until the year 1950, after which it starts
decreasing slowly. A possible explanation is that a negative
attitude towards the homosexual males might had been
increased for political and/or economic reasons, and in
addition, the word “gay” was assigned its current meaning in
the middle of 20th century, which could also cause the rise of
comparative WEAT score. Later, it started declining,
possibly because of the changes in public outlook on different
sexual orientations. It should be mentioned that the results
can also be influenced by the fact that some of the words may
possess several meanings (as with the word “straight”).
Figure 10: Temporal changes in bias of gender-related
words
ii. Racial bias
Here, we looked at the groups and compared them in terms
of bias in educational level and wealth. First pair (Europe vs

Africa / Educated vs Illiterate) there is no strong trend
towards descent or ascent of mutual score: if interpolated, the
score would be insignificantly below zero, as most of the
values fall within the range [0.5; 0.5] with a majority of points
below a zero line. That means that there is no consistent
strong bias in English literature in a given corpus regarding
illiteracy of Africans compared to Europeans, although in
some cases values fall below –0.5 (years 1940 and 1960).
In the second case, we can easily distinguish the stereotype in
the literature that white people are generally more well-off
than Asian people, with some exceptions in the data (years
1820 and 1900), which might have happened due to lack of
close topics. Generally, the fluctuation of the plot reduces in
time, which is possibly explained by a higher confidence of
writers about higher level of life of Caucasian people.
Nevertheless, all the inferences which are being made here
are just our own hypotheses, and the true reason may differ
from the average trend for particular years and cases.

In addition, we analyzed the potential bias between two
nations (Americans vs Russians) being treated as friends or
enemies. Generally, we see a huge variation of values in the
positive region of Y-axis values which means that Russians
are not treated as friends compared to Americans in English
literature. For some certain points, the inclination may be
viewed as a result of global historical events, such as a Cold
War which might be a reason of dramatic increase of the
mutual score from 1950 to 1960. Nevertheless, there is no
common pattern that can be reproduced from this plot.
Figure 11: Temporal changes in bias of race-related words
Figure 12: Temporal changes in bias of nationality-related
words
iii. Religious bias
Our third group is bias in religion. First pair (Christianity vs
Islam) shows a slight bias towards Christianity being treated
as a more peaceful religion that Islam. For some years, the
score value reflects a high bias (in the 19th century), but it is

mitigated over time, staying close to 0.5
For the second pair (Protestant vs Orthodox / Rich vs Poor),
we see a very interesting pattern: until the middle of 20th
century, orthodox branch is more associated with the wealth
that protestant, but then the dramatic growth is occurring. It
can be understood along the lines of development of market
economy in first world countries, where many protestant
followers have been residing, as well as by Protestant work
ethic, which does not forbid and even promote trading in
some sense, unlike with orthodox morals.
Figure 13: Temporal changes in bias of religion-related
words
iv. Social bias
In the “Social bias” category, we will consider groups related
to age and political preferences.
The pair (Old vs Young / Polite vs Impolite) does not indicate
a specific bias, as most of the values fall into the range [ -0.5;

0.5], where the bias does exist, but not so extreme to represent
high value for researchers. That being said, in the literature
sources stored in the database there was not a distinct
relationship between the age and the degree of politeness.
Similar tendency is observed with the second pair
(Libertarian vs Conservative / Popular vs Unpopular), where
the majority of values fluctuate around zero and do not
surpass 0.5 in absolute value. It means that none of mentioned
political views can be considered as being inclined towards
being popular in comparison to another based on the text
corpora used.
Figure 14: Temporal changes in bias of society-related
words
V. CONCLUSION AND FUTURE WORK
This work focused on analyzing biases in word embeddings
and performed a category based analysis using WEAT to
measure it. The results indicated that bias in present in many
popular datasets and sometimes strong bias too. Particularly

in cases with strong bias, it is evident in most datasets.
Gender bias was significant, followed by other, close together
such as racial, social and religious. The study went a step
head and split each category into different categories to
illustrate where exactly most bias is observed when talking
about gender and others. The same was then done when
comparing a biased model as well as specific cases of historic
development.
There are some limitations in our study such as the fact that a
limited amount of words could be chosen to represent each
set for comparison. For example, in specific occupations,
such as orthodox, the word could be specified by only a few
synonyms. Having a larger collection of appropriate words
would perhaps remove some of the noise introduced by
having less words. Conversely, some words may represent a
broad range of meanings which can create problems for the
approach to make sensible inferences (we observed such
examples during analysis of WEAT score temporal changes

for some words pairs, like “straight vs gay”).
Our current findings can be extended to be part of larger study
in the future that can attempt to develop a new method for
debiasing, in a way that it can eliminate bias in multiple
categories at the same time. Also, the thorough categorization
that was performed could be one of the ways to perform such
model testing. The historical analysis could give an
interesting outlook at how trends occur and where more
debiasing is needed for a model now or in the future
following a specific trend.
REFERENCES
[1] L. S. D. J. J. Z. Nikhil Garg, "Word embeddings
quantify 100 years of gender and ethnic stereotypes,"
in PNAS, 2018.
[2] A. M. Kaytlin Chaloner, "Measuring Gender Bias in
Word Embeddings across Domains and Discovering
New Gender Bias Word Categories".
[3] T. B. a. K.-W. C. a. J. Z. a. V. S. a. Adam, "Man is to

Computer Programmer as Woman is to Homemaker?
Debiasing Word Embeddings," 2016.
[4] D. B. Masahiro Kaneko, "Gender-preserving
Debiasing for Pre-trained Word Embeddings," in
Association for Computational Linguistics, 2019.
[5] https://code.google.com/archive/p/word2vec/,
"word2vec".
[6] R. S. C. D. M. Jeffrey Pennington, "GloVe: Global
Vectors for Word Representation," 2015.
[7] "Wikipedia," 1 5 2020. [Online]. Available:
https://en.wikipedia.org/wiki/Word2vec. [Accessed 5
5 2020].
[8] R. C. D. M. Jeffrey Pennington, "GloVe:
GlobalVectorsforWordRepresentation".
[9] "Gensim topic medelling for humans," 1 11 2019.
[Online]. Available:
https://radimrehurek.com/gensim/index.html.
[Accessed 1 5 2020].

[10] tolga-b, "GitHub," 2016. [Online]. Available:
https://github.com/tolga-b/debiaswe. [Accessed 1 5
2020].
[11] W. L. H. a. J. L. a. D. Jurafsky, "Diachronic Word
Embeddings Reveal Statistical Laws of Semantic
Change," in Association for Computational Linguistics
, 2016.
[12] williamleif, "histwords," 25 10 2015. [Online].
Available: https://github.com/williamleif/histwords.
[13] "SimplyPsychology," [Online]. Available:
https://www.simplypsychology.org/effect-size.html.
[14] "compare-embedding-bias," GitHub, 27 5 2019.
[Online]. Available:
https://github.com/hljames/compare-embedding-bias.

[15] M. L. Salvador, "Text analytics techniques in the
digital world: Word embeddings and bias," Irish
Communication Review, vol. 16, no. 1, 2018.
BUSA 205 Management Fundamentals
Chp. 11 Exercise: What Do Students Want From Their Jobs
NAME ___________________________________________
In this assignment, you will be iden4fying whether each
numbered item is considered an Extrinsic or an Intrinsic
Factor based on Herzberg's Two Factor Theory. An Extrinsic
factor tends to be something given to you by
management (an external) while an Intrinsic factor is something
that appeals to you from within yourself (an
internal mo4va4ng factor). The survey will help you to assess
what is important to you.
OBJECTIVES
1. To demonstrate individual differences in job expectations.
2. To illustrate individual differences in need and motivational
structures.
3. To examine and compare extrinsic and intrinsic rewards as
determined by Herzberg’s Two Factor Theory
What I Want from My Job
INSTRUCTIONS

1. Determine what you want from a job by circling the level of
importance of each of the following job rewards and place
an E (Extrinsic) or I (Intrinsic) in the first column identifyi ng
each reward as Hygiene (Extrinsic) or Motivation (I) based on
Herzberg’s Two Factor Theory.
2. Answer Questions # 1-3
Identify
(E) or (I)
Very
Important
Important Indifferent Unimportant Very
Unimportant
1. Advancement
Opportuni3es
5 4 3 2 1
2. Appropriate company
Policies
5 4 3 2 1
3. Authority 5 4 3 2 1
4. Autonomy and freedom on
the job
5 4 3 2 1
5. Challenging work 5 4 3 2 1

6. Company reputa3on 5 4 3 2 1
7. Fringe benefits 5 4 3 2 1
8. Geographic loca3on 5 4 3 2 1
9. Good co-workers 5 4 3 2 1
10. Good supervision 5 4 3 2 1
11. Job security 5 4 3 2 1
QUESTIONS
1. Which items received the highest and lowest scores from
you? Why?
_____________________________________________________
____________________________________________________
_____________________________________________________
____________________________________________________
_____________________________________________________
____________________________________________________
_____________________________________________________
___________________________________________________-
2. Were more response differences found in intrinsic or in
extrinsic rewards?
_____________________________________________________
_____________________________________________________

_
_____________________________________________________
_____________________________________________________
_
_____________________________________________________
_____________________________________________________
_
3. What are the implications for managers in trying to create
rewards that motivate?
_____________________________________________________
_____________________________________________________
_
_____________________________________________________
_____________________________________________________
_
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
_____________________________________________________
12. Money 5 4 3 2 1
13. Opportunity for
self-development
5 4 3 2 1

14. Pleasant office and
working condi3ons
5 4 3 2 1
15. Performance feedback 5 4 3 2 1
16. Pres3gious job 3tle 5 4 3 2 1
17. Recogni3on for doing a
good job
5 4 3 2 1
18. Responsibility 5 4 3 2 1
19. Sense of achievement 5 4 3 2 1
20. Training programs 5 4 3 2 1
21. Type of work 5 4 3 2 1
22. Working with people 5 4 3 2 1
Bias and fairness in Machine Learning
Motivation

Wide Application
scenarios of ML systems
Face recognition
system
Speech recognition
system
Intrusion
Detection System
Autonomous
Driving
Automatic information
management system
Wireless
communication
Is there any ethic issue?
Machine learning pipeline
Data
Machine learning
algorithms
Data-Driven Decision
Making
Dataset bias Algorithm fairness

Questions:
◦ What is the bias for ML datasets and how it affects the
decision making process?
◦ What is the fairness for ML algorithms and how it affects the
decision making process?
◦ Our contribution: Try to distinguish a biased or unfair issue on
real-life dataset and find out corresponding
solutions.
Bias for datasets
Definition: When scientific or technological decisions are based
on a narrow set of systemic,
structural or social concepts and norms, the resulting
technology can privilege certain groups
and harm others [BiasFairness18].
Classification [BiasClass]:
◦ Sample bias
◦ Exclusion bias
◦ Measurement bias
◦ Recall bias
◦ Observer bias
◦ Racial bias
◦ Association bias
[BiasFairness18] Bias and Fairness in AI/ML models
https://fpf.org/wp-content/uploads/2018/11/Presentation-
2_DDF-1_Dr-Swati-Gupta.pdf
[BiasClass] 7 Types of Data Bias in Machine Learning
https://lionbridge.ai/articles/7-types-of-data-bias-in-machine-
learning/
[Survey19] Mehrabi, Ninareh, et al. "A survey on bias and

fairness in machine learning." arXiv preprint arXiv:1908.09635
(2019).
Example - IMAGENET sample bias [Survey19]:
https://lionbridge.ai/articles/7-types-of-data-bias-in-machine-
learning/
Fairness for algorithms[Fairness18]
Definition[Intro17]:
◦ No Universal definition
• Unawareness
• Demographic Parity
• Equalized Odds
• Predictive Rate Parity
• Individual Fairness
• Counterfactual fairness
Example – COMPAS algorithm[Fairness18]:
◦ A machine learning system used by U.S officials
to do recidivism prediction
◦ Suppose to be a fair algorithm but actually show
bias against minority groups
[Intro17] A Tutorial on Fairness in Machine Learning
https://towardsdatascience.com/a-tutorial-on-fairness-in-
machine-learning-3ff8ba1040cb
[Fairness18] Chouldechova, Alexandra, and Aaron Roth. "The
frontiers of fairness in machine learning." arXiv preprint
arXiv:1810.08810 (2018).

Related datasets[Survey19]
Dataset Name Size Area Reference
UCI Adult dataset 48842 income records Social A. Asuncion
and D.J. Newman. 2007. UCI Machine Learning Repository.
(2007).
http://www.ics.uci.edu/$sim$mlearn/
German credit dataset 1000 credit records Financial Dheeru Dua
and Casey Graff. 2017. UCI Machine Learning Repository.
(2017).
http://archive.ics.uci.edu/ml
Pilot parliaments
benchmark dataset
1270 images Facial images Joy Buolamwini and Timnit Gebru.
2018. Gender Shades: Intersectional Accuracy Disparities in
Commercial Gender Classification. In Proceedings of the 1st
Conference on Fairness,
Accountability and Transparency (Proceedings of Machine
Learning Research), Sorelle A.
Friedler and Christo Wilson (Eds.), Vol. 81. PMLR, New York,
NY, USA, 77–91.
http://proceedings.mlr.press/v81/buolamwini18a.html
WinoBias 3160 sentences Coreference
resolution
Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and
Kai-Wei Chang. 2018. Gender
Bias in Coreference Resolution: Evaluation and Debiasing
Methods. (2018).
arXiv:cs.CL/1804.06876
Communities and crime

dataset
1994 crime records Social M Redmond. 2011. Communities and
crime unnormalized data set. UCI Machine Learning
Repository. In website: http://www. ics. uci.
edu/mlearn/MLRepository. html (2011).
COMPAS Dataset 18610 crime records Social J Larson, S
Mattu, L Kirchner, and J Angwin. 2016. Compas analysis.
GitHub, available at:
https://github. com/propublica/compas-analysis[Google Scholar]
(2016).
Recidivism in juvenile
justice dataset
4753 crime records Social Manel Capdevila, Marta Ferrer, and
Eulália Luque. 2005. La reincidencia en el delito en la
justicia de menores. Centro de estudios jurídicos y formación
especializada, Generalitat de
Catalunya. Documento no publicado (2005).
Diversity in face dataset 1 million images Social Michele
Merler, Nalini Ratha, Rogerio S Feris, and John R Smith. 2019.
Diversity in Faces. arXiv
preprint arXiv:1901.10436 (2019).
Recent Related works
Category Name Citations Reference
Survey A Survey on Bias and Fairness in
Machine Learning
258 Mehrabi, Ninareh, et al. "A survey on bias and fairness in

machine
learning." arXiv preprint arXiv:1908.09635 (2019).
Fairness in machine learning: A
survey
10 Caton, Simon, and Christian Haas. "Fairness in Machine
Learning: A
Survey." arXiv preprint arXiv:2010.04053 (2020).
Bias Ethical Implications of Bias in
Machine Learning
38 Yapo, Adrienne, and Joseph Weiss. "Ethical implications of
bias in machine
learning." Proceedings of the 51st Hawaii International
Conference on System
Sciences. 2018.
Identifying and Correcting Label
Bias in Machine Learning
41 Jiang, Heinrich, and Ofir Nachum. "Identifying and
correcting label bias in
machine learning." International Conference on Artificial
Intelligence and
Statistics. PMLR, 2020.
Understanding Bias in Machine
Learning
3 Gu, Jindong, and Daniela Oelke. "Understanding bias in
machine
learning." arXiv preprint arXiv:1909.01866 (2019).
Fairness Fairness in machine learning 117 Barocas, Solon,

Moritz Hardt, and Arvind Narayanan. "Fairness in machine
learning." Nips tutorial 1 (2017): 2.
The frontiers of fairness in machine
learning
133 Chouldechova, Alexandra, and Aaron Roth. "The frontiers
of fairness in
machine learning." arXiv preprint arXiv:1810.08810 (2018).
Improving fairness in machine
learning systems: What do
industrial practitioner need?
135 Holstein, Kenneth, et al. "Improving fairness in machine
learning systems:
What do industry practitioners need?." Proceedings of the 2019
CHI
conference on human factors in computing systems. 2019.
Q&A
Bias and fairness in Machine LearningMotivationMachine
learning pipelineBias for datasetsFairness for
algorithms[Fairness18] Related datasets[Survey19]Rece nt
Related works幻灯片编号 8

All authors contributed equally.An Analysis of Categoric

Recommended

Recommended

More Related Content

Similar to All authors contributed equally.An Analysis of Categoric

Similar to All authors contributed equally.An Analysis of Categoric (20)

More from MargaritoWhitt221

More from MargaritoWhitt221 (20)

Recently uploaded

Recently uploaded (20)

All authors contributed equally.An Analysis of Categoric