Soft cardinality has been shown to be a very strong text-overlapping baseline for the task of measuring semantic textual similarity (STS), obtaining 3rd place in SemEval-2012. At *SEM-2013 shared task, beside the plain text-overlapping approach, we tested within soft cardinality two distributional word-similarity functions derived from the ukWack corpus. Unfortunately, we combined these measures with other features using regression, obtaining positions 18th, 22nd and 23rd among the 90 participants systems in the official ranking. Already after the release of the gold standard annotations of the test data, we observed that using only the similarity measures without combining them with other features would have obtained positions 6th, 7th and 8th; moreover, an arithmetic average of these similarity measures would have been 4th(mean=0.5747). This paper describes both the 3 systems as they were submitted and the similarity measures that would obtained those better results.

1. STSqgrams STSnpmi
STScsds
SOFTCARDINALITY-CORE: Improving Text Overlap with
Distributional Measures for Semantic Textual Similarity
Sergio Jimenez Claudia Becerra Alexander Gelbukh
Soft cardinality has been shown to be a very strong text-
overlapping baseline for the task of measuring semantic textual
similarity (STS), obtaining 3rd place in SemEval-2012. At *SEM-
2013 shared task, beside the plain text-overlapping approach, we
tested within soft cardinality two distributional word-similarity
functions derived from the ukWack corpus. Unfortunately, we
combined these measures with other features using
regression, obtaining positions 18th, 22nd and 23rd among the 90
participants systems in the official ranking. Already after the
release of the gold standard annotations of the test data, we
observed that using only the similarity measures without
combining them with other features would have obtained
positions 6th, 7th and 8th; moreover, an arithmetic average of
these similarity measures would have been 4th (mean=0.5747).
This paper describes both the 3 systems as they were submitted
and the similarity measures that would obtained those better
results.
Abstract
A=
, ,
B=
, ,
|A|=3
|B|=3
Classical
(integer)
Soft
(real)
Soft Cardinality
Cardinality: number of different elements in a
collection, i.e. set definition.
C=
,
= |C|=1 |C|’=1.0
inter-elements
similarity
elements
weights
“softness”
control
word-to-word
similarity
idf term
weighting
Symmetrical Tversky’s
Ratio Model
Original Tversky’s ratio model
A
B
α β
Character
q-grams
(strm reused with
a range of q-grams)
PMI
(normalized
in [0,1])
Context-set
distributional sim.
(strm re-used with
sentence frequencies)
Un-official Results
• Tokenization
• Lowecasing
• Stop-words
removal
Pre-processing
Distributional measures trained
in the ukWack corpus
You can build a simple semantic textual
similarity (STS) function by …
1. Representing sentences as un-ordered
collections of words (bags of words)
2. Counting different elements in those
collections and their unions using classic
set cardinality (but you better use our
soft cardinality.)
3. Comparing collection pairs using any
cardinality-based resemblance coefficient
such as Jaccard or Dice coefficient (but
you better use our Symmetrical Tversky’s
ratio model)
• Punctuation removal
• Stemming (Porter)
• IDF term weighting Word and character
q-grams overlap