oft cardinality (SC) is a softened version of the classical cardinality of set theory. However, given its prohibitive cost of computing (exponential order), an approximation quadratic in the number of terms in the text has been proposed in the past. SC Spectra is a new method of approximation in linear time for text strings, which divides text strings into consecutive substrings (i.e., q-grams) of different sizes. Thus, SC in combination with resemblance coefficients allowed the construction of a family of similarity functions for text comparison. These similarity measures have been used in the past to address a problem of entity resolution (name matching) outperforming SoftTFIDF measure. SC spectra method improves the previous results using less time and getting better performance. This allows the new method to be used with relatively large documents such as those included in classic information retrieval collections. SC spectra method exceeded SoftTFIDF and cosine tf-idf baselines with an approach that requires no term weighing.

1. SC Spectra: A Linear-Time
Soft Cardinality Approximation for
Text Comparison
Sergio Jimenez
Universidad Nacional de Colombia
Alexander Gelbukh
CIC-IPN
Doctoral
Consortium

2. Sets ?model better? human similarity
judgements than vectors.
• Yes… Tvesky: Similarity statements, images,
sounds, symbols, etc.
• Useful at task where …. (e.g. Entity
Resolution, Name Matching, … )
SC Spectra: A Linear-Time Soft Cardinality Approximation for Text Comparison

3. Set Similarity Measures
Cardinality-based
||
||
BA
BA
|)||(|5.0
||
BA
BA
||||
||
BA
BA
ppp
BA
BA
/1
)|||(|5.0
||
Jaccard Dice Cosine Overlap
|)||,min(|
||
BA
BA
Generalized Mean
0,,
||||||
||
ABBABA
BA
Tversky’s Ratio Model
p= 1 Dice
p 0 Cosine
p ∞ Overlap
α=β =1 Jaccard
α=β =0.5 Dice
commonalities
referent
A
B
You only need to count the number of elements in a set!

4. But cardinality-based measures are crisp ….
,B
,A
BA 2
1
|)||(|5.0
||
BA
BA
… because classic set cardinality doesn’t take into
account similarities between elements.
Dice coeficient

5. So, we proposed soft cardinality …
… a cardinality function that counts elements in a
softened way, denoted
, ,
, ,
For a set },,,{ 21 naaaA 
the soft cardinality of A is

n
i
iaA
1
'
A
B
9.2
'
A
2.1
'
B
'
*

6. Soft Cardinality for Name Matching
Sergio Gonzalo Jiménez
Cergio G. Jimenes
Sergio
Gonzalo
Jiménez
Cergio
G.
Jimenes
Sergio
Gonzalo
Jiménez
Cergio
G.
Jimenes
|'| BA
'
|| A
'
|| B
A
B

7. But …
the soft cardinality definition
requires the computation of the cardinality of
the union of n sets, which requires 2n-1 terms
and n-ary intersections.
e.g.

n
i
iaA
1
'
a1 a2
a1
321 aaa
321313221321 aaaaaaaaaaaa
1 2 3 4 5 6 7
3-ary intersectionbinary intersections

8. So, we proposed an approximation …
… using only:
1. Pair-wise intersections (binary similarity function)
2. n2 terms for a set of n elements
n
i
n
j
ji aasim
A
1
1
'
),(
1
),( jiji aasimaa
But (again) O(n2) is fine for short text (i.e. name matching)
but still impractical for lorger texts.

9. Approximating Soft Cardinality using
character q-grams (n-grams)
An example using 2-grams: }"","{" GonzalezGonzaloA
""1 Gonzaloa
""2 Gonzaleza
},,,,,,,{]2[
1  oloalzanzonGoGa
},,,,,,,,{]2[
2  zezlealzanzonGoGa
},,,,,,,,,,{]2[
2
]2[
1
]2[
 zezleoloalzanzonGoGaaA
0.100
0.105
0.110
0.115
0.120
0.125
0.130
◄G Go on nz za al lo o► le ez z►
2grams in both 2grams only in a1 2grams only in a2