Genetic programming is used to evolve data matching configurations that maximize accuracy on test data. The algorithm generates random initial configurations, evaluates them on the test data, and uses genetic operations of selection, crossover and mutation to evolve better configurations over generations. On several datasets, the genetic algorithm is able to find configurations that improve accuracy over manual configurations. However, the evolved configurations are not always intuitive and may represent local optima rather than global optima. More techniques from genetic programming literature could help address these issues.

1. Experiments in genetic programming
Bouvet BigOne, 2012-03-29
Lars Marius Garshol, <larsga@bouvet.no>
http://twitter.com/larsga
1

2. The background
• Duke
– open source data matching engine (Java)
– can find near-duplicate database records
– probabilistic configuration
– http://code.google.com/p/duke/
• People find making configurations difficult
– can we help them? Field Record 1 Record 2 Probability
Name acme inc acme inc 0.9
Assoc no 177477707 0.5
Zip code 9161 9161 0.6
Country norway norway 0.51
Address 1 mb 113 mailbox 113 0.49
Address 2 0.5
2

3. The idea
• Given
– a test file showing the correct linkages
• can we
– evolve a configuration
• using
– genetic algorithms?
3

4. What a configuration looks like
• Threshold for accepting matches
– a number between 0.0 and 1.0
• For each property
– a comparator function (Exact, Levenshtein, numeric...)
– a low probability (0.0-0.5)
– a high probability (0.5-1.0)
4

5. The hill-climbing problem
5

6. How it works
1. Generate a population of 100 random
configurations
2. Evaluate the population
3. Throw away the 25 worst, duplicate the 25
best
4. Randomly modify the entire population
5. Go back to 2
6

7. Actual code
for generation in range(POPULATIONS):
print "===== GENERATION %s ================================" % generation
for c in population:
f = evaluate(c)
if f > highest:
best = c
highest = f
show_best(best, False)
# make new generation
population = sorted(population, key = lambda c: 1.0 - index[c])
# ditch lower quartile
population = population[ : -25]
# double upper quartile
population = population[ : 25] + population
# mutate
population = [c.make_new(population) for c in population]
7

8. Actual code #2
class GeneticConfiguration:
def __init__(self):
self._props = []
self._threshold = 0.0
# set/get threshold, add/get properties
def make_new(self, population):
# either we make a number or random modifications, or we mate.
# draw a number, if 0 modifications, we mate.
mods = random.randint(0, 3)
if mods:
return self._mutate(mods)
else:
return self._mate(random.choice(population))
def _mutate(self, mods):
c = self._copy()
for ix in range(mods):
aspect = random.choice(aspects)
aspect.modify(c)
return c
def _mate(self, other):
c = self._copy()
for aspect in aspects:
aspect.set(c, aspect.get(random.choice([self, other])))
return c
def _copy(self):
c = GeneticConfiguration()
c.set_threshold(self._threshold)
for prop in self.get_properties():
if prop.getName() == "ID":
c.add_property(Property(prop.getName()))
else:
c.add_property(Property(prop.getName(), prop.getComparator(), prop.getLowProbability(), prop.getHighProbability()))
return c
8

9. But ... does it work?!?
9

10. Linking countries
• Linking countries from DBpedia and Mondial
– no common identifiers
• Manually I manage 95.4% accuracy
– genetic script manages 95.7% in first generation
– then improves to 98.9%
– this was too easy...
DBPEDIA MONDIAL
Id http://dbpedia.org/resource/Samoa Id 17019
Name Samoa Name Western Samoa
Capital Apia Capital Apia, Samoa
Area 2831 Area 2860
10

11. The actual configuration
Threshold 0.6
PROPERTY COMPARATOR LOW HIGH
NAME Exact 0.19 0.91
CAPITAL Exact 0.25 0.86
AREA Numeric 0.36 0.72
Confusing.
Why exact name comparisons?
Why is area comparison given such weight?
Who knows. There’s nobody to ask.
11

12. Semantic dogfood
• Data about papers presented at semantic web
conferences
– has duplicate speakers
– about 7,000 records, many long string values
• Manually I get 88% accuracy
– after two weeks, the script gets 82% accuracy
– but it’s only half-way
Name Grigorios Antoniou Name Grigoris Antoniou
Homepage http://www.ics.forth.gr/~antoniou Homepage http://www.ics.forth.gr/~antoniou
Mbox_Sha1 f44cd7769f416e96864ac43498b08215 Mbox_Sha1 f44cd7769f416e96864ac43498b08215
5196829e 5196829e
Affiliation Affiliation http://data.semanticweb.org/organizat
ion/forth-ics
12

13. The configuration
Threshold 0.91
PROPERTY COMPARATOR LOW HIGH
NAME JaroWinklerTokenized 0.2 0.9
AFFILIATION DiceCoefficient 0.49 0.61
HOMEPAGE Exact 0.09 0.67
MBOX_HASH PersonNameComparator 0.42 0.87
Some strange choices of comparator.
PersonNameComparator?!?
DiceCoefficient is essentially same as Exact, for those values.
Otherwise as expected.
13

14. Hafslund
• I took a subset of customer data from Hafslund
– roughly 3000 records
– then made a difficult manual test file, where different
parts of organizations are treated as different
– so NSB Logistikk != NSB Bane
– then made another subset for testing
• Manually I can do no better than 64% on this
data set
– interestingly, on the test data set I score 84%
• With a cut-down data set, I could run the
script overnight, and have a result in the
morning
14

15. The progress of evolution
• 1st generation
– best scores: 0.47, 0.43, 0.3
• 2nd generation
– mutated 0.47 configuration scores 0.136, 0.467, 0.002,
and 0.49
– best scores: 0.49, 0.467, 0.4, and 0.38
• 3rd generation
– mutated 0.49 scores 0.001, 0.49, 0.46, and 0.25
– best scores: 0.49, 0.46, 0.45, and 0.42
• 4th generation
– we hit 0.525 (modified from 0.21)
15

16. The progress of evolution #2
• 5th generation
– we hit 0.568 (modified from 0.479)
• 6th generation
– 0.602
• 7th generation
– 0.702
• ...
• 60th generation
– 0.765
– I’d done no better than 0.64 manually
16

17. Evaluation
CONFIGURATION TRAINING TEST
Genetic #1 0.766 0.881
Genetic #2 0.776 0.859
Manual #1 0.57 0.838
Manual #2 0.64 0.803
Threshold: 0.98 Threshold: 0.95
PROPERTY COMPARATOR LOW HIGH PROPERTY COMPARATOR LOW HIGH
NAME Levenshtein 0.17 0.95 NAME Levenshtein 0.42 0.96
ASSOCIATION_NO Exact 0.06 0.69 ASSOCIATION_NO DiceCoefficien 0.0 0.67
t
ADDRESS1 Numeric 0.02 0.92
ADDRESS1 Numeric 0.1 0.61
ADDRESS2 PersonName 0.18 0.76
ADDRESS2 Levenshtein 0.03 0.8
ZIP_CODE DiceCoefficien 0.47 0.79
t ZIP_CODE DiceCoefficien 0.35 0.69
17 t
COUNTRY Levenshtein 0.12 0.64
COUNTRY JaroWinklerT. 0.44 0.68

18. Does it find the best configuration?
• We don’t know
• The experts say genetic algorithms tend to get
stuck at local maxima
– they also point out that well-known techniques for
dealing with this are described in the literature
• Rerunning tends to produce similar
configurations
18

19. The literature
http://www.cleveralgorithms.com/ http://www.gp-field-guide.org.uk/
19

20. Conclusion
• Easy to implement
– you don’t need a GP library
• Requires reliable test data
• It actually works
• Configurations may not be very tweakable
– because they don’t necessarily make any sense
• This is a big field, with lots to learn
20 http://www.garshol.priv.no/blog/225.html