The document describes how Elasticsearch can be used for name matching through custom analyzers, mappings, and rescoring queries. Key points extracted: - Names are indexed using a custom NameMapper that generates keys for name parts using different analyzers and stores them in separate fields. - Queries generate analogous keys to find candidate matches. A rescore query then scores how well the query name matches names in candidate documents to reorder results. - The rescore query uses a custom name_score function that retrieves the indexed name from a document and scores it against the query name using a cached scorer, returning a similarity score. This allows computationally expensive but high precision name matching.

1. Name Matching
with Elasticsearch
June 25, 2015
Graham Morehead
gmorehead@basistech.com

2. April 15 2013 2:49 PM .

8. Real life example
David K. Murgatroyd
VP of Engineering
Boarding Pass

9. Best Practice using Elasticsearch?
● NameMapper
(http://stackoverflow.com/questions/20632042/elasticsearch-searching-for-human-names)
"mappings": { ... "type": "multi_field", "fields": {
"pty_surename": { "type": "string", "analyzer": "simple" },
"metaphone": { "type": "string", "analyzer": "metaphone" },
"porter": { "type": "string", "analyzer": "porter" } …
● rescore_query

10. “Jesus Alfonso Lopez Diaz”
vs.
“LobezDias, Chuy”

11. RNI

15. Rescore Query
Main Query
Plug-in Implementation
match : {
name:
"Bob Smitty" }
bool:
name.Key1:...
name.Key2:...
name.Key3:...
User Query
Rescore
name_score : {
field : "name",
name : "Bob
Smitty")
name:"Robert
Smith"
dob:2/13/1987
score : .79
Indexing
{ name: "Robert
Smith"
dob:"
1987/02/13" }
{ name: "Robert
Smith"
name.Key1:…
name.Key2:…
name.Key3:…
dob:
"1987/02/13" }
User Doc
Index
subset

16. Demo

17. Elastic + RNI

18. Name Matching
with Elasticsearch
June 25, 2015
Graham Morehead
gmorehead@basistech.com

19. How could you use such a Field?
● Plugin contains custom mapper which does
all the work behind the scenes
PUT /ofac/ofac/_mapping
{
"ofac" : {
"properties" : {
"name" : { "type:" : "rni_name" }
"aka" : { "type:" : "rni_name" }
}
}
}

20. What happens at index time?
● NameMapper indexes keys for different
phenomena in separate (sub) fields
@Override
public void parse(ParseContext context) throws IOException {
Name name = NameBuilder.data(nameString).build();
//Generate keys for name
Collection<FieldSpec> fields = helper.deriveFieldsForName(name);
//Parse each key with the appropriate Mapper
for (FieldSpec field : fields) {
Mapper mapper = keyMappers.get(field.getField().fieldName());
context = context.createExternalValueContext(field.getStringValue());
mapper.parse(context);
}
}

21. What happens at query time?
● Step #1: NameMapper generates analogous
keys for a custom Lucene query that finds
good candidates for re-scoring
@Override
public Query termQuery(Object value, @Nullable QueryParseContext context) {
//Parse name string
Name name = NameBuilder.data(value.toString()).build();
QuerySpec spec = helper.buildQuerySpec(new NameIndexQuery(name));
//Build Lucene query
Query query = spec.accept(new ESQueryVisitor(names.indexName() + "."));
return query;
}

22. What else happens at query time?
● Step #2: Uses a Rescore query to score names in the
best candidate documents and reorder accordingly
○ Tuned for high precision name matching
○ Computationally expensive
"rescore" : {
"query" : {
"rescore_query" : {
"function_score" : {
"name_score" : {
"field" : "name",
"query_name" : "LobEzDiaS, Chuy"
}
...

23. ● The 'name_score' function matches the
query name against the indexed name in
every candidate document and returns the
similarity score
@Override
public double score(int docId, float subQueryScore) {
//Create a scorer for the query name
CachedScorer cs = createCachedScorer(queryName);
//Retrieve name data from doc values
nameByteData.setDocument(docId);
Name indexName = bytesToName(nameByteData.valueAt(i).bytes);
//Score the query against the indexed name in this document
return cs.score(indexName);
}
What does that function do?