Ontotext's GraphDB Connectors

GraphDB Connectors
Nikola Petrov<nikola.petrov@ontotext.com>
Ontotext AD
September 8, 2014
Nikola Petrov<nikola.petrov@ontotext.com> GraphDB Connectors

OWLIM GraphDB Family
GRAPHDB Server - the database engine
GRAPHDB Workbench - tool support that replaces the sesame
workbench
GRAPHDB Connectors - implements advanced search scenarios

Why? What we are trying to solve
Our models are really rich/complex but we still want to get the
results fast

results fast
Most of the queries in the search space are of the form:
SELECT < p r o j e c t i o n var iables >
WHERE {
{
. . . many t r i p l e pat terns to f i l t e r data . . .
. . . p o t e n t i a l l y regex or f u l l t e x t search . . .
}
. . .
{
. . . many t r i p l e pat terns to r e t r i e v e data . . .
. . . of ten many o p t i o n a l and aggregates . . .
}
}
Nikola Petrovnikola.petrov@ontotext.com GraphDB Connectors

results fast
Most of the queries in the search space are of the form:
SELECT p r o j e c t i o n var iables
WHERE {
{
. . . many t r i p l e pat terns to f i l t e r data . . .
. . . p o t e n t i a l l y regex or f u l l t e x t search . . .
}
. . .
{
. . . many t r i p l e pat terns to r e t r i e v e data . . .
. . . of ten many o p t i o n a l and aggregates . . .
}
}
We want to optimize the filtering part

Design philosophy

How it all started
Analyzed all major production systems projects
Collected feedback from partners
Reused features and know-how from multiple projects

Main Features
Real time synchronization
Property chains support (data denormalization)
Full-text search and result snippets
Facet and co-occurrence
Visual interface for index creation
Everything is integrated in SPARQL

Replication model

Create a link with a special predicate and we are going to
index/synchronize resources in the external store
PREFIX : http://www.ontotext.com/connectors/elasticsearch#
PREFIX inst: http://www.ontotext.com/connectors/elasticsearch/instance#
INSERT DATA{
inst:my_index :createConnector ’’’
# tell us what resources you want to index and which predicate
# values about them. We support predicate chains if the properties of
# the resource are not direct
’’’
}

INSERT DATA{
’’’
}
When you add/change resources that match the criteria, we will
automatically update the external store

INSERT DATA{
’’’
}
When you add/change resources that match the criteria, we will
automatically update the external store
Note
This might slow your insert statements a little bit(our tests show that this time is
negligible)

Example data with wines

@prefix rdf: http://www.w3.org/1999/02/22-rdf-syntax-ns# .
@prefix rdfs: http://www.w3.org/2000/01/rdf-schema# .
@prefix xsd: http://www.w3.org/2001/XMLSchema# .
@prefix : http://www.ontotext.com/example/wine# .
:RedWine rdfs:subClassOf :Wine .
:WhiteWine rdfs:subClassOf :Wine .
:RoseWine rdfs:subClassOf :Wine .
:Merlo
rdf:type :Grape ;
rdfs:label Merlo .
:CabernetSauvignon
rdf:type :Grape ;
rdfs:label Cabernet Sauvignon .
:CabernetFranc
rdf:type :Grape ;
rdfs:label Cabernet Franc .
:PinotNoir
rdf:type :Grape ;
rdfs:label Pinot Noir .
:Chardonnay
rdf:type :Grape ;
rdfs:label Chardonnay .

:Yoyowine
rdf:type :RedWine ;
:madeFromGrape :CabernetSauvignon ;
:hasSugar dry ;
:hasYear 2013^^xsd:integer .
:Franvino
rdf:type :RedWine ;
:madeFromGrape :Merlo ;
:madeFromGrape :CabernetFranc ;
:hasSugar dry ;
:Noirette
rdf:type :RedWine ;
:madeFromGrape :PinotNoir ;
:hasSugar medium ;
:Blanquito
rdf:type :WhiteWine ;
:madeFromGrape :Chardonnay ;
:hasSugar dry ;
:Rozova
rdf:type :RoseWine ;
:madeFromGrape :PinotNoir ;
:hasSugar medium ;

Create our wines index
INSERT DATA {
inst:wines :createConnector ’’’
{
elasticsearchNode: localhost:9300,
types: [
http://www.ontotext.com/example/wine#Wine
],
fields: [
{
fieldName: grape,
propertyChain: [
http://www.ontotext.com/example/wine#madeFromGrape,
http://www.w3.org/2000/01/rdf-schema#label
]
},
{
fieldName: sugar,
propertyChain: [http://www.ontotext.com/example/wine#hasSugar],
sort: true
},
{
fieldName: year,
propertyChain: [http://www.ontotext.com/example/wine#hasYear]
}
]
}
’’’ .
}

But wait, I don’t understand all this json stuff...

Elasticsearch index
Given our wines small dataset, here is what will get indexed in
elasticsearch
wine year grape sugar
:Yoyowine 2013 Cabernet Sauvignon dry
:Noirette 2012 Pinot Noir medium
:Blanquito 2012 Chardonnay dry
:Franvino 2012 Merlo, Cabernet Franc dry
:Rozova 2013 Pinot Noir medium

You can query the index and join the results in SPARQL
SELECT ?entity {
?search a inst:index-name ;
:query full-text-query ;
:entities ?entity .
# Do anything you want with the bound entity
}

Get wines that are made from cabernet grape
PREFIX wine: http://www.ontotext.com/example/wine#
SELECT ?wine ?year {
?search a inst:wines ;
:query grape:cabernet ;
:entities ?wine .
# We combine the results from elasticsearch and join them in graphdb
?wine wine:hasYear ?year
}

Get wines that are made from cabernet grape
PREFIX wine: http://www.ontotext.com/example/wine#
SELECT ?wine ?year {
?search a inst:wines ;
:query grape:cabernet ;
:entities ?wine .
# We combine the results from elasticsearch and join them in graphdb
?wine wine:hasYear ?year
}
Wine Year
:Yoyowine 2013
:Franvino 2012

Other things that we expose for the matches
The matching snippet for the full-text query

The score from elasticsearch for the match

Total number of hits in the external store

Total number of hits in the external store
Ordering the external store results

Faceting Usecase LCIE Co-occurrence

Faceting Usecase LCIE Facets and Full-text

Faceting Usecase Global Capital

Faceting Usecase
SELECT ?facetName ?facetValue ?facetCount WHERE{
# note empty query is allowed and will just match all documents, hence no :query
?r a inst:wines_index ;
:facetFields year,sugar ;
:facets _:f .
_:f :facetName ?facetName .
_:f :facetValue ?facetValue .
_:f :facetCount ?facetCount .
}

Faceting Usecase
SELECT ?facetName ?facetValue ?facetCount WHERE{
# note empty query is allowed and will just match all documents, hence no :query
?r a inst:wines_index ;
:facetFields year,sugar ;
:facets _:f .
_:f :facetName ?facetName .
_:f :facetValue ?facetValue .
_:f :facetCount ?facetCount .
}
Facet Name Facet Value Facet Count
year 2012 3
year 2013 2
sugar dry 3
sugar medium 2

Entity filtering(Advanced topic)
A way to filter resources that land in the external store based the
field value

field value
Allows you to use the same predicate/property chain for different
fields(based on a filter)

field value
Allows you to use the same predicate/property chain for different
fields(based on a filter)
Consider the usecase of articles that reference people, locations,
etc

Entity filtering
INSERT DATA {
{
elasticsearchNode: localhost:9200,
types: [http://www.ontotext.com/example2#Article],
fields: [
{
fieldName: comment,
propertyChain: [http://www.w3.org/2000/01/rdf-schema#comment]
},
{
fieldName: taggedWithPerson,
propertyChain: [http://www.ontotext.com/example2#taggedWith]
},
{
fieldName: taggedWithLocation,
propertyChain: [http://www.ontotext.com/example2#taggedWith]
}
],
entityFilter: ?taggedWithPerson type in
(http://www.ontotext.com/example2#Person)
?taggedWithLocation type in (http://www.ontotext.com/example2#Location)
}
’’’ .
}

Things that don’t work right now but will in the near future
support for remote external stores(solr, elasticsearch) in graphdb
cluster setup
filtering anywhere in the property chain(we support only filtering
the indexed value)
filtering implicit from explicit statements

Query tree

Thank You
https://confluence.ontotext.com/display/GraphDB6/GraphDB+Connectors
Questions?

Ontotext's GraphDB Connectors

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (6)

Similar to Ontotext's GraphDB Connectors

Similar to Ontotext's GraphDB Connectors (20)

Recently uploaded

Recently uploaded (20)

Ontotext's GraphDB Connectors