The document describes Ephedra, a SPARQL federation engine that efficiently combines distributed RDF data and services using SPARQL queries. Ephedra extends the RDF4J API to treat compute services as virtual RDF repositories. It performs optimizations like reordering clauses, pushing limits/orders down, and parallel competing joins. An evaluation on cultural heritage and life science queries showed runtime improvements over no optimization. Future work includes backend-aware optimizations and collecting service statistics for improved planning. Ephedra provides an architecture for integrating diverse data sources and services through SPARQL federation.

1. Ephedra: efficiently combining RDF data and services
using SPARQL federation
Andriy Nikolov
Peter Haase
Johannes Trame
Artem Kozlov

2. 2
• Motivation: traditional vs hybrid federation
• Background: metaphactory platform
• Ephedra SPARQL federation engine:
• Architecture
• Optimization techniques
• Evaluation
Table of Contents

3. 3
• Multiple physically distributed data repositories
• Each data repository represents a SPARQL endpoint
• sometimes, a relational database exposed using R2RML
”Canonical” SPARQL 1.1 federation scenario
Relational Database
OBDA / ETL
Graph database Graph database Graph database Graph database

4. 4
• Not only SPARQL endpoints!
• RDF data
• Custom indices
• Compute services
• Machine learning models
• Enterprise APIs
• …
Hybrid federation scenario
Graph database
Machine learning model Custom API
?
Specialized indices

5. 5
• Variety of data sources
• Local/remote RDF stores, virtual RDF
(Ontop)
• Variety of data modalities
• Graph, text, temporal, geospatial
• Variety of data processing techniques
• Graph analytics
• Statistical analysis/machine learning
• Domain-specific (BLAST genome
sequence similarity)
Challenges: Hybrid federation queries
Graph database
Relational Database
OBDA / ETL
NoSQL Data
/ Elastic Search Engines
Deep Learning
Service
BLAST Service

6. 6
Background: metaphactory platform
Alexa
Semantic Search VisualizationAuthoring
Tableau
metaphactory Frontend – Modular W3C Web Components
metaphactory Backend – Java 8
Ontology Mgmt.
Service
Tableau Connector
Service
NLP Intent
Service
Hybrid SPARQL ServiceSPARQL
Service
RDF Graph Store
- Data Ingestion
Query Catalog
Service
Query as a
Service
metaphactory - Java servlet, Jetty Deployment
REST via JAVAX/Jersey, ACL/Security via Apache Shiro
3rd Party Tools / Services
Runtime / Configuration Data
Relational Database
OBDA / ETL
Graph database Main repository

7. 7
Solution: Ephedra architecture
Graph Databases Blazegraph
Triplestore
Alexa
Semantic Search VisualizationAuthoring
Tableau
metaphactory Frontend – Modular W3C Web Components
metaphactory Backend – Java 8
Ontology Mgmt.
Service
NoSQL Data
/ Elastic Search Engines
Tableau Connector
Service
NLP Intent
Service
Hybrid SPARQL ServiceSPARQL
Service
RDF Graph Store
- Data Ingestion
Query Catalog
Service
Query as a
Service
metaphactory - Java servlet, Jetty Deployment
REST via JAVAX/Jersey, ACL/Security via Apache Shiro
3rd Party Tools / Services
RDF Data Structured Data / Indices (Text / Spatial…) Analytics Results
Relational Database
OBDA / ETL
Deep Learning
Service
R Service
Runtime / Configuration Data
RDF4J Federation SAIL API
Query plan optimizer
Runtime query execution engine
Service
Registry
Ephedra

8. 8
• Extends RDF4J (ex-Sesame) API
• Compute services are wrapped into “virtual” RDF4J
repositories
• SPARQL graph patterns are transformed into API calls
• Service wrapper repositories are explicitly described in the
service registry
• Input/output parameters
• Expected graph patterns
• SPARQL 1.1 federation using the SERVICE keyword
• No automatic source selection
Main principles

9. 9
# Select a painter similar to rembrandt
SELECT ?painter WHERE {
SERVICE eph:word2vec {
wd:Q5598 word2vec:hasSimilar ?painter .
}
?painter wdt:P106 wd:Q1028181 . # occupation: painter
}
Describing services
word2vec
embeddings
in: URI out: URI[]
# Service type descriptor (extended SPIN)
eph:word2vec a eph:Service ;
eph:hasSPARQLPattern [
sp:subject :_inputURI ;
sp:predicate word2vec:hasSimilar ;
sp:object :_outputURI .
] ;
spin:constraint [ spl:predicate _inputURI ] ;
spin:column [ spl:predicate _outputURI ] .
Service
Registry
wd:Q5598
?painter =
wd:Q5597
# Raphael
Matching service inputs/outputs to
SPARQL patterns
# Service instance descriptor (RDF4J SAIL)
serviceURL = http://wikidatatest.metaphacts.com/word2vec

10. 10
Static optimizations
• Reordering of clauses based on input/output constraints
• Rank-aware optimizations
• Pushing LIMIT and ORDER BY operations down the tree
?uri
SELECT * WHERE {
SERVICE eph:word2vec {
?uri word2vec:hasSimilar ?painter .
}
SERVICE eph:wikidataText {
?uri wikidata:search “rembrandt”
}
}
SELECT * WHERE {
SERVICE eph:wikidataText {
?uri wikidata:search “rembrandt”
}
SERVICE eph:word2vec {
?uri word2vec:hasSimilar ?painter .
}
}

11. 11
• Synchronizing loop join requests
• Synchronous vs asynchronous
• Separate requests vs batch
• “Breadth-first” vs “depth-first”
Runtime optimizations
?id1 wikidata:search
“hokusai”.
?id1BM owl:sameAs ?id1 .
?id1BM :collaboratedWith
?id2BM .
?id2BM owl:sameAs ?id2 .
?id2 word2vec:hasSimilar ?id3.
Σ": Wikidata text search API Σ#: British Museum Σ$: word2vec
𝜇" 𝜇""
𝜇"#
𝜇"""
𝜇""#
𝜇" 𝜇""
𝜇"#
𝜇"""
𝜇""#

12. 12
• Multiple alternative plans possible:
• Selectivity hard to estimate in advance
• Alternative plans executed in parallel (competing)
• Plans revised during execution
Parallel competing join
SELECT * WHERE {
SERVICE eph:wikidataText {
?home wikidata:search “florence” .
}
SERVICE eph:wikidataText {
?char wikidata:search “mary” .
}
?image wdt:P180 ?character . # depicts
?image wdt:P179 ?creator . # creator
?creator wdt:P19 ?home . # place of birth
}
Σ"
Σ#
Σ$
Σ#
Obtained
results Σ$
Obtained
results
Hash join
Σ"

13. 13
• Cultural heritage
• RDF stores
• British Museum
• Wikidata
• Services
• Wikidata text search
• word2vec similarity service
• Life sciences
• RDF stores
• Wikidata, Nextprot, Uniprot
• Proprietary data
• Services
• Wikidata text search
• BLAST sequence similarity
Applications

14. 14
• Benchmark queries from 2 use cases domains:
• 4 from cultural heritage, 3 from life sciences
• Compared runtime with and without the Ephedra
optimization techniques
Evaluation
0
2
4
6
8
10
12
14
CH1 (2+2) CH2 (1+1) CH3 (2+0) CH4 (2+2) PH1 (1+2) PH2 (1+1) PH3 (2+1)
Query runtime
No runtime optimization Runtime optimization

15. 15
• Architecture for integrating SPARQL endpoints and
compute services using SPARQL 1.1 federation
• Explicit mappings between SPARQL nodes and service
input/output parameters
• Static and runtime optimizations for hybrid queries
• Future work:
• Backend-aware optimizations
• Special treatment of relational databases exposed via Ontop
• Optimizations based on service calls statistics
• Applications
• Integrated query results as input for machine learning
• Browsing integrated data + services as a virtual knowledge
graph
Summary

16. 16
• Questions?