Improving and Assessing Data Quality
of Knowledge Graphs
Ben De Meester
Public defense presentation
May 28th, 2020
Promotors Dr. Anastasia Dimou, Prof. Ruben Verborgh
Jury Prof. Filip De Turck, Prof. Steven Verstockt,
Prof. Erik Mannens, Dr. Juan Sequeda,
Prof. Heiko Paulheim, Prof. Jose Emilio Labra Gayo

Short story

Improving and Assessing Data Quality
of Knowledge Graphs

Better Data Integration

Why/When do you need to integrate
data?

Why data integration is needed
Reuse already existing data
Needed on all scales
People, companies, governments, …

Better data integration: how?
By providing methods and tools for
Cleaning data
Checking data
before, during, and after data integration.

Better data integration: how?
By providing methods and tools for
Cleaning data
Checking data
before, during, and after integration.
Simple enough!

Better data integration: how?
By providing methods and tools for
Cleaning data
Checking data
before, during, and after integration.
Challenge:
automatic
&
reusable

Automatic?

There’s a lotta data
Automatic?

(Re-)building systems per use case
is $$$.
Reusable?

I worked on reusable systems
to automatically improve or check
the quality when integrating data
Short story

What we’ll discuss
Intro
Integrating Data: Knowledge Graphs
Quality Problems
Cleaning: Data Quality
Checking: Schema Quality
What now?

What we’ll discuss
Intro
Integrating Data: Knowledge Graphs
Quality Problems
Cleaning: Data Quality
Checking: Schema Quality
What now?

Data Integration Problems
Hard to exchange and reuse existing data
Different formats
Different meanings

Knowledge graphs
Framework to make structure and meaning of data
explicit
by adding meaningful (semantic) annotations
Why semantics?
How do I add these annotations?

Why semantics?

Semantics?
Bass

Semantics?

Semantics?

Also works the other way around
Name
Lastname
Family name
Achternaam
lname
Name of his father

Semantics!
By making the meaning explicit, it’s easier to
integrate
AND …

You know more than you know!
= Inferencing!
My bass has 4…

You know more than you know!
= Inferencing!
My bass has 4…

You know more than you know!
= Inferencing!
My bass has 4 strings!

How to add these annotations?

Can we use a global information system?
The Web!
Let’s steal/reuse from there.
We use links to look up further information
<http://ben.be/myBass>
<http://dict.com/hasGuitarStrings> "4"

Knowledge Graph
"Data
values"
<Semantic
annotations>+

What we’ll discuss
Intro
Data Integration: Knowledge Graphs
Quality Problems
Cleaning: Data Quality
Checking: Schema Quality
What now?

Garbage in = Garbage out
Data integration is great…
If quality is good
Data
Schema (= combined use of semantic annotations)

Bad data quality
<myBass> <hasGuitarStrings> "bunny"
<myBass> <hasGuitarStrings> "four"
<myBass> <hasGuitarStrings> "4.0"

Bad schema quality
<myBass> <hasUnderwearStrings> "4"
<Bass> <hasGuitarStrings> "4"
<Bass> <likesToEat> <Worm>
?

Bad schema quality
<myBass> <hasUnderwearStrings> "4"
<Bass> <hasGuitarStrings> "4"
<Bass> <likesToEat> <Worm>

Yeah, but how bad can it be?

Yeah, but how bad can it be?
Mars Climate Orbiter
Imperial vs metric system
320 million $

Yeah, but how bad can it be?
Mars Climate Orbiter
Imperial vs metric system
320 million $

BEN’S PHD STRATEGY

BEN’S PHD STRATEGY

BEN’S PHD STRATEGY

BEN’S PHD STRATEGY

BEN’S PHD STRATEGY
DATA VALUES
• Messy data when generating a
knowledge graph
DATA VALUES
• Clean data values using data
transformations
Phase 1a
Today

BEN’S PHD STRATEGY
DATA VALUES
• Incomplete data in a knowledge
graph
DATA VALUES
• Automatically derived data
values
Phase 1b
Today

BEN’S PHD STRATEGY
SCHEMA
• Mismatch when inferring new
things and checking/validating
the knowledge graph
SCHEMA
• Combine inferring en validating
Phase 2
Today

Knowledge Graph
"Data
values"
<Semantic
annotations>+

RML + FnO
Knowledge GraphImprove quality
of literals
during generation "Data
values"
<Semantic
annotations>+1a

FnO.io
RML + FnO
Knowledge Graph
Generalizes
Improve quality
of literals
during generation
Can complete
"Data
values"
<Semantic
annotations>+
1b
1a

FnO.io
RML + FnO
Validatrr
Knowledge Graph
Generalizes
Improve quality
of literals
during generation
Can complete Validates
"Data
values"
<Semantic
annotations>+
21b
1a

What we’ll discuss
Intro
Data Integration: Knowledge Graphs
Quality Problems
Cleaning: Data Quality
Data transformations
Checking: Schema Quality
What now?
1a

What we’ll discuss
Cleaning: Data Quality
When generating knowledge graphs: data
transformations
Knowledge graph generation: RML.io
Data transformations: FnO
Combining: RML+FnO
1a

Knowledge graph generation

Adding a new piece of data...

Process it...

To make it fit

To make it fit

Costs a lot!

RML.io
Set of technologies and tools
to stitch relevant pieces of data together,
by configuration

Take relevant data...

Take relevant data...

Take relevant data...

Take relevant data...

... And configure how to link it

Integrate multiple sources

What if the values themselves don’t fit?
“Thursday, May 28th 2020”

What if the values themselves don’t fit?
“Thursday, May 28th 2020”
“May 28 2020”
“28/05”
“Today”

ROCKBORINGEM
WWW.ROCK-BORINGEM.BE

How to describe any reusable
data transformation?

Configure how to process source data
“Thursday, May 28th 2020”
"2020-05-28"
FnO.io

Configure how to process source data
“Thursday, May 28th 2020”
"2020-05-28"
FnO.io

FnO: declarative functions
Function: from inputs to outputs
Function
Input Output

FnO: declarative functions
Function: from inputs to outputs
Date Parsing Function
Text Date

Music Instrument
Two hands Music

Music Instrument
MusicTwo hands

Music Instrument
MusicTwo hands

Music Instrument
MusicTwo hands

Function
Input Output

RML + FnO

RML + FnO

Use aligned RML and FnO
configurations
Filled gap: data transformations during Knowledge
Graph generation
Because it is a high-level configuration model
no restriction on complexity, use case, or implementation
not dependent on RML.io

Evaluation?
See whether we can handle real-world use case
with complex data transformations

Evaluation? DBpedia

We generated roughly the same
DBpedia data, and:
The system is reusable
You don’t depend on the implementation or use case
DBpedia data transformations can be reused
elsewhere
Alternative solutions would require re-implementation

What we’ll discuss
Cleaning: Data Quality
When generating knowledge graphs: data transformations
When completing knowledge graphs: automatic
calculations
FnO.io
1b

How can we automatically calculate
derived data?

Automatic calculation? Three-step plan
1. Discover relevant functions
2. Get implementations
3. Execute them automatically

FnO.io
Extended the FnO model
Implementation-independent
Describe links to actual implementations
Function Hub
Function and implementation discovery
Function Handler
Automatic function/implementation execution

Complements automatic calculations!
Discover relevant functions
Function Hub, using semantic descriptions
Find implementations
Carefully described using the extended FnO model
Execute them automatically
The Function Handler knows how to interpret these
descriptions to linked implementations

What we’ll discuss
Intro
Data Integration: Knowledge Graphs
Quality Problems
Cleaning: Data Quality
Checking: Schema Quality
What now?
2

Knowledge Graph
"Data
values"
<Semantic
annotations>+

From individual values
to sets of statements
<myBass> <hasGuitarStrings> "4"
<myBass> <model> <Ibanez>
<myBass> <hasColor> <purple>

Shapes
SHACL
ShEx

The All Acoustic music store
Stradivarius violin OK
Nord piano Not OK
Current languages/tools don’t allow
the combination of inferring and validating

How can we create a system
that can check shapes,
whilst inferring new things?

Contribution: Validatrr, using a rule-
based reasoner
You can combine rules to
Check the shapes
Infer new things
Support existing languages
Backtrace the used rules: know exactly why
something was wrong
Better that existing validators
e.g., SHACL has problems with disjunction: 4 string bass
guitar OR 6 string electric guitar

Evaluation
Compared to existing validators (RDFUnit)
Can check the same kind of shapes
Combining inferencing and checking
More detailed root cause description
Up to an order faster for smaller datasets (<100 000
statements)
Good alternative if you want more control, and
data isn’t too big

FnO.io
RML + FnO
Validatrr
Knowledge Graph
Generalizes
Improve quality
of literals
during generation
Can complete Validates
"Data
values"
<Semantic
annotations>+

What we’ll discuss
Intro
Data Integration: Knowledge Graphs
Quality Problems
Cleaning: Data Quality
Checking: Schema Quality
What now?

Conclusion
RML + FnO: complete generation toolkit
Uptake in multiple projects and companies
Need for standardization: community group

Conclusion
RML + FnO: complete generation toolkit
FnO.io: automatic function execution,
not only for knowledge graphs
Further exploitation of semantic descriptions
to be investigated
Need for more links to existing
functions / implementations

Conclusion
RML + FnO: complete generation toolkit
FnO.io: automatic function execution,
not only for knowledge graphs
Validatrr: expressive data checking
Get it standards-compliant
Exploit the combination of inferring and checking
Improve performance

What’s in it for you?
Right now?
Companies with complex data integrations
are already looking to us though.
So hopefully, in the not so distant future…
Nothing.

Improving and Assessing Data Quality
of Knowledge Graphs
Ben De Meester
Public defense presentation
May 28th, 2020
https://ben.de-meester.org/phd
(it has the afterword!)