This document discusses data quality and data integration in the context of big data. It presents two possible workflows for linking relational data to semantic data on the web. The first workflow performs record linkage and transformation together, while the second performs transformation and semantic enrichment before matching. The second workflow is able to improve completeness by leveraging external semantic knowledge from DBpedia. An example is provided to illustrate how the workflows may be applied to link sample relational data to semantic web resources.

1. From Data Quality to Big Data Quality:
A Data Integration Scenario
Carlo Batini*, Anisa Rula+
*University of Milano-Bicocca
+Department of Information Engineering,
University of Brescia

2. Data Quality and Data Integration in a 3-dimensional
Data Space
2
- Relational data
- Traditional information systems
Data Types
Big Data
lifecycle
Data Quality Clusters
1
2
3

3. Big Data Lifecycle
Source selection
Discovery
Retrieval
Browsing
Internet of Things Design
Extraction
Sampling
Modeling
Filtering
Creation
Management
Interpretation
Modeling
Semantic enrichement
Profiling
Metadata management
Persistence
Representation transformation
Matching
Entity resolution (or record linkage)
Integration
Semantic Alignement
Fusion
Abstraction
Refinement
Restructuring
Quality assessment
Elementary data
Statistics
Quality improvement
Elementary data
Statistics
Learning
Supervised
Unsupervised
Reinforcement
Interpretation
Validation
Hypothesis testing
Statistical analysis
Sensitivity analysis
Model validation
Value driven analysis
Correlation
Mining
Discovery
Learning
Supervised
Unsupervised
Reinforcement
Prediction (predictive modeling)
Forecasting
Visualization
Optimization
Acquisition
Analysis
1

4. Data Quality and Data Integration among the 3Vs
• Volume
• Velocity
• Variety

5. Data Quality Dimension Clusters*
1. Accuracy, correctness, validity and precision focus on the adherence of data to a given reality of interest. Including time
related issues
2. Completeness, pertinence and relevance refer to the capability of representing all and only the relevant aspects of the
reality of interest
3. Redundancy, minimality, compactness and conciseness refer to the capability of representing the reality of interest
with the minimal use of informative resources
4. Readability, comprehensibility, clarity and simplicity refer to ease of understanding of data by users.
5. Accessibility and availability are related to the ability of the user to access data from his or her culture, physical
status/functions, and technologies available
6. Consistency, cohesion and coherence refer to the capability of data to comply without contradictions to all properties
of the reality of interest, as specified in terms of integrity constraints, data edits, business rules and other formalisms
7. Trust, including believability, reliability and reputation, catching how much data derive from an authoritative source
Relational data
Traditional information systems
DQ Clusters &
cluster dimensions
2
* From Data Quality to Big Data Quality. J. Database Manag. 2015. C. Batini, A. Rula, M. Scannapieco, G. Viscusi

6. Role of Structural Characteristics – Linked Data
How do data quality dimensions and metrics evolve in the transformation
process from tabular to linked data?
Data type Structural characteristics
Linked Data
• Dereferenceable Resource
• SPARQL Endpoint
• RDF dump
• Interlinking
• Licensing
Relational data
Types of Big Data
3
Enables us managing the huge variability of methods and techniques
needed to manage data quality in linked data

7. Dimensions and their Evolution
Completeness
Structural characteristics
Dereferenceable
Resource
SPARQL
Endpoint
RDF dumps Interlinking Licensing
Quality
dimension
Accessibility
Resource
accessibility
Dataset
accessibility
Browsing
accessibility
Integration
accessibility
Reuse
accessibility
LD completenesses
dimensions
Accuracy
LD accuracy
dimensions

8. Toward a Cost Effectiveness Model
Management
Interpretation
Modeling
Semantic enrichement
Profiling
Metadata management
Persistence
Representation transformation
Matching
Entity resolution (or record linkage)
Integration
Semantic Alignement
Fusion
Abstraction
Refinement
Restructuring
Quality assessment
Elementary data
Statistics
Quality improvement
Elementary data
Statistics
…
From relational to RDF
Consider the four steps together

9. Two Possible Workflows
P…. G….
alppg ttygb
gnnn null
P… G….
alpbg ttygt
gnggr eryuui
Record
linkage
Transfor-
mation &
Alignment
P…. G….
alppg ttygb
gnnn null
kkkyyt vvvy
Quality management
P…. G….
Alppg ttygb
Gnnn null
P… G….
alpbg ttygt
gnggr eryuui
Transfor-
mation
Quality management
Align-
ment

10. Strategy of Workflow 1
10
• Exploit syntactic distance between concepts
• Differentiate distance:
–for alphanumeric strings use edit distance
–for lists of items use Jaccard distance
P…. G….
alppg ttygb
gnnn null
P… G….
alpbg ttygt
gnggr eryuui
Record
linkage
Transfor-
mation &
Alignment
P…. G….
alppg ttygb
gnnn null
kkkyyt vvvy
Quality management

11. Running Example – Relational Data
11
Id Denomination Istat Code Type
126759 Torgian 54053 Historic Place
125007 Saint Peter Abbey 246 Building
Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint Peter null Religious Building
Tourist attraction
Place

12. Running Example – Attributes Selection
12
Id Denomination Istat Code Type
126759 Torgian 54053 Historic Place
125007 Saint Peter Abbey 246 Building
Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint Peter null Religious Building
Tourist attraction
Place

13. First pair – Edit Distance
13
Id Denomination Istat Code Type
126759 Torgian 54053 Historic Place
125007 Saint Peter Abbey 246 Building
Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint Peter null Religious Building
Tourist attraction
Place
Edit-d = 1
YES!

14. Second Pair – Jaccard Distance
14
Id Denomination Istat Code Type
126759 Torgian 54053 Historic Place
125007 Saint Peter Abbey 246 Building
Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint Peter null Religious Building
Tourist attraction
Place
Jaccard =
1 - 2/3 =
1/3
YES!

15. Third Pair – Jaccard Distance
15
Id Denomination Istat Code Type
126759 Torgian 54053 Historic Place
125007 Saint Peter Abbey 246 Building
Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint Peter null Religious Building
Tourist attraction
Place
Jaccard =
1 - 2/5 =
3/5
NO!

16. Workflow 1 – Outcome
16
Id Denomination Istat Code Type
126759 Torgian 54053 Historic Place
125007 Saint Peter Abbey 246 Building
Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint Peter null Religious Building
Tourist attraction
Place
Final outcome:
1 pair true positive
1 pair false positive
1 pair false negative
Completeness
assessment:
1 unsolvable null

17. Strategy of Workflow 2
17
• Exploit semantic distance between concepts
• Improve completeness through DBpedia
P…. G….
alppg ttygb
gnnn null
P… G….
alpbg ttygt
gnggr eryuui
Transformation/
Semantic
Enrichment
Alignment/
Matching
http://dbpedia.org/resource/St._Peter
“Saint Peter”
label
location
http://dbpedia.org/resource/Sicily
37
lat
14
long
http://example/resource/ 125970
246 “Saint Peter Abbey”
istatCode label
Match/
Non-Match

18. Id Denomination Istat Code Type
126759 Torgian 54053 Historic
Place
125007 Saint Peter Abbey 246 Building
Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint
Peter
null Religious
Building
Running Example – Model Transformation & Alignment 1
18
http://example/resource/126759
“Torgian”
54053
type
istatCode
denomination
HistoricPlace
PopulatedPlace
http://dbpedia.org/resource/Torgiano
“Torgiano”
type
label
Tourist attraction
Place

19. PopulatedPlace
http://dbpedia.org/resource/Torgiano
“Torgiano”
type
label
http://example/resource/126759
Id Denomination Istat Code Type
126759 Torgian 54053 Historic
Place
125007 Saint Peter Abbey 246 Building
Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint
Peter
null Religious
Building
“Torgian”
54053
type
istatCode
label
HistoricPlace
Running Example – Model Transformation & Alignment 1
Alignement
Tourist attraction
Place

20. PopulatedPlace
http://dbpedia.org/resource/Torgiano
“Torgiano”
type
label
http://example/resource/126759
Id Denomination Istat Code Type
126759 Torgian 54053 Historic
Place
125007 Saint Peter Abbey 246 Building
Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint
Peter
null Religious
Building
“Torgian”
54053
type
istatCode
label
HistoricPlace
Running Example – Model Transformation & Alignment 1
20
Yes!
Tourist attraction
Place

21. Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint
Peter
null Religious
Building
Running Example – Model Transformation & Alignment 2
21
Id Denomination Istat Code Type
126759 Torgian 54053 Historic
Place
125007 Saint Peter Abbey 246 Building
http://example/resource/ 125970
246 “Saint Peter Abbey”
istatCode
label
http://dbpedia.org/resource/St._Peter_Basilica
“Basilica of Saint Peter”
label
Tourist attraction
Place
http://dbpedia.org/resource/St._Peter
“Saint Peter”
label
location
http://dbpedia.org/resource/Sicily
37
lat
14
long

22. http://dbpedia.org/resource/St._Peter
http://example/resource/ 125970
246 “Saint Peter Abbey”
istatCode
label
“Saint Peter”
label
ArchitecturalStructure HistoricPlace
Monument
PopulatedPlace
Building
ReligiousBuilding
Place
…
Region
AdministrativeRegion HistoricRegion
HistoricBuilding
DBpedia Ontology
type
Running Example – Typing
22
location
http://dbpedia.org/resource/Sicily
http://dbpedia.org/resource/St._Peter_Basilica
“Basilica of Saint Peter”
label
http://dbpedia.org/resource/Vatican_City
location
41
lat
12
long
37
lat
14
long
type
type

23. ArchitecturalStructure HistoricPlace
Monument
PopulatedPlace
Building
ReligiousBuilding
Place
…
Region
AdministrativeRegion HistoricRegion
HistoricBuilding
DBpedia Ontology
http://example/resource/ 125970
246 “Saint Peter Abbey”
istatCode
label
type
Running Example – typing
23
http://dbpedia.org/resource/St._Peter_Basilica
“Basilica of Saint Peter”
label
http://dbpedia.org/resource/Vatican_City
location
41
lat
12
long
http://dbpedia.org/resource/St._Peter
“Saint Peter”
label location
http://dbpedia.org/resource/Sicily
37
lat
14
long
type
type
Distance = 5
No!

24. ArchitecturalStructure HistoricPlace
Monument
PopulatedPlace
Building
ReligiousBuilding
Place
…
Region
AdministrativeRegion HistoricRegion
HistoricBuilding
type
DBpedia Ontology
Distance = 1
http://example/resource/ 125970
http://dbpedia.org/resource/St._Peter
246 “Saint Peter Abbey”
istatCode
label
“Saint Peter”
label
type
Workflow 2 – Outcome
24
location
http://dbpedia.org/resource/Sicily
http://dbpedia.org/resource/St._Peter_Basilica
“Basilica of Saint Peter”
label
http://dbpedia.org/resource/Vatican_City
location
41
lat
12
long
37
lat
14
long
type
Yes!
Final outcome:
2 pairs true positive
1 pair true negative
0 pairs false positive
0 pair false negative

25. Solving Missing Data
25
http://dbpedia.org/resource/St._Peter_Basilica
“Basilica of Saint Peter”
label
http://dbpedia.org/resource/Vatican_City
location
41
lat
12
long
Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint
Peter
null Religious Building
Place
Id Label Location Type
1 Torgiano Torgiano Populated Place
2 Saint Peter Sicily Historic Region
3 Basilica of Saint
Peter
Vatican City Religious Building

26. Conclusions and Futur Works
26
• Integration followed by transformation if no domain knowledge is needed
• Transformation followed by integration is better when:
– Aligning attributes by using existing vocabularies
– Distance between concepts is defined
– Enriching the dataset with the missing values
• Data integration of may depend on how good is the transformation
outcome
• Define it as an optimization problem
- Define functions for measuring the cost and the gain
- The cost and gain are different for different integration models

27. Thank you! Questions?
27