by Irini Fundulaki GRAPH-TA 2016

1. Holistic Benchmarking of Big Linked Data
HOBBIT
Irini Fundulaki
Institute of Computer Science - FORTH
Greece
4th Graph-TA
Barcelona, Spain, March 3, 2016
Irini Fundulaki (FORTH) HOBBIT March 3, 2016 1 / 9

2. Voluminous Datasets
1
1http://www.ibmbigdatahub.com/infographic/four-vs-big-data
Irini Fundulaki (FORTH) HOBBIT March 3, 2016 2 / 9

3. Large number of Linked Data Technologies
2
2https://cdn.datafloq.com/cms/os_big_data_open_source_tools-v2.png
Irini Fundulaki (FORTH) HOBBIT March 3, 2016 3 / 9

4. Issues to address for Linked Data Technologies
√
Which are the critical steps of the Linked
Data lifecycle?
√
Which are the current bottlenecks?
√
Which are the available market solutions?
√
Which are the relevant key performance
indicators?
√
What is the performance of existing tools
w.r.t. relevant performance indicators?
Irini Fundulaki (FORTH) HOBBIT March 3, 2016 4 / 9

5. Holistic Benchmarking of Big Linked Data
(HOBBIT)
HOBBIT focuses on Big Linked Data and intends to cover the
business-critical steps of the Linked Data lifecycle
√
Objectives
Gather real Performance indicators and Performance thresholds from
industry
Develop benchmarks based on real data
Provide universal benchmarking platform on standardized hardware to
provide comparable results
Provide periodic benchmarking challenges and reporting
Found independent Hobbit association
Irini Fundulaki (FORTH) HOBBIT March 3, 2016 5 / 9

6. HOBBIT Benchmarks
√
Generation and Acquisition: will test the performance of the systems that
implement approaches for extracting RDF data from
semi-structured data streams such as sensor data (smart metering) and
unstructured data streams (Twitter, RSS feeds, etc.)
√
Analysis and Processing
linking benchmark will test the performance of instance matching
methods for Linked Data
analytics benchmark will test the performance of supervised and
unsupervised machine learning methods
Irini Fundulaki (FORTH) HOBBIT March 3, 2016 6 / 9

7. HOBBIT Benchmarks
√
Storage and Curation
storage benchmark will test the performance storage components and
small storage subsystems
versioning benchmark will examine how versioning systems perform
w.r.t. size of the multi-version repository and the efficiency of
cross-snapshot query answering
√
Visualization and Services: will test the performance of systems regarding
question answering and
faceted browsing.
Irini Fundulaki (FORTH) HOBBIT March 3, 2016 7 / 9

8. HOBBIT Overview
Irini Fundulaki (FORTH) HOBBIT March 3, 2016 8 / 9

9. HOBBIT Logistics
√
Horizon 2020, 3 years duration (01/12/2016–30/11/2018)
√
Coordinator: Dr. Axel-Cyrille Ngonga-Ngomo, Institute of Applied
Informatics, Germany
√
Partners:
Institute for applied informatics (InfAI), Germany
Fraunhofer IAIS, Germany
Foundation for Research and Technology (FORTH), Greece
National Center for Scientific Research "Demokritos" (NCSR), Greece
iMinds, Belgium
USU Software AG, Germany
ONTOS AG, Switzerland
OpenLink, UK
AGT Group R&D GMBH, Germany
TomTom, Poland
Irini Fundulaki (FORTH) HOBBIT March 3, 2016 9 / 9