Download to read offline
Uploaded byHolistic Benchmarking of Big Linked Data
Workshop Report Benchmarking Linked Data
The workshop report outlines key questions and insights regarding the benchmarking of linked data tools, emphasizing the need for scalable application stacks and realistic benchmarks. It highlights the emergence of benchmarking platforms and the importance of community-driven efforts for improving tool performance. The report calls for the establishment of a benchmark repository and addresses the necessity to tackle real-world challenges.
More Related Content
Similar to Workshop Report Benchmarking Linked Data
More from Holistic Benchmarking of Big Linked Data
Workshop Report Benchmarking Linked Data
- 1. Workshop Report Benchmarking LinkedData Axel-Cyrille Ngonga Ngomo Irini Fundulaki Anastasia Krithara Horizon 2020 GA No 688227 01/12/2016–30/11/2018 ISWC 2016 Kobe, Japan November 10, 2016 Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 1 / 5
- 2. Core Questions Core questions 1Developers: How good is my tool? 2 Vendors: Who is my tool good for? 3 Users: Which tool(s) should I use for my application? Many questions 1 Tools ⇒ Bottlenecks, solved problems, distinctive features 2 Benchmarks ⇒ Novel bechmarks, limitations, updates 3 Performance indicators ⇒ Limitations, correlations, innovations 4 Comparative studies ⇒ Across benchmarks, fields of application Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 2 / 5
- 3. Core Questions Goal ofBLINK Provide a forum where the community can gather to address these challenges. Core questions 1 Developers: How good is my tool? 2 Vendors: Who is my tool good for? 3 Users: Which tool(s) should I use for my application? Many questions 1 Tools ⇒ Bottlenecks, solved problems, distinctive features 2 Benchmarks ⇒ Novel bechmarks, limitations, updates 3 Performance indicators ⇒ Limitations, correlations, innovations 4 Comparative studies ⇒ Across benchmarks, fields of application Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 2 / 5
- 4. Highlights Insights 1 Scalable applicationstacks for Linked Data in demand (Keynote) 2 Increasing number of realistic benchmarks (Kotsev et al., Spasic et al.) 3 Benchmarking as motor for improvements (Saveta et al., BioASQ, GERBIL) Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 3 / 5
- 5. Highlights Insights 1 Scalable applicationstacks for Linked Data in demand (Keynote) 2 Increasing number of realistic benchmarks (Kotsev et al., Spasic et al.) 3 Benchmarking as motor for improvements (Saveta et al., BioASQ, GERBIL) Positive 1 Emerging benchmarking platforms (GERBIL QA, HOBBIT) 2 Increase of tool performance with benchmarks (OAEI, BioASQ, GERBIL) 3 Uptake of methodology-based and data-driven benchmark design Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 3 / 5
- 6. Highlights Insights 1 Scalable applicationstacks for Linked Data in demand (Keynote) 2 Increasing number of realistic benchmarks (Kotsev et al., Spasic et al.) 3 Benchmarking as motor for improvements (Saveta et al., BioASQ, GERBIL) Positive 1 Emerging benchmarking platforms (GERBIL QA, HOBBIT) 2 Increase of tool performance with benchmarks (OAEI, BioASQ, GERBIL) 3 Uptake of methodology-based and data-driven benchmark design Todo 1 Need for benchmark repository (CKAN) 2 Community effort necessary for uptake of more benchmarks (HOBBIT) 3 Untackled real-world challenges (benchmarks coming up) Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 3 / 5
- 7. That’s all Folks! @hobbit_project AxelNgonga AKSW Research Group Augustusplatz 10, Room P905 04109 Leipzig, Germany ngonga@informatik.uni-leipzig.de @NgongaAxel Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 4 / 5
- 8. Acknowledgment This work wassupported by grants from the EU H2020 Framework Programme provided for the project HOBBIT (GA no. 688227). Ngonga Ngomo et al. (InfAI) BLINK November 10, 2016 5 / 5