SlideShare a Scribd company logo
1 of 21
Benchmarking Faceted Browsing Capabilities
of Triple Stores
Horizon 2020
GA No 688227
01/12/2015 – 30/11/2018
Henning Petzka, Claus Stadler, Georgios Katsimpras, Bastian Haarmann, Jens Lehmann
13.09.2017
SEMANTiCS Amsterdam 2017
HOllistic Benchmarking of Big lInked daTa
Rationale:
A community-driven unified benchmarking platform for the community
• Focus on Big Linked Data
• Provide benchmarks and baselines
• Provide reference implementation of KPIs
• Extensible and referenceable
• Result analysis
• Open Source
http://project-hobbit.eu
Platform Overview
• Benchmarks I: Generation & Acquisition
measures performance of SPARQL query processing systems when faced with streams of
data in terms of efficiency and completeness
• Benchmarks II: Analysis & Processing
test performance on instance matching tools for Linked Data and performance on machine
learning methods for data analytics
• Benchmarks III: Storage & Curation
has its focus on storage components and versioning systems to efficiently manage evolving
linked datasets
• Benchmarks IV: Visualization & Services
has its focus on benchmarks regarding question answering and faceted browsing.
Faceted Browsing
stands for a session-based and state-dependent
interactive method for query formulation over a multi-
dimensional information space.
A browsing scenario consists of applying (or removing) filter restrictions defined by
object-valued properties or of changing the range of a property value of various data
types.
[Google Shopping]
Faceted Browsing - Example
Faceted Browsing - Example
[Google Shopping]
Choke Points
! In a browsing scenario it is the efficient transition
from one state to next one that determines the user
experience !
Three basic types of transition
1. Class-based transition
2. Property- or property path-based transition
3. Entity type switch
Choke Points
We collected a list of 14 choke points:
The underlying dataset
Scenarios
• make sense in a real-world browing scenario and
• cover all types of transitions as specified by the choke points
Key Performance Indicators
• Instance retrieval:
• Query-per-second score
• Precision
• Recall
• F1-Score
• Facet counts:
• Query-per-second score
• Several metrics for accuracy
Over all queries and for each choke point
individually
MOCHA Challenge at ESWC 2017
Benchmark on Faceted Browsing was part of the
Mighty Storage Challenge at the ESWC 2017
Two participants vs. baseline system
• QUAD by Ontos
• Virtuoso 8.0 Commercial Edition (beta release)
vs. Virtuoso 7.2 Open-Source Edition
No results for QUAD due to time out.
Preliminary results
Georgala, Spasic, Jovanovik, Petzka, Röder, Ngonga Ngomo. MOCHA2017: The Mighty Storage Challenge at ESWC 2017, ESWC challenge proceedings (Springer)
Problems for generic solutions
Dependency issue
Problems for generic solutions
Non-changing transitions
Problems for generic solutions
Tree-based transitions
Thank you!
http://project-hobbit.eu https://twitter.com/hobbit_project
This work was supported by grants from the EU H2020 Framework Programme provided for the
project HOBBIT(GA no. 688227).

More Related Content

Similar to Benchmarking Faceted Browsing Capabilities of Triple Stores

Similar to Benchmarking Faceted Browsing Capabilities of Triple Stores (20)

Technology Breakout – Chris Cameron, Elemica: “Achieving Greater Supply Chain...
Technology Breakout – Chris Cameron, Elemica: “Achieving Greater Supply Chain...Technology Breakout – Chris Cameron, Elemica: “Achieving Greater Supply Chain...
Technology Breakout – Chris Cameron, Elemica: “Achieving Greater Supply Chain...
 
AI projects - Lifecyle & Best Practices
AI projects - Lifecyle & Best PracticesAI projects - Lifecyle & Best Practices
AI projects - Lifecyle & Best Practices
 
Network and IT Ops Series: Build Production Solutions
Network and IT Ops Series: Build Production Solutions Network and IT Ops Series: Build Production Solutions
Network and IT Ops Series: Build Production Solutions
 
Building a Real-Time Security Application Using Log Data and Machine Learning...
Building a Real-Time Security Application Using Log Data and Machine Learning...Building a Real-Time Security Application Using Log Data and Machine Learning...
Building a Real-Time Security Application Using Log Data and Machine Learning...
 
Integrated production dash board for mining operations a PoV
Integrated production dash board for mining operations a PoVIntegrated production dash board for mining operations a PoV
Integrated production dash board for mining operations a PoV
 
SA 2014 - Integrating the heterogeneous enterprise
SA 2014 - Integrating the heterogeneous enterpriseSA 2014 - Integrating the heterogeneous enterprise
SA 2014 - Integrating the heterogeneous enterprise
 
Webinar: 10-Step Guide to Creating a Single View of your Business
Webinar: 10-Step Guide to Creating a Single View of your BusinessWebinar: 10-Step Guide to Creating a Single View of your Business
Webinar: 10-Step Guide to Creating a Single View of your Business
 
Assessing New Databases– Translytical Use Cases
Assessing New Databases– Translytical Use CasesAssessing New Databases– Translytical Use Cases
Assessing New Databases– Translytical Use Cases
 
Best Practices in Recommender System Challenges
Best Practices in Recommender System ChallengesBest Practices in Recommender System Challenges
Best Practices in Recommender System Challenges
 
Furuyama - analysis of factors that affect productivity
Furuyama - analysis of factors that affect productivityFuruyama - analysis of factors that affect productivity
Furuyama - analysis of factors that affect productivity
 
Outsourcing your share point hosting the cloud's fine print magnified
Outsourcing your share point hosting   the cloud's fine print magnifiedOutsourcing your share point hosting   the cloud's fine print magnified
Outsourcing your share point hosting the cloud's fine print magnified
 
10-Step Methodology to Building a Single View with MongoDB
10-Step Methodology to Building a Single View with MongoDB10-Step Methodology to Building a Single View with MongoDB
10-Step Methodology to Building a Single View with MongoDB
 
Implementing Advanced Analytics Platform
Implementing Advanced Analytics PlatformImplementing Advanced Analytics Platform
Implementing Advanced Analytics Platform
 
WP4-QoS Management in the Cloud
WP4-QoS Management in the CloudWP4-QoS Management in the Cloud
WP4-QoS Management in the Cloud
 
Database Trends in Retail and Ecommerce [Webcast]
Database Trends in Retail and Ecommerce [Webcast]Database Trends in Retail and Ecommerce [Webcast]
Database Trends in Retail and Ecommerce [Webcast]
 
ATAGTR2017 Bee-Hive approach for Big Data Testing [End to End Continuous Test...
ATAGTR2017 Bee-Hive approach for Big Data Testing [End to End Continuous Test...ATAGTR2017 Bee-Hive approach for Big Data Testing [End to End Continuous Test...
ATAGTR2017 Bee-Hive approach for Big Data Testing [End to End Continuous Test...
 
Webinar: The Slippery Slope of Migrating to SharePoint Online or On-Premise
Webinar: The Slippery Slope of Migrating to SharePoint Online or On-PremiseWebinar: The Slippery Slope of Migrating to SharePoint Online or On-Premise
Webinar: The Slippery Slope of Migrating to SharePoint Online or On-Premise
 
An intro to building an architecture repository meta model and modeling frame...
An intro to building an architecture repository meta model and modeling frame...An intro to building an architecture repository meta model and modeling frame...
An intro to building an architecture repository meta model and modeling frame...
 
Decision Matrix for IoT Product Development
Decision Matrix for IoT Product DevelopmentDecision Matrix for IoT Product Development
Decision Matrix for IoT Product Development
 
Mortgage Data for Machine Learning Algorithms
Mortgage Data for Machine Learning AlgorithmsMortgage Data for Machine Learning Algorithms
Mortgage Data for Machine Learning Algorithms
 

More from Holistic Benchmarking of Big Linked Data

More from Holistic Benchmarking of Big Linked Data (20)

EARL: Joint Entity and Relation Linking for Question Answering over Knowledge...
EARL: Joint Entity and Relation Linking for Question Answering over Knowledge...EARL: Joint Entity and Relation Linking for Question Answering over Knowledge...
EARL: Joint Entity and Relation Linking for Question Answering over Knowledge...
 
Benchmarking Big Linked Data: The case of the HOBBIT Project
Benchmarking Big Linked Data: The case of the HOBBIT ProjectBenchmarking Big Linked Data: The case of the HOBBIT Project
Benchmarking Big Linked Data: The case of the HOBBIT Project
 
Assessing Linked Data Versioning Systems: The Semantic Publishing Versioning ...
Assessing Linked Data Versioning Systems: The Semantic Publishing Versioning ...Assessing Linked Data Versioning Systems: The Semantic Publishing Versioning ...
Assessing Linked Data Versioning Systems: The Semantic Publishing Versioning ...
 
The DEBS Grand Challenge 2018
The DEBS Grand Challenge 2018The DEBS Grand Challenge 2018
The DEBS Grand Challenge 2018
 
SQCFramework: SPARQL Query Containment Benchmarks Generation Framework
SQCFramework: SPARQL Query Containment Benchmarks Generation FrameworkSQCFramework: SPARQL Query Containment Benchmarks Generation Framework
SQCFramework: SPARQL Query Containment Benchmarks Generation Framework
 
LargeRDFBench: A billion triples benchmark for SPARQL endpoint federation
LargeRDFBench: A billion triples benchmark for SPARQL endpoint federationLargeRDFBench: A billion triples benchmark for SPARQL endpoint federation
LargeRDFBench: A billion triples benchmark for SPARQL endpoint federation
 
The DEBS Grand Challenge 2017
The DEBS Grand Challenge 2017The DEBS Grand Challenge 2017
The DEBS Grand Challenge 2017
 
4th Natural Language Interface over the Web of Data (NLIWoD) workshop and QAL...
4th Natural Language Interface over the Web of Data (NLIWoD) workshop and QAL...4th Natural Language Interface over the Web of Data (NLIWoD) workshop and QAL...
4th Natural Language Interface over the Web of Data (NLIWoD) workshop and QAL...
 
Scalable Link Discovery for Modern Data-Driven Applications (poster)
Scalable Link Discovery for Modern Data-Driven Applications (poster)Scalable Link Discovery for Modern Data-Driven Applications (poster)
Scalable Link Discovery for Modern Data-Driven Applications (poster)
 
An Evaluation of Models for Runtime Approximation in Link Discovery
An Evaluation of Models for Runtime Approximation in Link DiscoveryAn Evaluation of Models for Runtime Approximation in Link Discovery
An Evaluation of Models for Runtime Approximation in Link Discovery
 
Scalable Link Discovery for Modern Data-Driven Applications
Scalable Link Discovery for Modern Data-Driven ApplicationsScalable Link Discovery for Modern Data-Driven Applications
Scalable Link Discovery for Modern Data-Driven Applications
 
Extending LargeRDFBench for Multi-Source Data at Scale for SPARQL Endpoint F...
 Extending LargeRDFBench for Multi-Source Data at Scale for SPARQL Endpoint F... Extending LargeRDFBench for Multi-Source Data at Scale for SPARQL Endpoint F...
Extending LargeRDFBench for Multi-Source Data at Scale for SPARQL Endpoint F...
 
SPgen: A Benchmark Generator for Spatial Link Discovery Tools
SPgen: A Benchmark Generator for Spatial Link Discovery ToolsSPgen: A Benchmark Generator for Spatial Link Discovery Tools
SPgen: A Benchmark Generator for Spatial Link Discovery Tools
 
Introducing the HOBBIT platform into the Ontology Alignment Evaluation Campaign
Introducing the HOBBIT platform into the Ontology Alignment Evaluation CampaignIntroducing the HOBBIT platform into the Ontology Alignment Evaluation Campaign
Introducing the HOBBIT platform into the Ontology Alignment Evaluation Campaign
 
OKE2018 Challenge @ ESWC2018
OKE2018 Challenge @ ESWC2018OKE2018 Challenge @ ESWC2018
OKE2018 Challenge @ ESWC2018
 
MOCHA 2018 Challenge @ ESWC2018
MOCHA 2018 Challenge @ ESWC2018MOCHA 2018 Challenge @ ESWC2018
MOCHA 2018 Challenge @ ESWC2018
 
Dynamic planning for link discovery - ESWC 2018
Dynamic planning for link discovery - ESWC 2018Dynamic planning for link discovery - ESWC 2018
Dynamic planning for link discovery - ESWC 2018
 
Hobbit project overview presented at EBDVF 2017
Hobbit project overview presented at EBDVF 2017Hobbit project overview presented at EBDVF 2017
Hobbit project overview presented at EBDVF 2017
 
Leopard ISWC Semantic Web Challenge 2017
Leopard ISWC Semantic Web Challenge 2017Leopard ISWC Semantic Web Challenge 2017
Leopard ISWC Semantic Web Challenge 2017
 
Benchmarking Link Discovery Systems for Geo-Spatial Data - BLINK ISWC2017.
Benchmarking Link Discovery Systems for Geo-Spatial Data - BLINK  ISWC2017. Benchmarking Link Discovery Systems for Geo-Spatial Data - BLINK  ISWC2017.
Benchmarking Link Discovery Systems for Geo-Spatial Data - BLINK ISWC2017.
 

Recently uploaded

IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
Enterprise Knowledge
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
Earley Information Science
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
giselly40
 

Recently uploaded (20)

GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
Mastering MySQL Database Architecture: Deep Dive into MySQL Shell and MySQL R...
 
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptxEIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
EIS-Webinar-Prompt-Knowledge-Eng-2024-04-08.pptx
 
Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...Driving Behavioral Change for Information Management through Data-Driven Gree...
Driving Behavioral Change for Information Management through Data-Driven Gree...
 
presentation ICT roal in 21st century education
presentation ICT roal in 21st century educationpresentation ICT roal in 21st century education
presentation ICT roal in 21st century education
 
Presentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreterPresentation on how to chat with PDF using ChatGPT code interpreter
Presentation on how to chat with PDF using ChatGPT code interpreter
 
Scaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organizationScaling API-first – The story of a global engineering organization
Scaling API-first – The story of a global engineering organization
 
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
Raspberry Pi 5: Challenges and Solutions in Bringing up an OpenGL/Vulkan Driv...
 
Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024Finology Group – Insurtech Innovation Award 2024
Finology Group – Insurtech Innovation Award 2024
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot TakeoffStrategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
Strategize a Smooth Tenant-to-tenant Migration and Copilot Takeoff
 
How to convert PDF to text with Nanonets
How to convert PDF to text with NanonetsHow to convert PDF to text with Nanonets
How to convert PDF to text with Nanonets
 
Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024Partners Life - Insurer Innovation Award 2024
Partners Life - Insurer Innovation Award 2024
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time AutomationFrom Event to Action: Accelerate Your Decision Making with Real-Time Automation
From Event to Action: Accelerate Your Decision Making with Real-Time Automation
 
CNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of ServiceCNv6 Instructor Chapter 6 Quality of Service
CNv6 Instructor Chapter 6 Quality of Service
 

Benchmarking Faceted Browsing Capabilities of Triple Stores

  • 1. Benchmarking Faceted Browsing Capabilities of Triple Stores Horizon 2020 GA No 688227 01/12/2015 – 30/11/2018 Henning Petzka, Claus Stadler, Georgios Katsimpras, Bastian Haarmann, Jens Lehmann 13.09.2017 SEMANTiCS Amsterdam 2017
  • 2. HOllistic Benchmarking of Big lInked daTa Rationale: A community-driven unified benchmarking platform for the community • Focus on Big Linked Data • Provide benchmarks and baselines • Provide reference implementation of KPIs • Extensible and referenceable • Result analysis • Open Source http://project-hobbit.eu
  • 3.
  • 5.
  • 6.
  • 7. • Benchmarks I: Generation & Acquisition measures performance of SPARQL query processing systems when faced with streams of data in terms of efficiency and completeness • Benchmarks II: Analysis & Processing test performance on instance matching tools for Linked Data and performance on machine learning methods for data analytics • Benchmarks III: Storage & Curation has its focus on storage components and versioning systems to efficiently manage evolving linked datasets • Benchmarks IV: Visualization & Services has its focus on benchmarks regarding question answering and faceted browsing.
  • 8. Faceted Browsing stands for a session-based and state-dependent interactive method for query formulation over a multi- dimensional information space. A browsing scenario consists of applying (or removing) filter restrictions defined by object-valued properties or of changing the range of a property value of various data types.
  • 10. Faceted Browsing - Example [Google Shopping]
  • 11. Choke Points ! In a browsing scenario it is the efficient transition from one state to next one that determines the user experience ! Three basic types of transition 1. Class-based transition 2. Property- or property path-based transition 3. Entity type switch
  • 12. Choke Points We collected a list of 14 choke points:
  • 14. Scenarios • make sense in a real-world browing scenario and • cover all types of transitions as specified by the choke points
  • 15. Key Performance Indicators • Instance retrieval: • Query-per-second score • Precision • Recall • F1-Score • Facet counts: • Query-per-second score • Several metrics for accuracy Over all queries and for each choke point individually
  • 16. MOCHA Challenge at ESWC 2017 Benchmark on Faceted Browsing was part of the Mighty Storage Challenge at the ESWC 2017 Two participants vs. baseline system • QUAD by Ontos • Virtuoso 8.0 Commercial Edition (beta release) vs. Virtuoso 7.2 Open-Source Edition No results for QUAD due to time out.
  • 17. Preliminary results Georgala, Spasic, Jovanovik, Petzka, Röder, Ngonga Ngomo. MOCHA2017: The Mighty Storage Challenge at ESWC 2017, ESWC challenge proceedings (Springer)
  • 18. Problems for generic solutions Dependency issue
  • 19. Problems for generic solutions Non-changing transitions
  • 20. Problems for generic solutions Tree-based transitions
  • 21. Thank you! http://project-hobbit.eu https://twitter.com/hobbit_project This work was supported by grants from the EU H2020 Framework Programme provided for the project HOBBIT(GA no. 688227).