The document discusses the SQCFramework for query containment, which includes methods for selection, normalization, and clustering of SPARQL queries to generate benchmarks. It evaluates various approaches and metrics relating to query performance and benchmark generation using real data, emphasizing the framework's flexibility and customization for specific use cases. Results indicate that similarity error is inversely proportional to benchmark size, and the proposed framework can effectively handle query logs while supporting benchmarking for the Semantic Web.

1. Muhammad Saleem , Claus Stadler, Qaiser Mehmood, Jens
Lehmann, Axel-Cyrille Ngonga Ngomo
(K-Cap 2017, Austin, USA)
AKSW, University of Leipzig, Germany
DICE, University of Paderborn, Germany
SDA, University of Bonn, Germany
1

2.  Query containment
 Why SQCFramework?
 SQCFramework
 Input queries
 Important query features
 Benchmark generation
 Benchmark personalization
 Evaluation and results
 Conclusion
2

3. Deciding whether the result set of one query is
included in the result set of another?
3
Formally:

4.  Query optimization
 Caching mechanisms
 Data integration
 View maintenance
 Query rewriting
4

5.  Real data
 Real log queries
 Flexible
 Customizable
 Use-case specific
5

6. 6
SPARQL
queries
Selection
criteria
Containment
benchmark
1. Selection of super-queries
2. Normalization of feature vectors
3. Generation of clusters
4. Selection of most representative queries

7.  Manually provided by user
 Selection from LSQ
 Linked SPARQL Queries datasets
 Extracted from endpoint queries log
 Structural and data-driven statistics
7
20 datasets available from (http://hobbitdata.informatik.uni-leipzig.de/lsq-dumps/)

8.  Number of entailments/sub-queries
 Number of projection variables
 Number of BGPs
 Number of triple patterns
 Max. number BGP triple patterns
 Min. number BGP triple patterns
 Number of join vertices
 Mean join vertex degree
 Number of LSQ features
8

9. 1. Selection of super-queries
2. Normalized feature vectors
3. Generation of clusters
4. Selection of most representative queries
9

10. 10

11. 11
2
2
1
5
5
5
3
2.3
2
 Number of entailments/sub-queries
 Number of projection variables
 Number of BGPs
 Number of triple patterns
 Max. number BGP triple patterns
 Min. number BGP triple patterns
 Number of join vertices
 Mean join vertex degree
 Number of LSQ features
10
8
6
12
5
10
10
5
30
0.2
0.25
0.16
0.41
1
0.5
0.33
0.46
0.06
Feature vector Max. feature vector Normalized feature vector
F M F/M

12.  FEASIBLE
 FEASIBLE-Exemplars
 KMeans++
 DBSCAN+KMeans++
 Agglomerative
 Random selection
12

13. 13
Plot normalized feature vectors in a multidimensional space
Query F1 F2
Q1 0.2 0.2
Q2 0.5 0.3
Q3 0.8 0.3
Q4 0.9 0.1
Q5 0.5 0.5
Q6 0.2 0.7
Q7 0.1 0.8
Q8 0.13 0.65
Q9 0.9 0.5
Q10 0.1 0.5
Suppose we need a benchmark of 3 queries
Q1
Q2 Q3
Q4
Q5
Q6
Q7
Q8
Q9Q10
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1

14. Q1
Q2 Q3
Q4
Q5
Q6
Q7
Q8
Q9Q10
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1
14

15. 15
Q1
Q2 Q3
Q4
Q5
Q6
Q7
Q8
Q9Q10
Avg.
Avg.
Avg.
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1
Calculate Average across each cluster

16. 16
Q1
Q2 Q3
Q4
Q5
Q6
Q7
Q8
Q9Q10
Avg.
Avg.
Avg.
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1
Calculate distance of each point in cluster to the average

17. 17
Q1
Q2 Q3
Q4
Q5
Q6
Q7
Q8
Q9Q10
Avg.
Avg.
Avg.
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
0,8
0,9
0 0,1 0,2 0,3 0,4 0,5 0,6 0,7 0,8 0,9 1
Select minimum distance query as the final benchmark
query from that cluster
Purple, i.e., Q2 is the final selected query from yellow cluster

18.  Number of projection variables in the
super-queries should be at most 2
 Number of BGPs should be greater than 1 or
the number of triple patterns should be
greater than 3
 Benchmark should be selected from the
most recently executed 1000 queries
18

19. 19
 Similarity error
 Diversity score
L is the query log, B is the benchmark,
and k is the set of all features
 We compared
 FEASIBLE
 FEASIBLE-Exemplars
 KMeans++
 DBSCAN+KMeans++
 Random selection
 Number of containment tests (#T)
 Benchmark generation time (G) in sec

20. 20
 Query Mixes per Hour (QMpH)
 Number of handled test cases
 Number of timed out test cases
 We compared
 TreeSolver
 AFMU
 SPARQL-Algebra
 JSAC
We generated benchmarks using Semantic Web
Dog Food (SWDF) and DBpedia queries logs

21. 21
0
0,01
0,02
0,03
0,04
0,05
15 25 50 75 100 125
SIMILARITYERROR
#SUPER QUERIES
FEASIBLE KMeans++
DBScan+KMeans++ Random
FEASIBLE-Exemplars
0
0,01
0,02
0,03
0,04
0,05
0,06
0,07
2 4 6 9 12 15
SIMILARITYERROR
#SUPER QUERIES
FEASIBLE KMeans++
DBScan+KMeans++ Random
FEASIBLE-Exemplars
(SWDF) (DBpedia)
• Similarity error is inversely (in general) proportional to benchmark size
• Random selection in general generates benchmarks of smaller similarity
errors

22. 22
(SWDF) (DBpedia)
• Diversity score is inversely (in general) proportional to benchmark size
• FEASIBLE-Exemplars generates the more diverse benchmarks
0
0,05
0,1
0,15
0,2
0,25
0,3
0,35
0,4
15 25 50 75 100 125
DIVERSITYSCORE
#SUPER QUERIES
FEASIBLE KMeans++
DBScan+KMeans++ Random
FEASIBLE-Exemplars
0
0,1
0,2
0,3
0,4
0,5
2 4 6 9 12 15
DIVERSITYSCORE
#SUPER QUERIES
FEASIBLE KMeans++
DBScan+KMeans++ Random
FEASIBLE-Exemplars

23. 23
• Not significant differences

24. 24
0
0,05
0,1
0,15
0,2
0,25
0,3
0,35
0,4
0,45
0,5
NormalizedS.D.
SQCFrameWork-FEASIBLE-Exemplars
SQC-Benchmark
• SQCFrameWork-FEASIBLE-Exemplars is more diverse
across majority of the query features
*SQC-Benchmark:http://sparql-qc-bench.inrialpes.fr/

25. 25
• JSAC correctly handled all cases in with reasonable QMpH
0
0,5
1
1,5
2
QMpH
TreeSolver AFMU
JSAC SPARQL-Algebra
Total
Tests
#Handled
Tests
#Correct
Test
#Timeout
Tests
TreeSolver 1192 5 5 2
AFMU 1192 5 5 12
SPARQL-Algebra 1192 0 0 0
JSAC 1192 1192 1192 0

26. 26
 SQCFramework:
 Based on real data, real log queries
 Flexible
 Customizable
 Use-case specific
 Similarity error is inversely (in general) proportional to benchmark size
 Random selection in general generates benchmarks of smaller similarity errors
 Diversity score is inversely (in general) proportional to benchmark size
 FEASIBLE-Exemplars generates the more diverse benchmarks
 JSAC correctly handled all cases in with reasonable QMpH
 SQCFramework available from (https://github.com/dice-group/sqcframework)

27. 27

28. This work was supported by grants from the EU H2020 Framework Programme
provided for the project HOBBIT (GA no. 688227).

29. Thanks !
saleem@informatik.uni-leipzig.de
29