Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
1/20
Introduction Method Experiments Conclusions
Sequential Query Expansion
using Concept Graph
Saeid Balaneshin-kordan Al...
2/20
Introduction Method Experiments Conclusions
Introduction
Method
Experiments
Conclusions
Balaneshin, Kotov Wayne State...
3/20
Introduction Method Experiments Conclusions
Introduction
Method
Experiments
Conclusions
Balaneshin, Kotov Wayne State...
4/20
Introduction Method Experiments Conclusions
Concept Graph - Example
§ Query: poach wildlife preserve.
§ Concept Graph...
4/20
Introduction Method Experiments Conclusions
Concept Graph - Example
§ Query: poach wildlife preserve.
§ Concept Graph...
4/20
Introduction Method Experiments Conclusions
Concept Graph - Example
§ Query: poach wildlife preserve.
§ Concept Graph...
4/20
Introduction Method Experiments Conclusions
Concept Graph - Example
§ Query: poach wildlife preserve.
§ Concept Graph...
4/20
Introduction Method Experiments Conclusions
Concept Graph - Example
§ Query: poach wildlife preserve.
§ Concept Graph...
4/20
Introduction Method Experiments Conclusions
Concept Graph - Example
§ Query: poach wildlife preserve.
§ Concept Graph...
5/20
Introduction Method Experiments Conclusions
Challenges
§ The number of candidate concepts to evaluate increases
expon...
6/20
Introduction Method Experiments Conclusions
Optimization Problem
§ Objective Function: total number of evaluated conc...
7/20
Introduction Method Experiments Conclusions
Introduction
Method
Experiments
Conclusions
Balaneshin, Kotov Wayne State...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
8/20
Introduction Method Experiments Conclusions
Proposed Method
(1,1)
(1,2)
(1,3)
(2,1)
(2,2)
(2,3)
(2,4) (2,5)
(2,6)
(3,...
9/20
Introduction Method Experiments Conclusions
Summary of the proposed method and the baselines
Method
Optimization Prob...
10/20
Introduction Method Experiments Conclusions
Baselines (With Fixed Number of Layers)
Selection
Region Selection
Regio...
11/20
Introduction Method Experiments Conclusions
Introduction
Method
Experiments
Conclusions
Balaneshin, Kotov Wayne Stat...
12/20
Introduction Method Experiments Conclusions
Inexpensive Features
§ Retrieval score of the highest ranked document co...
13/20
Introduction Method Experiments Conclusions
Expensive Features
§ Avg co-occurrence of C(i,j) with query concepts
§ M...
14/20
Introduction Method Experiments Conclusions
Feature Ablation on ROBUST04 collection
hgstDocScore
maxTCooccur
maxCooc...
15/20
Introduction Method Experiments Conclusions
The impact of the upper and lower thresholds on
MAP (i.e., βU and βL) at...
15/20
Introduction Method Experiments Conclusions
The impact of the upper and lower thresholds on
MAP (i.e., βU and βL) at...
16/20
Introduction Method Experiments Conclusions
Retrieval Performance
Col.
Method
Concept Layer
1st 2nd 3rd 4th
TREC7-8 ...
16/20
Introduction Method Experiments Conclusions
Retrieval Performance
Col.
Method
Concept Layer
1st 2nd 3rd 4th
ROBUST04...
16/20
Introduction Method Experiments Conclusions
Retrieval Performance
Col.
Method
Concept Layer
1st 2nd 3rd 4th
GOV Meth...
17/20
Introduction Method Experiments Conclusions
Retrieval Performance
Without PRF Concepts
Collection
Evaluation
QL SDM
...
17/20
Introduction Method Experiments Conclusions
Retrieval Performance
With PRF Concepts
Collection
Evaluation
RM LCE
Met...
18/20
Introduction Method Experiments Conclusions
Introduction
Method
Experiments
Conclusions
Balaneshin, Kotov Wayne Stat...
19/20
Introduction Method Experiments Conclusions
Conclusions
§ The main contribution of this work:
§ A two-stage method f...
19/20
Introduction Method Experiments Conclusions
Conclusions
§ Experimental evaluation indicates that:
§ The proposed met...
Many Thanks to the ACM SIGIR Student Travel Grant!
20/20
Upcoming SlideShare
Loading in …5
×

Sequential Query Expansion using Concept Graph

215 views

Published on

Manually and automatically constructed concept graphs (or semantic networks), in which the nodes correspond to words or phrases and the typed edges designate semantic relationships between words and phrases, have been previously shown to be rich sources of effective latent concepts for query expansion. However, finding good expansion concepts for a given query in large and dense concept graphs is a challenging problem, since the number of candidate concepts that are related to query terms and phrases and need to be examined increases exponentially with the distance from the original query concepts. In this paper, we propose a two-stage feature-based method for sequential selection of the most effective concepts for query expansion from a concept graph. In the first stage, the proposed method weighs the concepts according to different types of computationally inexpensive features, including collection and concept graph statistics. In the second stage, a sequential concept selection algorithm utilizing more expensive features is applied to find the most effective expansion concepts at different distances from the original query concepts. Experiments on TREC datasets of different type indicate that the proposed method achieves significant improvement in retrieval accuracy over state-of-the-art methods for query expansion using concept graphs.

Published in: Science
  • Be the first to comment

  • Be the first to like this

Sequential Query Expansion using Concept Graph

  1. 1. 1/20 Introduction Method Experiments Conclusions Sequential Query Expansion using Concept Graph Saeid Balaneshin-kordan Alexander Kotov Wayne State University October 25, 2016 Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  2. 2. 2/20 Introduction Method Experiments Conclusions Introduction Method Experiments Conclusions Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  3. 3. 3/20 Introduction Method Experiments Conclusions Introduction Method Experiments Conclusions Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  4. 4. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  5. 5. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  6. 6. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  7. 7. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  8. 8. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  9. 9. 4/20 Introduction Method Experiments Conclusions Concept Graph - Example § Query: poach wildlife preserve. § Concept Graph: ConceptNet 5 § The first number in parenthesis indicates concept layer, the second number is the index of a concept in the concept layer. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  10. 10. 5/20 Introduction Method Experiments Conclusions Challenges § The number of candidate concepts to evaluate increases exponentially with the number of layers to consider. § Only a small fraction of hundreds or potentially thousands of concepts that can improve retrieval results, while others need to be discarded to avoid noise and concept drift. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  11. 11. 6/20 Introduction Method Experiments Conclusions Optimization Problem § Objective Function: total number of evaluated concepts Constraint: precision of retrieval results min ˜Cut k " kÿ i=1 Ni * s.t. E(˜RΛ; T) ą θQ , § Approximate Solution: Decision Criterion Select concept C(i,j) & If ˜Qr(C(i,j)) ě βUcontinue with the same concept layer Discard concept C(i,j) & If βL ď ˜Qr(C(i,j)) ă βUcontinue with the same concept layer Discard concept C(i,j) & If ˜Qr(C(i,j)) ă βLmove to the next concept layer Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  12. 12. 7/20 Introduction Method Experiments Conclusions Introduction Method Experiments Conclusions Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  13. 13. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  14. 14. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  15. 15. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  16. 16. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action Select Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  17. 17. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action Select Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  18. 18. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinue Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  19. 19. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinue Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  20. 20. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelect Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  21. 21. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelect Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  22. 22. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinue Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  23. 23. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscard Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  24. 24. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscard Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  25. 25. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinue Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  26. 26. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelect Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  27. 27. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelectDiscard Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  28. 28. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelectDiscard Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  29. 29. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelectDiscardDiscard Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  30. 30. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelectDiscardDiscardSTOP Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  31. 31. 8/20 Introduction Method Experiments Conclusions Proposed Method (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qs(c) Stage I: Initial Concept Sorting Sort Concepts at Each Layer according to ˜Qs(C(i,j)) Selection Region Rejection Region Selection Region Uncertainty Region Rejection Region Selection Region Uncertainty Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.200 0.225 0.250 0.275 0.300 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qr(c) Stage II: Sequential Concept Selection Decision Criterion Select concept & ˜Qr(C(i,j)) ě βU continue Discard concept & βL ď ˜Qr(C(i,j)) ă βU continue Discard concept & ˜Qr(C(i,j)) ă βL move to next layer Action SelectContinueSelectContinueDiscardContinueSelectDiscardDiscardSTOP Number of Evaluated Concepts: 11 (26% less) Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  32. 32. 9/20 Introduction Method Experiments Conclusions Summary of the proposed method and the baselines Method Optimization Problem Criteria in the Approximate Solution Objective Constraint Selecting Rejecting Stopping Method A mint řk i=0 Liu E( ˜Rk Λ; T) ą θ Qb(c) ą βQ Qb(c) ă βQ i ą k Method B maxtE( ˜Rk Λ; T)u řk i=0 Li ă θ Ii(c) ă βI Ii(c) ą βI i ą k Method C mint řk i=0 Liu E( ˜Rk Λ; T) ą θ Qb(c) ą βQ Qb(c) ă βQ i ą k Method D maxtE( ˜Rk Λ; T)u řk i=0 Li ă θ I(c) ă βI I(c) ą βI i ą k Proposed mint řk i=0 Niu E( ˜Rk Λ; T) ą θ Qr(c) ą βU Qr(c) ă βL Li = 0 § Qb(C(i,j)): Quality measure computed as a linear weighted combination of the feature functions. § I(c): Index of a concept in the sorted set of concepts § Li: Number of selected concepts from the i-th concept layer. § the set of features used to calculate the quality measure Qb(c) for the baselines is the same as the set of features used to calculate Qs(c) in for our proposed method. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  33. 33. 10/20 Introduction Method Experiments Conclusions Baselines (With Fixed Number of Layers) Selection Region Selection Region Rejection Region Selection Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qb(c) Selection Region Selection Region Rejection Region Selection Region Rejection Region Rejection Region (1,1) (1,2) (1,3) (2,1) (2,2) (2,3) (2,4) (2,5) (2,6) (3,1) (3,2) (3,3) (4,1) (4,2) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qb(c) Selection Region Rejection Region (1,1) (3,1) (2,1) (1,2) (2,2) (3,2) (2,3) (2,4) (2,5) (1,3) (3,3) (4,1) (4,2) (2,6) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qb(c) Selection Region Rejection Region (1,1) (3,1) (2,1) (1,2) (2,2) (3,2) (2,3) (2,4) (2,5) (1,3) (3,3) (4,1) (4,2) (2,6) (4,3) 0.24 0.28 0.32 0.36 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 concepts Qb(c) SingleThreshold onEachLayer SingleThreshold onAllLayers A B C D Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  34. 34. 11/20 Introduction Method Experiments Conclusions Introduction Method Experiments Conclusions Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  35. 35. 12/20 Introduction Method Experiments Conclusions Inexpensive Features § Retrieval score of the highest ranked document containing C(i,j) § Avg retrieval score of all documents containing C(i,j) § Variance of retrieval score of all documents containing C(i,j) § Avg retrieval scores of the top documents containing C(i,j) § Number of occurrences of C(i,j) in the top documents § Number of top documents containing C(i,j) § Node degree of C(i,j) in the concept graph § Avg number of paths between C(i,j) and query concepts § Max number of paths between C(i,j) and query concepts Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  36. 36. 13/20 Introduction Method Experiments Conclusions Expensive Features § Avg co-occurrence of C(i,j) with query concepts § Max co-occurrence of C(i,j) with query concepts § Avg co-occurrence of C(i,j) with query concepts in top-docs § Max co-occurrence of C(i,j) with query concepts in top-docs § Avg co-occurrence of C(i,j) with at least a pair of query concepts in top-docs § Max co-occurrence of C(i,j) with at least a pair of query concepts in top-docs § Avg co-occurrence of C(i,j) with all previously selected concepts in top-docs § Max co-occurrence of C(i,j) with all previously selected concepts in top-docs § Avg co-occurrence of C(i,j) with selected concepts in concept layer i ´ 1 § Max co-occurrence of C(i,j) with selected concepts in concept layer i ´ 1 § Avg co-occurrence of C(i,j) with selected concepts in concept layer i ´ 1 in top-docs § Max co-occurrence of C(i,j) with selected concepts in concept layer i ´ 1 in top-docs § Avg mutual information of C(i,j) with at least a pair of query concepts in top-docs § Max mutual information of C(i,j) with at least a pair of query concepts in top-docs § Avg mutual information of C(i,j) with selected concepts in concept layer i ´ 1 in top-docs § Max mutual information of C(i,j) with selected concepts in concept layer i ´ 1 in top-docs Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  37. 37. 14/20 Introduction Method Experiments Conclusions Feature Ablation on ROBUST04 collection hgstDocScore maxTCooccur maxCooccurL* nodeDegree maxTCooccurL* maxNumLinks maxTCooccur* maxCooccur* varDocScore avgTCooccurL* avgCooccurL* maxTMiL* avgCooccur* avgTMiL* avgNumLinks maxTCooccurP* avgTCooccur* maxTMiP* avgDocScore avgTDocScore avgTCooccurP* avgTCooccur avgTMiP* docFreqTpDoc termFreqTpDoc none 0.15 0.20 0.25 0.30 MAP Feature The feature that results in the highest reduction of MAP after being removed from the feature set: § The features that are utilized in both stages of the proposed method § The features that are dependent on the collection Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  38. 38. 15/20 Introduction Method Experiments Conclusions The impact of the upper and lower thresholds on MAP (i.e., βU and βL) at the concept layer i = 2 (e) TREC7-8 (f) ROBUST04 (g) GOV § Upper threshold < the optimum value: more non-useful concepts are added to the candidate list of expansion concepts. § Upper threshold > the optimum value: some useful concepts may not be selected as expansion concepts. § Lower threshold < the optimum value: the selection process may terminate earlier and a number of useful concepts may not be examined at all. § Lower threshold > the optimum value: the proposed method will evaluate more concepts in total, which is against its main objective. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  39. 39. 15/20 Introduction Method Experiments Conclusions The impact of the upper and lower thresholds on MAP (i.e., βU and βL) at the concept layer i = 2 (h) TREC7-8 (i) ROBUST04 (j) GOV § Overall, although the upper and lower thresholds are dependent on each other, the upper threshold has the main effect on the accuracy of selected concepts, while the lower threshold has the main effect on the number of examined concepts. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  40. 40. 16/20 Introduction Method Experiments Conclusions Retrieval Performance Col. Method Concept Layer 1st 2nd 3rd 4th TREC7-8 Method D-HAL 0.2220 0.2239 0.2155 0.2120 Method D-CNet 0.2205 0.2245 0.2214 0.2183 Method C-HAL 0.2152 0.2227 0.2185 0.2133 Method C-CNet 0.2182 0.2265 0.2225 0.2218 Method B-HAL 0.2207 0.2171 0.2266 0.2236 Method B-CNet 0.2188 0.2294 0.2255 0.2294 Method A-HAL 0.2172 0.2251 0.2290 0.2282 Method A-CNet 0.2183 0.2290 0.2329 0.2335 Proposed-HAL 0.2249 0.2348 0.2418 0.2457 Proposed-CNet 0.2222 0.2377 0.2449 0.2484 SDM 0.2124 —— —— —— § Best performing baseline: Method A § Methods with multiple thresholds tend to perform better. § The methods using ConceptNet-based concept graph (CNet) obtain higher MAP than the ones using HAL. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  41. 41. 16/20 Introduction Method Experiments Conclusions Retrieval Performance Col. Method Concept Layer 1st 2nd 3rd 4th ROBUST04 Method D-HAL 0.2660 0.2644 0.2569 0.2554 Method D-CNet 0.2640 0.2651 0.2568 0.2555 Method C-HAL 0.2675 0.2655 0.2608 0.2516 Method C-CNet 0.2637 0.2628 0.2683 0.2695 Method B-HAL 0.2684 0.2718 0.2598 0.2535 Method B-CNet 0.2616 0.2710 0.2665 0.2675 Method A-HAL 0.2614 0.2758 0.2757 0.2764 Method A-CNet 0.2689 0.2732 0.2851 0.2793 Proposed-HAL 0.2721 0.2786 0.2865 0.2898 Proposed-CNet 0.2748 0.2814 0.2889 0.2963 SDM 0.2359 —— —— —— § Best performing baseline: Method A § Methods with multiple thresholds tend to perform better. § The methods using ConceptNet-based concept graph (CNet) obtain higher MAP than the ones using HAL. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  42. 42. 16/20 Introduction Method Experiments Conclusions Retrieval Performance Col. Method Concept Layer 1st 2nd 3rd 4th GOV Method D-HAL 0.2337 0.2428 0.2355 0.2319 Method D-CNet 0.2348 0.2396 0.2355 0.2382 Method C-HAL 0.2404 0.2406 0.2459 0.2322 Method C-CNet 0.2416 0.2451 0.2378 0.2379 Method B-HAL 0.2359 0.2466 0.2418 0.2397 Method B-CNet 0.2420 0.2452 0.2484 0.2421 Method A-HAL 0.2434 0.2442 0.2491 0.2420 Method A-CNet 0.2365 0.2455 0.2524 0.2422 Proposed-HAL 0.2455 0.2429 0.2570 0.2578 Proposed-CNet 0.2449 0.2514 0.2575 0.2591 SDM 0.2184 —— —— —— § Best performing baseline: Method A § Methods with multiple thresholds tend to perform better. § The methods using ConceptNet-based concept graph (CNet) obtain higher MAP than the ones using HAL. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  43. 43. 17/20 Introduction Method Experiments Conclusions Retrieval Performance Without PRF Concepts Collection Evaluation QL SDM Method A Method A Proposed Proposed Metric HAL CNet HAL CNet TREC7-8 MAP 0.1982 0.2124 0.2282˚: 0.2335˚: 0.2457˚: 0.2484˚: (7.44%) (9.93%) (15.68%/7.67%) (16.95%/6.38%) P@20 0.3540 0.3765 0.3762 0.3783 0.3785˚ 0.3796˚ (-0.08%) (0.48%) (0.53%/0.61%) (0.82%/0.34%) ROBUST04 MAP 0.2359 0.2510 0.2764˚: 0.2851˚: 0.2898˚: 0.2963˚: (10.12%) (13.59%) (15.46%/4.85%) (18.05%/3.93%) P@20 0.3339 0.3667 0.3679 0.3773˚: 0.3802˚: 0.3795˚: (0.33%) (2.89%) (3.68%/3.34%) 3.49%/0.58% GOV MAP 0.2184 0.2333 0.2491˚ 0.2524˚: 0.2578˚: 0.2591˚: (6.77%) (8.19%) (10.5%/3.49%) 11.06%/2.65% P@20 0.0389 0.0451 0.0476 0.0493˚ 0.0558˚: 0.0552˚: (5.54%) (9.31%) (23.73%/17.23%) 22.39%/11.97% § Method A provides a significant improvement over SDM in the 5 cases. § Although the parameters are estimated with the goal of maximizing MAP, the proposed method demonstrates significant improvement over the baselines (QE and SDM) also in terms of P@20. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  44. 44. 17/20 Introduction Method Experiments Conclusions Retrieval Performance With PRF Concepts Collection Evaluation RM LCE Method A* Method A* Proposed* Proposed* Metric HAL CNet HAL CNet TREC7-8 MAP 0.2151 0.2423 0.2503˚ 0.2558˚: 0.2642˚: 0.2672˚: (3.3%) (5.57%) (9.04%/5.55%) 10.28%/4.46% P@20 0.3641 0.3836 0.3883 0.3927˚ 0.3934˚: 0.4035˚: (1.23%) (2.37%) (2.55%/1.31%) 5.19%/2.75% ROBUST04 MAP 0.2683 0.2826 0.2935˚ 0.2979˚ 0.3034˚: 0.3053˚: (3.86%) (5.41%) (7.36%/3.37%) 8.03%/2.48% P@20 0.3561 0.3785 0.3826˚ 0.3834˚ 0.3893˚: 0.3965˚: (1.08%) (1.29%) (2.85%/1.75%) 4.76%/3.42% GOV MAP 0.2403 0.2678 0.2693 0.2730˚ 0.2793˚: 0.2811˚: (0.56%) (1.94%) (4.29%/3.71%) 4.97%/2.97% P@20 0.0483 0.0566 0.0583 0.0617˚ 0.0706˚ 0.0720˚: (3.00%) (9.01%) (24.73%/21.1%) 27.21%/16.69% § The proposed method has significant improvements over the baselines QL and SDM whether the concept graph is generated by HAL or ConceptNet. § Method A provides a significant improvement over LCE only in one of the cases when it uses PRF concepts. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  45. 45. 18/20 Introduction Method Experiments Conclusions Introduction Method Experiments Conclusions Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  46. 46. 19/20 Introduction Method Experiments Conclusions Conclusions § The main contribution of this work: § A two-stage method for sequential selection of effective concepts for query expansion from the concept graph. § The optimization problem of the proposed method: § Objective: having least possible number of candidate concepts § Constraint: achieve a given precision of retrieval results. § Stages of the proposed method: § First Stage: the candidate concepts are sorted using a number of computationally inexpensive features. § Second Stage: This sorting is utilized in the second stage to sequentially select expansion concepts by using computationally expensive features. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  47. 47. 19/20 Introduction Method Experiments Conclusions Conclusions § Experimental evaluation indicates that: § The proposed method outperforms state-of-the-art baselines, which instead of minimizing the number of evaluated concepts, aim to minimize the number of selected concepts or maximize a concept quality measure. § The proposed method and the baselines produce more accurate results using ConceptNet-based than collection-based concept graph. § Future Work: § We believe that applying the proposed method to the case of entity-based queries and knowledge graphs is an interesting future direction for extending this work. Balaneshin, Kotov Wayne State University Sequential Query Expansion using Concept Graph
  48. 48. Many Thanks to the ACM SIGIR Student Travel Grant! 20/20

×