Glasgow Reading Group Invited Talk - Causality-driven Adhoc IR.pdf

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Causality-driven Ad-hoc Information Retrieval
Suchana Datta
PhD Student
University College Dublin
Under the supervision of
Dr. Derek Greene (University College Dublin)
&
Dr. Debasis Ganguly (University of Glasgow)
23rd
May, 2022
S. Datta (UCD) CausalIR 23rd May 1 / 39

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Outline
1 Causal Information Retrieval - the Challenges
2 A Factored Causal Relevance Model (SIGIR’20)
3 A ‘Pointwise-Query, Listwise-Document’-based QPP Approach (SIGIR’22)
4 Concluding Remarks

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Causal Information Retrieval - the Challenges

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
What are we interested in?

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
May I ask you a question?

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
What does Google say?

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
What does Google tell us?

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Let’s make the query more specific

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Where is the ’WHY’?

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Is a traditional search system adequate for retrieving causally relevant
information?

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Is a traditional search system adequate for retrieving causally relevant
information?
Is a new research paradigm required to address the requirements of
identifying causally-relevant information (i.e., Causal Information
Retrieval)?

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Dataset Construction
https://www.telegraphindia.com
http://fire.irsi.res.in/fire/static/data

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Nature of Causal Documents
Term overlaps between topical
and causal set of documents
Cosine similarities between topical
and causal documents

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Initial Investigations

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
A Factored Causal Relevance Model (SIGIR’20)

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
We Propose - FCRLM
A feedback model to estimate a distribution of terms which are relatively
infrequent but associated with high weights in the topically relevant
distribution, leading to potential causal relevance.
– Datta, S., Ganguly, D., Roy, D., Bonin, F., Jochim, C. and Mitra, M., 2020,
July. Retrieving potential causes from a query event. In Proceedings of the 43rd
International ACM SIGIR Conference on Research and Development in
Information Retrieval (pp. 1689-1692).

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Model Architecture

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Model Architecture
Available at : https://github.com/suchanadatta/Factored-Causal-RLM.git

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
How FCRLM Works?

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
FCRLM - Provides us Causally Relevant Information

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Performance of FCRLM

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
CAIR - A Shared Task
For Dataset :
https://github.com/suchanadatta/CAIR-
DataSet.git
Datta, S., Ganguly, D., Roy, D., Greene,
D., Jochim, C. and Bonin, F., 2020,
December. Overview of the
Causality-driven Adhoc Information
Retrieval (CAIR) task at FIRE-2020. In
Forum for Information Retrieval Evaluation
(pp. 14-17).
Datta, S., Ganguly, D., Roy, D., Greene,
D., 2021, December. Overview of the
Causality-driven Adhoc Information
Retrieval (CAIR) task at FIRE-2021. In
Forum for Information Retrieval Evaluation
(pp. 25-27).

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
We Want More Fine-grained Information

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
We Want More Fine-grained Information
Datta, S., Greene, D., Ganguly, D., Roy, D. and Mitra, M., 2020. Where’s the
Why? In Search of Chains of Causes for Query Events. In AICS (pp. 109-120).

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Which piece of text is a Query?
QPP - Predicting the quality of retrieved documents to satisfy the
information needs behind the query.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
A ‘Pointwise-Query, Listwise-Document’-based QPP Approach (SIGIR’22)
A ‘Pointwise-Query, Listwise-Document’-based QPP Approach
(SIGIR’22)

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
We propose - qppBERT-PL
An end-to-end neural cross-encoder-based approach - trained pointwise on
individual queries, but listwise over the top ranked documents (split into
chunks).
– Datta, S., MacAvaney, S., Ganguly, D., Greene, D. A ‘Pointwise-Query,
Listwise-Document’based Query Performance Prediction Approach (to appear in
the proceedings of SIGIR’22).

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
What do we Propose? - qppBERT-PL
A novel architecture and objective function for a pointwise neural QPP.
Transformed the pointwise QPP objective into a classification task, not a
regression model.
Models the top-ranked documents as a sequence of chunks (Listwise), not
as a whole set.
Incorporates the relative Positions (or ranks) of the top documents.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
End-to-end Architecture of qppBERT-PL
Popular unsupervised
QPP methods (e.g.
NQC, WIG) work well
when information used
from the top-100
documents.
Encoding long
sequences of 100
documents is likely to
be noisy.
Top-ranked set is
segmented into equal
sized partitions
(chunks).

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
BERT-based
cross-encoder is used to
model the interactions
between the query and
the document terms of
each chunk.
LSTM-encoded
representation of this
interaction sequence.
Ranks are encoded via
BERT positional
embeddings.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Passed through a fully
connected layer (FC).
Terminates at a p + 1
dimensional Softmax
representing the
probability of finding r
relevant documents
within this p-sized
chunk
(r ∈ {0, 1, . . . , p}).

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Compute a weighted
average from the
outputs of the network,
predicted for each
p-sized partition of the
top documents.
Aggregated scores are
used to sort the queries
in descending order.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Compute a weighted
average from the
outputs of the network,
predicted for each
p-sized partition of the
top documents.
Aggregated scores are
used to sort the queries
in descending order.
Available at : https://github.com/suchanadatta/qppBERT-PL.git

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Performance of qppBERT-PL
qppBERT-PL is more effective at predicting query performance than
other supervised and unsupervised methods.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Performance of qppBERT-PL
Sequence modeling, chunking and Rank Embeddings are critical
components of qppBERT-PL.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Concluding Remarks
Concluding Remarks

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Concluding Remarks
Concluding Remarks
FCRLM - a feedback model to estimate a distribution of terms which are
relatively infrequent but associated with high weights in the topically relevant
distribution, leading to potential causal relevance.
We would like to incorporate adaptive feedback as a feature that would help
users deciding whether or not to use feedback.
qppBERT-PL - the first contribution in QPP that transforms the pointwise
QPP objective into a classification task.
We are interested in exploring ways to aggregate information from short
passages and predict QPP scores for longer documents.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
Concluding Remarks
Thank you!
Many thanks to the IR Group of University of Glasgow for inviting me.
For any questions you may have, please e-mail me at :
suchana.datta@ucdconnect.ie

Glasgow Reading Group Invited Talk - Causality-driven Adhoc IR.pdf

More Related Content

Similar to Glasgow Reading Group Invited Talk - Causality-driven Adhoc IR.pdf

More from suchanadatta3

Recently uploaded

Glasgow Reading Group Invited Talk - Causality-driven Adhoc IR.pdf