This document provides an agenda and overview for the 10th BIR Workshop on Bibliometric-enhanced Information Retrieval held on April 14, 2020 online. The workshop brings together researchers working at the intersection of information retrieval and bibliometrics. The agenda includes a keynote on metrics for assessing scientific impact and sessions on expert finding, citations, learning to rank, and evaluation. The document discusses the interdisciplinary nature of BIR research and provides additional reading and future research opportunities in the field.

1. 10th BIR Workshop
Bibliometric-enhanced Information Retrieval
Philipp Mayr
Ingo Frommholz
Guillaume Cabanac
April 14, 2020
ECIR 2020, Lisbon -> Online
Proceedings: http://ceur-ws.org/Vol-2591/
#bir2020
Happy birthday BIR!

2. 2
The BIR Team
BIR 2020
Philipp Mayr
@philipp_mayrIngo Frommholz
@ifromm
Guillaume Cabanac
@gcabanac
…and of course our authors, programme
committee and participants…thank you!

3. 3
B + IR
(Academic) Search
Information Retrieval
Exploratory Data Analysis
Scientometrics
BIR 2020

4. 4
What is B+IR?
BIR 2020
 Bibliometrics + IR (BIR): tackles issues related to academic search, at
the crossroads (intersection) between Information Retrieval and
Bibliometrics
Waltman, 2018
Typical BIR topics
• IR and recommendation tool
development and evaluation
• Bibliometric IR experiments
• Document Clustering for IR
• Citation Contexts and
Analysis
10.1016/j.joi.2015.12.003

5. 5
Past Activities
BIR 2020
 Workshops
 2014 BIR@ECIR 1st ed. Amsterdam, NL
 2015 BIR@ECIR 2nd ed. Vienna, AT
 2016 BIR@ECIR 3rd ed. Padua, IT
 2016 BIRNDL@JCDL 4th joint ed. Newark, US
 2017 BIR@ECIR 5th ed. Aberdeen, UK
 2017 BIRNDL@SIGIR 6th joint ed. Tokyo, JP
 2018 BIR@ECIR 7th ed. Grenoble, FR
 2019 BIR@ECIR 8th ed. Cologne, Germany
 2019 BIRNDL@SIGIR 9th joint ed. Paris, France
 2020 BIR@ECIR 10th ed. Lisbon, Portugal - ONLINE
 Special Issues
 2018 International Journal on Digital Libraries
 2018 Scientometrics
+ Ingo
+ Guillaume
+ BIRNDL
https://dblp.org/db/conf/bir
ws/index.html
BIR Bibliography:
https://bit.ly/BIR-Bib

6. 6
BIR is interdisciplinary
10.1007/978-3-030-45442-5_85
Wolfram, 2016
Common background
• Language-based methods
have greatly benefitted IR
and bibliometrics research
• Digital libraries represent
an ideal environment to
study the intersection

7. 7
BIR is relevant
COVID-19 Preprints (cumulative)
since January 2020
What has this to do with BIR?
Selecting relevant papers?
Selecting hot papers?
Avoid misinformation!
https://github.com/nicholasmfraser/covid19_preprints

8. 8
Further BIR reading
BIR 2020
 Overview paper “Bibliometric-enhanced Information Retrieval 10th
Anniversary Workshop Edition”
https://doi.org/10.1007/978-3-030-45442-5_85
 “Bibliometric-enhanced Information Retrieval: Connecting IR with
Bibliometrics” @european summer school for scientometrics
https://www.slideshare.net/Philipp_Mayr/bibliometricenhanced-information-retrieval-
connecting-ir-with-bibliometrics
 Our BIR Bibliography incl. special issues https://bit.ly/BIR-Bib

9. 9
Greeting Notes
BIR 2020
https://bit.ly/BIR-greetings
• Andrea Scharnhorst (DANS, Netherlands)
• Dietmar Wolfram (University of
Wisconsin-Milwaukee, USA)
• Suzan Verberne (Leiden University,
Netherlands)
• Mike Thelwall (University of
Wolverhampton, UK)
• Iana Atanassova (Université Bourgogne
Franche-Comté, France)
and Marc Bertin (University of Lyon,
France)
• Henry Small (SciTech Strategies, USA)
• Staša Milojević (Indiana University, USA)
• Min-Yen Kan (National University of
Singapore, Singapore)
• Ludo Waltman (Leiden University, The
Netherlands)
• Aparna Basu (South Asian University,
India)
• Wolfgang Glänzel (KU Leuven, Belgium)
• Akiko Aizawa (National Institute of
Informatics, Japan)
• Muthu Kumar Chandrasekaran (Amazon,
USA)
• Michel Zitt (INRA Nantes, France)
• Howard D. White (Drexel University, USA)

10. 10
Agenda: Morning
BIR 2020
09:30-
10:15
Keynote:
Metrics and trends in assessing the
scientific impact Georgios Tsatsaronis
Session: Expert finding and
ranking models
10:15-
10:35
New Datasets and a Benchmark of
Document Network Embedding
Methods for Scientific Expert Finding
Robin Brochier, Antoine
Gourru, Adrien Guille and
Julien Velcin
10:35-
10:55
Conference Indexing in Digital
Libraries: A Ranking Model and Case
Study on dblp
Christopher Michels,
Mandy Neumann, Philipp
Schaer and Ralf Schenkel
11:00-
11:20 Coffee break ☕
All times are Western European Summer Time (WEST, Porto time, UTC+1)

11. 11
Agenda: Morning
BIR 2020
Session: Citations, Citations,
Citations
11:20-
11:40
Usage and Citation Metrics for
Ranking Algorithms in Legal
Information Retrieval Systems
Gineke Wiggers and
Suzan Verberne
11:40-
12:00
Browsing citation clusters for
academic literature search: A
simulation study with systematic
reviews
Juan Pablo Bascur,
Suzan Verberne, Nees
Jan Van Eck and Ludo
Waltman
12:00-
12:20
Neural Citation Recommendation: A
Reproducibility Study
Michael Färber, Timo
Klein and Joan Sigloch
12:20-
13:00 Interactive demos (40min) / Coffee ☕

12. 12
Agenda: Afternoon
BIR 2020
Session: Learning to Rank and
Evaluation
13:00-
13:20
Learning to Rank Research Articles:
A case study of collaborative filtering
and learning to rank in ScienceDirect
Daniel Kershaw,
Benjamin Pettit, Maya
Hristakeva and Kris Jack
13:20-
13:40
Evaluating Pretrained Transformer
Models for Citation Recommendation
Rodrigo Nogueira,
Zhiying Jiang, Kyunghyun
Cho and Jimmy Lin
13:40-
14:00
Relations Between Relevance
Assessments, Bibliometrics and
Altmetrics
Timo Breuer, Philipp
Schaer and Dirk Tunger

13. 13
Agenda: Afternoon
BIR 2020
14:00-
15:00 Lunch Break / Keynote Joana de Sá (Room A)
15:15-
15:45 Discussion and further steps
15:45-
16:00 End + Greeting Notes

14. 14
Agenda: Morning
BIR 2020
KEYNOTE
Metrics and trends in assessing
the scientific impact
 George Tsatsaronis
http://ceur-ws.org/Vol-2591/paper-02.pdf

15. 15
The Future
BIR 2020
 Frontiers in Research Metrics and Analytics
Research Topic: Mining Scientific Papers Volume II
 https://www.frontiersin.org/research-topics/13388/mining-scientific-
papers-volume-ii-knowledge-discovery-and-data-exploitation
 Open call; extended versions of BIR papers highly appreciated
 Manuscript deadline: September 01, 2020
 SDP @ EMNLP 2020 – First Workshop on Scholarly
Document Processing (SDP)
 https://ornlcda.github.io/SDProc/
 Submissions deadline: July 15, 2020
 The workshop will consist of a research track and a shared
task track (6th edition of CL-SciSumm shared task and two
new summarization tasks – CL-LaySumm and LongSumm).

16. Thank you!
#bir2020