Presentation on the occasion of the 60th anniversary of the Econometric Institute at Erasmus University Rotterdam. Rotterdam, The Netherlands, May 27, 2016.

1. From econometrics to bibliometrics
Ludo Waltman
Centre for Science and Technology Studies, Leiden University
Econometrics and Beyond
Rotterdam
May 27, 2016

2. About myself
• 2003: BSc in Informatics & Economics
• 2005: MSc in Economics & Informatics
• 2005–2009: PhD student at Econometric Institute
• 2009–2016: Researcher at CWTS
1

3. About CWTS
• Centre for Science and Technology
Studies (CWTS)
• Research center at Leiden University
focusing on science and technology
studies
• Strong emphasis on bibliometric and
scientometric research
• About 30 staff members
• 25th anniversary in 2015
2

4. Current activities
• Leading the Advanced Bibliometric Methods research
program at CWTS
– Indicators
– Network analysis
– Text mining
– Visualization
• Bibliometric consultancy for CWTS BV
• Coordinating the CWTS Leiden Ranking
• Deputy director CWTS
• Editor-in-Chief Journal of Informetrics
3

5. CWTS Leiden Ranking
4

6. From econo-
metrics to
bibliometrics
5

7. From econometrics to bibliometrics: Two
examples
• Visualization of network data
• Indicator design
6

8. Visualization of
network data
7

9. 2006: 50 Years of Econometrics
8

10. 2006: Multidimensional scaling vs.
visualization of similarities (VOS)
9

11. 2007: Concept map viewer
10

12. 2009: VOSviewer 0.9.0
11

13. 2016: VOSviewer 1.6.4
12

14. Co-authorship network based on EUR
economics and business publications
13

15. Publications using VOSviewer
14
Visualizations of bibliometric networks nowadays are
produced mostly using the VOSviewer software and the
VOS technique; MDS is hardly used anymore

16. Indicator design
15

17. Bibliometric indicators
• Bibliometric indicators:
– Scientific activity
– Scientific impact
– Collaboration
– Interdisciplinarity
– Etc.
• Bibliometric indicators tend to be justified based on ad
hoc arguments rather than rigorous systematic
reasoning
16

18. CWTS ‘crown indicator’
• For each publication i of a research group, we know the
actual number of citations ci and the ‘expected’ number
of citations ei
• On average, how often have the publications of a
research group been cited relative to expectation?
17
crown indicator=
𝑐1+⋯+𝑐 𝑛
𝑒1+⋯+𝑒 𝑛

19. Consistency
• Suppose group A and B have the same number of
publications
• According to some indicator, group A is performing better
than B
• Suppose group A and B jointly produce a new publication
• Then, if our indicator is consistent, group A should still be
performing better than B
18

20. Inconsistency of crown indicator
• Crown indicator group A: 25 / 10 = 2.50
• Crown indicator group B: 10 / 5 = 2.00
• Group A and B co-author a new publication
• Crown indicator group A: 65 / 20 = 3.25
• Crown indicator group B: 50 / 15 = 3.33
• Ranking of group A and B has reversed
19
𝑐𝑖 𝑒𝑖
Pub. 1 Group A 25 10
Pub. 2 Group B 10 5
Pub. 3 Group A and B 40 10
𝑐𝑖 𝑒𝑖
Pub. 1 Group A 25 10
Pub. 2 Group B 10 5

21. Solving crown indicator inconsistency by
changing order of operations
• Inconsistent crown indicator:
• Consistent crown indicator:
20
𝑐1 + ⋯ + 𝑐 𝑛
𝑒1 + ⋯ + 𝑒 𝑛
1
𝑛
𝑐1
𝑒1
+ ⋯ +
𝑐 𝑛
𝑒 𝑛

22. Adoption in major analytical tools
21

23. 22
h-index
A scientist has index h if h of his papers have at least h
citations each and the other papers have at most h
citations each

24. 23
Consistency requirement
If two scientists achieve the same performance
improvement, their ranking relative to each other should
remain unchanged

25. 24
Inconsistency of the h-index

26. From econo-
metrics to
bibliometrics
25

27. From econometrics to bibliometrics:
Conclusions
• Bibliometrics is a quantitative field of science, but level of
sophistication of quantitative analyses is much lower
than in econometrics
• Background from Econometric Institute has been very
helpful to increase level of sophistication of bibliometric
research
• Illustrates the need to move toward an integrated data
science field, combining methods from econometrics,
statistics, operations research, computer science,
network science, etc.
26

28. Congratulations to the
Econometric Institute!
27
Thank you for your attention