Ranking

1. | 0
Elsevier’s support to
Ranking agencies –
an update
Kavit Yagnik, Elsevier
Sptember 2019

2. | 1
The importance of Rankings

3. | 2
Source: University selection by students (IDP Research)
The goal of the program is that by 2020 at
least 5 Russian universities will be in the top
100 universities as denoted by QS World
University ranking.
Rankings are cited by many stakeholders as being
important
85% of students find university ranking as
important in their selection of institute to study
33% of students find university ranking as the
most important factor (number 1 factor, followed by
21% employer recognition, etc.)
Students & Parents
University Management
David Willets (former UK Universities & Science
minister):
“We broadly accept the criteria used by the THE,
which is why our policies are focused on the
same areas.”
Policy Makers

4. | 3
Students primarily use a small number of powerhouse
rankings systems, although there are significant
variations by nationality
Source: Hobsons, INTERNATIONAL STUDENT SURVEY 2015, Value and the Modern International Student

5. | 4
Bibliometric data providers vs. ranking agencies
unique to Scopus / SciVal shared with Clarivate

6. | 5
All rankings have their strengths and potential
disadvantages and we do not rank the rankings!
• We believe in working on fundamentals with a “basket of
indicators”, always as a complement to peer opinion
• Informed decisions are better decisions
• Metrics should complement, not replace human judgment
• Well-selected metrics drive positive behaviors
• Metrics does not only mean bibliometrics
• Metrics can help monitor and eliminate biases
• Assessments are costly, but availability of new tools help bring
cost down
• Data sources to cover humanities are becoming more
complete
Elsevier’s position on university rankings and metrics
in general

7. | 6
THE

8. | 7
NEW
NEW
NEW
Affiliations: Primarily SciVal institutions
Support:
Reputation data: Elsevier runs the reputation survey using Elsevier author
list for THE
Affiliation handling: Affiliation corrections, mergers, split, etc. handled with
THE for the respective universities
Source: THE

9. | 8
Process
Reputation
Oct-Nov Dec-Mar Ongoing-May
Preparation for the
reputation survey:
- Questionnaire
- Random selection
of survey invitees
(Scopus author
profiles)
- List of institutions
for selection (all
SciVal institutions)
Reputation Survey
run:
- Regular update to
THE re. response
rate
- Addressing
possible issue (e.g.
emails blocks)
Bibliometrics
Affiliation validation
and clean up
- THE list against
Elsevier (re)mapped
and checked for
completeness
- SciVal institutions are
default Elsevier profiles
- If not in SciVal, then
added into SciVal or
otherwise from Scopus
- Ongoing feedback
processing from
institutions
- Pontential
methodology changes
tested and validated
May-July
Bibliometric
computation:
- Using May-finalized
list, the
bibliometrics:
Citation Score, total
# of papers and
total # of
international papers
- Analysis of the
results, validation
and sign-off
WUR
Launch
Sep
World
University
Ranking
launch
27/07/2012
Milestone Description
Reputation Data
(responses) delivery
27/07/2012
Milestone Description
Bibliometric Data Delivery
1

10. | 9
Source: THE
Elsevier contributes by providing Scopus
bibliometric data
Elsevier contributes by conducting the
reputation survey: Randomly selected authors
from Scopus are invited to complete the survey

11. | 10
Minimum requirements in order to be ranked
Institutional
Data
uploaded
through the
THE data
portal
1,000
papers for
the 5 years
window
At least
150 papers
per year

12. TITLE OF PRESENTATION
| 11
11
|
Main changes between 2015 and 2018 in THE’s WUR
Threshold
of
papers
per
year
200
Threshold
of
papers
per
year
150
Threshold
of
papers
per
year
Unchanged
Threshold
of
papers
per
year
Unchanged
Threshold
per
subject
0
Threshold
per
subject
500/250
Threshold
per
subject
Unchanged
Threshold
per
subject
Unchanged
Kilo
papers
Excluded Kilo
papers
Re-introduced but
with fractionalized
counting
Kilo
papers
Unchanged
Kilo
papers
Unchanged
Journals
All
Journals
Suspended
titles excluded
Journals
Unchanged
Journals
Unchanged
Document
types
AR, RE and
CP
Document
types
AR, RE, CP,
BK and CH
Document
types
Unchanged
Document
types
Unchanged
Number
of
universities
801
Number
of
universities
980
Number
of
universities
1,103
Number
of
universities
1,258
Subjects
6
Subject
8
Subject
11 main
disciplines + 32
sub-disciplines Subject
Unchanged
2015 2017
2016 2018

13. | 12
THE’s WUR 2017-18 Scopus Citation dataset
ALL SCOPUS 2012-2016 Articles,
Reviews, Conf.
papers, Books
and Book
chapters
Suspended
titles carved out
12,431,514
12,737,678
14,061,956
67,775,649
Scopus – 1 May 2017
1

14. | 13
ALL
SUBJECTS
(11)
The THE WUR 2017-18 Scopus dataset
PERIOD
2012-2016
Engineering &
Technology
3.75M
Clinical, Pre-clinical
& Health
4.76M
Physical Sciences
4.23M
Social Sciences
927K
Arts & Humanities
692K
Life Sciences
2.65M
Business and
Economics
494K
Psychology
326K
Education
222K
Computer Science
1.61M
Law
87K
DOCUMENT TYPES
• Article
9.07M
• Article Review
720K
• Conference Proceeding
2.08M
• Books (series)
71K
• Book chapters
499K
Not included: 310,367 publications from 303 suspended titles
1

15. TITLE OF PRESENTATION
| 14
14
|
QS

16. | 15
NEW
Source: QS
Affiliations: Primarily Scopus institutions
Support:
Self-citations, subject mapping, etc.

17. | 16
Source: QS

18. | 17
Bibliometric parameters
Institutions
based
compiled
by QS
based on
Scopus
profiles
NTCC as
the core
citation
metric
Subject
mapping to
ASJC
developed
by QS
Scopus
data June
2016
snapshot

19. TITLE OF PRESENTATION
| 18
18
|
Main changes between 2015 and 2018 in QS’ WUR
Metrics
calculation
End Q1
Metrics
calculation
Q3
(~June/Jul
y)
Metrics
calculation
Q2 (April)
Metrics
calculation
Unchanged
Z-score
Calculated for the
entire population
Z-score
Using a fixed range
for z-score
calculations to limit
divergent effects
Z-score
Unchanged
Z-score
Unchanged
Kilo
papers
> 10 affiliations
excluded
Kilo
papers Variable threshold
per subject
implemented
Kilo
papers
Unchang
ed
Kilo
papers
Unchanged
Citation
Impact
NTCC
(introduced in 2015
for the first time)
Citation
Impact
NTCC
Citation
Impact
Unchang
ed
Citation
Impact
Unchanged
Document
types
AR RE CP BK
CH AIP BZ
Document
types
AR RE CP BK
CH AIP BZ
Document
types
Unchanged
Document
types
Unchanged
Number
of
universities
890
Number
of
Universities
916
Number
of
Universities
979
Number
of
Universities
1,021
Main
Subjects
5
Main
Subjects
5
Main
Subjects
Unchang
ed Main
Subjects
Unchanged
2015 2017
2016

20. | 25
www.elsevier.com/research-intelligence
The End
For questions: k.yagnik@elsevier.com