1. 1
Academic Ranking of World Universities
Methodologies and Problems
August, 2008August, 2008
By Professor Nian Cai LIUBy Professor Nian Cai LIU
Center for World-Class Universities, Graduate School of EducationCenter for World-Class Universities, Graduate School of Education
Shanghai Jiao Tong University, ChinaShanghai Jiao Tong University, China
4. 4
Dream of Chinese for WCU
World-class university (WCU) is a dream for
generations of Chinese. It’s not only for pride, but also
for the future of China.
Recently, Chinese government has launched several
initiatives for research universities. The best-known one
is specially designed to build WCU (985 Project).
5. 5
Goals of Top Chinese Universities
Many top Chinese universities have setup their
strategic goals as WCU.
Most of them have also set time tables for achieving the
goal of WCU. For example:
2016 for Peking University
2020 for Tsinghua University
6. 6
Questions About WCU
Is there a clear definition for WCU?
How many WCU should there be in the world?
What are the positions of top Chinese universities in
the world?
How can Chinese universities improve themselves to
reach the goal of WCU?
7. 7
Academic Ranking of World Universities
Our original purpose of doing the Academic Ranking of
World Universities (ARWU) was to find out the position
of Chinese universities in the world and the gap between
them and WCU.
ARWU was put on the internet upon the encouragement
of colleagues from all over the world.
8. 8
Features of ARWU
ARWU uses a few carefully selected, objective criteria and
internationally comparable data that everyone could
verify in some way.
It has been carried out by a ranking team of four
researchers at Shanghai Jiao Tong University for their
academic interests.
It has been done independently without any financial
support from any external sources.
10. 10
Selection of Universities
Any university that has any Nobel Laureates, Fields
Medals, Highly Cited Researchers, or papers published
in Nature or Science.
Major universities of every country with significant
amount of papers indexed by Thomson.
Number of universities scanned: >2000
Number of universities actually ranked: >1000
Number of ranked universities on our web: 500
12. 12
Definition of Indicator: Alumni
The total number of the alumni of an institution
winning Nobel Prizes and Fields Medals.
Alumni are defined as those who obtain bachelor,
Master’s or doctoral degrees from the institution.
Different weights are set according to the periods of
obtaining degrees. The weight is 100% for alumni of
1991-2000, 90% for alumni of 1981-1990, 80% for
alumni of 1971-1980, and so on.
If a person obtains more than one degrees from an
institution, the institution is considered once only.
13. 13
Definition of Indicator: Award
The total number of the staff of an institution winning Nobel
prizes in physics, chemistry, medicine and economics and Fields
Medal in Mathematics.
Staff is defined as those who work at an institution at the time of
winning the prize.
Different weights are set according to the periods of winning the
prizes. The weight is 100% for winners since 2001, 90% for
winners in 1991-2000, 80% for winners in 1981-1990, 70% for
winners in 1971-1980, and so on.
If a winner is affiliated with more than one institution, each
institution is assigned the reciprocal of the number of institutions.
For Nobel prizes, if a prize is shared by more than one person,
weights are set for winners according to their proportion of prize.
14. 14
Definition of Indicator: HiCi
The number of highly cited researchers in 21 broad
subject categories in life sciences, medicine, physical
sciences, engineering and social sciences.
The definition of categories and detailed procedures
can be found at the website of Institute of Scientific
Information.
The total number of HiCi is about 5000, about 4000
of which is university staff.
15. 15
Definition of Indicator: N&S
The annual average number of articles published in
Nature and Science in the past five years.
To distinguish the order of author affiliation, a
weight of 100% is assigned for corresponding
author, 50% for first author (second author if the
first author is the same as corresponding author),
25% for the next author, and 10% for other
authors.
Only publications of article type are considered.
16. 16
Definition of Indicator: PUB
Total number of articles indexed in Science Citation
Index-expanded (SCIE) and Social Science Citation
Index (SSCI) in the past year.
A weight of 2 is assigned to articles indexed in SSCI
to compensate the bias against humanities and social
sciences.
Only publications of article type are considered.
17. 17
Definition of Indicator: PCP
The sub-total scores of the above five indicators
divided by the number of full-time equivalent
academic staff.
If the number of academic staff for institutions of a
country cannot be obtained, the total scores of the
above five indicators is used.
For ranking 2005, the number of full-time equivalent
academic staff is obtained for institutions in USA,
China, Australia, Italy, Netherlands, Sweden, and
Belgium etc.
18. 18
Main Sources of Data
Nobel laureates:
http://www.nobel.se
Fields Medals:
http://www.mathunion.org/medals/
Highly-cited researchers:
http://www.isihighlycited.com
Articles published in Nature and Science:
http://www.isiknowledge.com
Articles indexed in SCIE and SSCI:
http://www.isiknowledge.com
19. 19
Results of ARWU
http://www.arwu.org/
Top 500 universities in the world
Top 100 universities in North and Latin America
Top 100 universities in Asia/Oceania
Top 100 universities in Europe
Statistics of top universities by region and country
Percentage distribution of top universities by country
as compared with the share of global population and
GDP
21. 21
Methodological: Education and Service
Education is the basic function of any university,
however, it would be impossible to rank the quality of
education due to the huge differences among the national
systems.
Contribution to the national economic development is
becoming increasingly important for universities,
however, it is impossible to obtain internationally
comparable indicators and data.
The academic or research performance of universities, a
good indication of their reputation, can be ranked
internationally.
22. 22
Methodological: Humanities & Social Sciences
Many well-known institutions specialized in humanities
and social sciences are ranked relatively low.
Since 2004, the indicator of N&S is not considered for
institutions specialized in humanities and social sciences,
its weight is relocated to other indicators.
Since 2005, a weight of 2 for articles indexed by SSCI is
considered.
Nevertheless, if a university specialized in social sciences
and humanities had Nobel Laureates in economics and
Highly Cited Researchers in social sciences, it should have
good standing.
23. 23
Methodological: Language Bias
English is the language of international academic
community.
Any ranking based on academic performance will be
biased towards institutions in English-speaking
countries.
One possible solution: papers published in non-native
languages are offered a special weight.
Another possible solution: normalization of total
articles by the proportion of journal editors of each
country.
24. 24
Methodological: Award and Alumni
Universities which started after 1911 do not have a fair
chance.
Disciplines not related to the awarding fields do not have
a fair chance. Other important awards include Abel,
Pulitzer, Turing, Tyler, Pritzker, etc.
Institutions for winning awards and those for doing the
researches may not be the same.
Institutions for obtaining degrees and those for pursuing
the studies may not be the same.
Postdoctoral training is not considered.
25. 25
Methodological: Per Capita Performance
The weight of the PCP indicator for per capita
performance is rather low. Large institutions have
relatively high positions in the ranking.
However, it’s very difficult to obtain internationally
comparable data on the number of academic staff.
The types of academic staff: such as purely teaching staff,
teaching and research staff, purely research staff.
The ranks of academic staff: such as professor, associate
professor, reader, lecturer, research scientist etc.
26. 26
Technical: Attributions
Many universities have more than one commonly used
names: such as Virginia Tech and Virginia Polytechnic
and State University.
Variations due to translation: such as Univ Koln and
Univ Cologne, Univ Vienna and Univ Wien.
Abbreviated names: such as ETH Zurich for Swiss
Federal Institute of Technology Zurich.
Some authors only write their departmental or institute
names without mentioning their university names.
27. 27
Technical: Definition of Institution
University systems: such as Univ California system,
Univ London system.
Affiliated institutions and research organizations:
such as Ecole Polytechnique Montreal (affiliated to
University of Montreal), CNRS Labs (affiliated to
French universities).
Teaching and affiliated Hospitals: complex!
Our answer: according to author’s expression.
28. 28
Other Technical Problems
Merging, splitting, inheriting, discontinuing, name-
changing of institutions such as:
Univ Kwazulu-Natal in South Africa merged from Univ
Natal and Univ Durban-Westville.
University of Innsbruck in Austria splitted into Univ
Innsbruck and Innsbruck Medical Univ.
Humboldt Univ Berlin and Free Univ Berlin inheriting
the Nobel Prizes of the Berlin University before world
war II.
30. 30
Requests for Ranking of World Universities by
Broad subject fields or schools, colleges
and
Subject fields or programs, departments
★ In addition, many top Chinese universities want to
learn their positions in the world by broad subject
fields or disciplines.
31. 31
Definition of Broad Subject Fields
Natural Sciences and Mathematics (SCI)
Engineering/Technology and Computer Sciences (ENG)
Life and Agriculture Sciences (LIFE)
Clinical Medicine and Pharmacy (MED)
Social Sciences (SOC)
Arts and humanities are not ranked
32. 32
ARWU-FIELD Indicators and Weights
Code SCI ENG LIFE MED SOC
Alumni 10% 10% 10% 10%
Award 15% 15% 15% 15%
HiCi 25% 25% 25% 25% 25%
TOP 25% 25% 25% 25% 25%
PUB 25% 25% 25% 25% 25%
Fund 25%
33. 33
Changes in Indicators and Definition
N&S in ARWU is not used in ARWU-FIELD.
TOP is the percentage of articles published in the top
20% journals of each broad subject field.
Fund is the total engineering-related research
expenditures. It’s used only for ENG ranking.
Alumni and Award since 1951 are used for all rankings
fields except ENG.
PUB is the total number of articles indexed by Thomson
in the past year.
34. 34
Results of ARWU-FIELD
http://ed.sjtu.edu.cn/ranking.htm
Top 100 universities in SCI
Top 100 universities in ENG
Top 100 universities in LIFE
Top 100 universities in MED
Top 100 universities in SOC
Statistics of top universities by region & country
List of top universities by number of top fields
35. 35
Special Problems in ARWU-FIELD
It’s difficult to obtain data on engineering-related
research expenditures and make them comparable.
For 2007, Fund was obtained only for US and
Canadian universities.
It’s difficult to separate the Nobel Laureates in
Physiology or Medicine. They are used in both LIFE
and MED ranking.
37. 37
Controversy of Ranking
Any ranking is controversial and no ranking is
absolutely objective.
University rankings become popular in many
countries. Whether we agree or not, ranking systems
clearly are here to stay.
The key issue then becomes how to improve ranking
systems for the benefits of higher education (IREG).
38. 38
Use of Ranking
Rankings should be used with cautions. Their
methodologies must be read carefully before
reporting or using their results.
Rankings should be used in combination with other
types of evaluation whenever possible, such as
bench-marking, peer review, etc.
39. 39
Future Efforts
Study all the above mentioned problems and
continuously improve the ranking methodologies.
Establish more comprehensive databases of
WCU.
Update ARWU (every August) and ARWU-
FIELD annually (every February).
40. 40
Future Efforts (Cont’ed)
Provide ranking of universities specialized or
strong in engineering, medicine, etc. based on the
classification of world universities.
Provide ranking of universities with different
size, history, budget and function etc. once
internationally comparable data are obtained.
