Rons, N., POSTER presented at the 17th International Conference on Science and Technology Indicators, Montréal, Québec, Canada, 05-08 September 2012

1. Montreal, Quebec, Canada, 5-8 September 2012
http://2012.sticonference.org/
Bibliometric data sourced from Thomson Reuters Web of Knowledge (formerly referred
to as ISI Web of Science). Web of Science (WoS) accessed online 01.03-05.04.2012
Conclusions
 International and non-international publication productivities represent
different aspects of research performance.
 International peer esteem is related to the productivity in international
media, and to the % of all publications that is in international media.
 An output-driven funding or evaluation model should, according to its
aims, take into account the different roles and characteristics of
international and non-international publication media.
Introduction
The international publication and citation databases Web of Science and Scopus are
increasingly used in evaluation and funding systems. Both enlarged their coverage
towards local media in social sciences. The mostly international journals available
earlier were the basis for the development of current standard bibliometric indicators.
The same indicators may no longer measure exactly the same concepts when applied to
newly introduced or extended media categories, with possibly different characteristics
than those of international journals.
This paper investigates differences between media with and without international
dimension in publication data at team and author level.
Data (all domains, local data)
 Peer ratings: overall score, scientific merit, planning, innovation,
team quality, feasibility, productivity, scientific impact (scale 1-10)
 Publication productivity per leading researcher in media with and
without international referee system/conference scope: contributions
in journals, books and proceedings
from research assessments per discipline by international expert
panels (report years 1998 to 2011, each concerning a 5 year period)
Methodology
 Normalization of the data per discipline
 Grouped in two broad domains ‘Social sciences and humanities’ (D1)
and ‘Basic, applied and biomedical sciences’ (D2)
 Correlations between publication productivity and peer ratings
Findings
 Productivities in international and non-international subcategories are
uncorrelated for D1-teams, 88% of which publish in both
 Higher peer esteem corresponds to higher productivity for the
subcategories with international dimension only, observed for
contributions in journals, books and conference proceedings
Data (selected domains and countries)
 Social Sciences Citation Index (SSCI) publications, distinguishing
English and non-English publications as a proxy for international and
non-international media (types article, letter and review, 2006-2010)
Methodology
 Selection of 5 subject categories and 5 countries with highest %
non-English publications, >1000 in number, aiming for environments
with high presence and coverage of non-English publications
 Selection of 3 country-subject combinations with high % non-
English publications (to avoid low local journal coverage) and
substantial number of English publications (to avoid closed science
systems):
 Selection of samples of authors mainly situated in each combination
and active in the entire 5 year period: both English and local language
publications, ≥3 publications affiliated to the selected subject category
and country, ≥50% publications in the selected subject category, and
publications in the earliest and the most recent of the 5 years
 Correlations (per sample) between bibliometric parameters
determined for each author: number of publications, % in English, and
separately for English and non-English publications: % uncited and
highest number of citations to a publication (until 2011) as a proxy for
peer esteem
Findings
 Uncorrelated productivities in English versus non-English subcategories
 Higher numbers of citations to English publications correspond to a
higher % English language publications in an author’s record:
N = 43
r = 0.581
p = <.0001
N = 61
r = 0.398
p = 0.00075
N = 43
r = -0.355
p = 0.0097
N = 53
r = 0.021
p = 0.44
Characteristics of International versus Non-International
Scientific Publication Media in Team- and Author-Based Data
Nadine Rons, Nadine.Rons@vub.ac.be, http://rd-ir.vub.ac.be/en_GB/people/show/id/554
Research Coordination Unit, Vrije Universiteit Brussel (VUB), Pleinlaan 2, B-1050 Brussels, Belgium
Opportunities for further study: more thorough examination of the international dimension’s
role and its variability with country, language or discipline, using the enhanced possibilities in
international databases (media coverage, author identification).