Rons, N., POSTER presented at Creating Value for Users. 11th International Conference on Science and Technology Indicators. Leiden, The Netherlands, 9-11 September 2010

1. 11th International Conference on Science and Technology Indicators
Creating Value for Users ▬ Leiden ▬ 8-11 September 2010
Output and citation impact of interdisciplinary networks:
Experiences from a dedicated funding program
Nadine Rons, Nadine.Rons@vub.ac.be
Research Coordination Unit, Research & Development Department, Vrije Universiteit Brussel (VUB), B-1050 Brussels, Belgium
Introduction
In a context of ever more specialized scientists, interdisciplinarity receives increasing attention as innovating ideas are often situated where the disciplines meet. In many countries
science policy makers installed dedicated funding programs and policies. This induces a need for specific tools for their support. There is however not yet a generally accepted
quantitative method or set of criteria to recognize and evaluate interdisciplinary research outputs. Interdisciplinarity also takes on very different forms, as distinguished in overviews
from the first codifications (Klein, 1990) to the latest reference work (Frodeman et al., 2010). In the specific context of research measurement and evaluation, interdisciplinarity
was discussed e.g. by Rinia (2007) and Porter et al. (2006). The present empirical study aims to contribute to the understanding and the measuring of interdisciplinary research at
the micro level, in the form of new synergies between teams from different disciplines. Investigation of a specialized funding program shows how a new interdisciplinary
synergy and its citation impact are visible in co-publications and co-citations, and that these are important parameters for assessment. The results also demonstrate the effect of
funding, which is clearly present only after about three years.
‘Horizontal Research Actions’ Program
 Set up at the Vrije Universiteit Brussel in 2002.
 Collaborations joining expertise from different
disciplines on topics proposed by the applicants.
 Two-phased selection procedure involving the
Board of the Research Council as a selection
committee and oral presentations of pre-selected
projects (Rons, 2010).
 Funding spent primarily on researchers embodying
the link between the disciplines.
 Project duration initially 2 years, in the majority of
cases extended until 4 years.
Program Evaluation
 Evaluation started end 2009, when the first four
generations of applications could be followed for
three years after start of funding.
 Author-centered approach, based on the
applicants and their affiliated departments.
Key Figures
 4 generations of applications, per application on
average 4 applicants from 3 departments.
 36 applications including 3 resubmissions and
4 applications completely situated in Social
Sciences and Humanities  29 distinct networks
analyzed.
 12 funded applications.
 17 unfunded applications.
Bibliometric Study
 Co-publications: defined as joint publications by
applicants from different departments, monitored
as an indicator of interdisciplinary output (a similar
measure was used by Bordons et al., 1999).
 Co-citations: defined as publications citing
applicants from different departments, monitored
as an indicator for citation impact (obtained by
combining the results of ‘Cited Reference Search’
operations performed per applicant).
 Bibliometric data source: on line Web of Science
(SCIE, SSCI, AHCI and Proceedings indices).
 Applications completely situated in Social Sciences
and Humanities (4 out of 36) excluded due to
insufficient coverage.
 ! Absolute numbers of co-publications and co-
citations should be used with caution, not being
normalized for the interdisciplinary research areas.
Survey
 Directed to the five earliest funded networks that
could be followed for at least 4 full years after start
of funding at the time of the survey.
 Seeking confirmation of the validity and adequacy
of the data and indicators used in the bibliometric
study, as well as the applicants’ opinions regarding
stimulation and support of interdisciplinary
research.
Evaluation Results
Output & Citation Impact
 The majority of funded applications (9/12)
successfully generated both co-publications and co-
citations, while a minority showed neither (2/12).
 About half of the unfunded applications (8/17)
showed neither co-publications, nor co-citations.
 Despite not being funded, an important part of the
unfunded applications (7/17) did generate co-
publications and co-citations.
Effect of Funding
 SUBSET: 8 newly activated networks (i.e. co-
publications not yet present before application) that
could be followed for at least 4 full years after start
of funding: 4 networks funded for 4 years, and
4 unfunded networks
 The effect of funding is visible in more strongly
rising co-publications and co-citations from the
third year (Figure 1)
Figure 1: Co-publications by and co-citations to
partners of newly activated interdisciplinary
networks
Networks funded for 4 years (white) and unfunded
networks (gray)
Dashed border = with partner in social sciences and
humanities
73% of co-citations are citing co-publications
13% of co-citations are itself co-publications
Validation
 From a survey of the five earliest funded networks,
all newly activated.
 The large majority of the co-publications represent
a synergy of expertise related to the topic (23/28).
The other co-publications concern rather an
application of results from one discipline in another
(2/28) or are not related to the topic (3/28).
 Overall the majority of the co-citations is related to
the topic (83/130), with considerable differences
between networks.
 The co-publications and co-citations traced in the
Web of Science contain the majority of
interdisciplinary output and impact items
generated by the networks in relation to their topics
(next to other contributions in proceedings, in
Dutch, PhD’s and obtained funding).
Conclusions
 The survey indicates that co-publications and co-
citations are good representatives of output
and impact of interdisciplinary synergies, and
important parameters for their assessment.
 The presence of co-publications and co-citations
can be used to indicate a program’s success in
attracting, selecting and activating promising
applications for interdisciplinary synergies.
 New interdisciplinary synergies on a certain topic
may in addition lead to interdisciplinary
combinations of knowledge on another topic, in or
outside of the initial network.
 The added value of a funding program for new
interdisciplinary synergies in terms of output and
impact may become visible only after a sufficient
number of applications could be followed for 3 or
more years.
References
Bordons, M., Zulueta, M.A., Romero, F. and Barrigón,
S. (1999). Measuring interdisciplinary collaboration
within a university: the effects of the
Multidisciplinary Research Programme.
Scientometrics, 46(3):383-398.
Frodeman, R., Klein, J.T. and Mitcham, C. (Eds.)
(2010). The Oxford Handbook of Interdisciplinarity.
Oxford: Oxford University Press.
Klein, J.T. (Ed.) (1990). Interdisciplinarity: History,
Theory, and Practice. Detroit: Wayne State
University Press.
Porter, A.L., Roessner, J.D., Cohen, A.S. and Perreault,
M. (2006). Interdisciplinary research: meaning,
metrics and nurture. Research Evaluation, 15 (3),
187–195.
Rinia, E.J. (2007). Measurement and Evaluation of
Interdisciplinary Research and Knowledge Transfer.
Doctoral thesis, Leiden University.
Rons, N. (2010, forthcoming). Interdisciplinary
Research Collaborations: Evaluation of a Funding
Program. In: Proceedings of the Sixth International
Conference on Webometrics, Informetrics and
Scientometrics (WIS) and the Eleventh COLLNET
meeting, Mysore, India, October 19-22, 2010.