Rons, N., PRESENTATION at the Sixth International Conference on Webometrics, Informetrics and Scientometrics (WIS) and Eleventh COLLNET Meeting. University of Mysore, Mysore, India, 19-22 October, 2010

1. 17/05/2006 1Herhaling titel van presentatie
Interdisciplinary
Research
Collaborations:
Evaluation of a
Funding Program
Nadine Rons
Vrije Universiteit Brussel (VUB)
Sixth International Conference on Webometrics, Informetrics and Scientometrics (WIS) &
Eleventh COLLNET Meeting, 19-22 October 2010, Mysore, India
Version 13.10.2010

2. October 2010 - Slide 2
6th WIS & 11th COLLNET
Structure
o Place in the broader context of INTERDISCIPLINARITY
o Origins & scope
o Interdisciplinarity in research, studied bibliometrically
o Focus: synergies between researchers from different disciplines
o Research policy: search for best practices
o Experiences from a FUNDING PROGRAM
o Program presentation: procedures & criteria
o Program evaluation: network composition, output & impact, feedback
o Conclusions

3. October 2010 - Slide 3
6th WIS & 11th COLLNET
Origins of interdisciplinarity
o Human KNOWLEDGE divided in disciplines
o categorization of scientific knowledge, subject to change
o division in more specialized communities, focusing efforts in a
growing context of information produced and communicated
o if institutionalized: shaping knowledge production & distribution
o in their present form: fairly recent (±200 years), relatively stable,
instrument for science studies
o Need for CROSSING THE BOUNDARIES
o the ‘interdisciplinary’ nature of reality
o political & socio-economic priority areas (nanotechnology, climate
research), ‘transdisciplinary’ step towards application
o opportunities for progress through cross fertilization

4. October 2010 - Slide 4
6th WIS & 11th COLLNET
Kinds of interdisciplinarity

5. October 2010 - Slide 5
6th WIS & 11th COLLNET
Bibliometric studies of interdisciplinarity
o Research publication DATABASES
o Web of Science: created to “bring together material that would never
be collated by the usual subject indexing” in “an association-of-ideas
index” (Garfield, Science 122, 1955)
o Scopus
o Examples of empirical STUDIES
o Cross-disciplinary citations (Porter and Chubin, Scientometrics 8, 1985)
o Larger time lag (Rinia et al., Scientometrics 51, 2001)
o ‘Natural’ shares of cross-disciplinary activity (Rinia et al., Scientometrics 54,
2002)
o Evolution over time (Porter and Rafols, Scientometrics 81, 2009)
o Measures for integration and specialization (Porter et al., Scientometrics 72,
2007)
Bordons et al. Analysis of cross-disciplinary research through bibliometric tools. In: Moed et al.
(Eds.), Handbook of quantitative science and technology research, pp. 437-456, 2004

6. October 2010 - Slide 6
6th WIS & 11th COLLNET
Focus in current research policy
o FUNDING programs / policies for interdisciplinary research
o ‘Top down’, thematic
o Open call, reserved fraction
o ASSESSMENT of interdisciplinary research
o Dedicated issue of Research Evaluation (Research Evaluation 15, 2006)
o Heterogeneity
o No generally accepted methodology yet
o Search for best practices
o Crucial: composition and functioning of the evaluation committee
o VUB: HORIZONTAL RESEARCH ACTIONS (HOA) program
o Set up in 2002, evaluated in 2009

7. October 2010 - Slide 7
6th WIS & 11th COLLNET
HOA program focus
o INTERDISCIPLINARY RESEARCH
o Integrating expertise from different disciplines
o Offering an all-encompassing expertise in a theme proposed
‘bottom up’
o COLLABORATIONS
o Between researchers from different departments / faculties
o In the framework of a research project
o Stimulated / supported by a dedicated funding program

8. October 2010 - Slide 8
6th WIS & 11th COLLNET
HOA call & selection phases
o CALL for applications
o Applications incl. theme & its relevance, applicants & teams, project
description & budget, external collaborations
o Selection COMMITTEE
o Board of the Research Council
o Broad scope, open attitude to different standards, coherence, good
overview of expertise and performances, views expected to surpass
disciplinary perspectives
o PRE-SELECTION and SELECTION
o Pre-selection: advice from two experienced reviewers per
application (of which 1 member of the Board)
o Final selection: after oral presentation before the Board

9. October 2010 - Slide 9
6th WIS & 11th COLLNET
HOA criteria
o TOPIC
o Interdisciplinary and innovative character, added value, importance for
science and society
o PARTNERS
o Separate strengths in disciplinary performance
o All-encompassing expertise in the proposed theme
o PROJECT and PLANNING
o Planned function of the researcher(s) financed, embodying the
integration of expertise
o Planned international network participations
≈ How panel members define a good proposal (Lamont et al., Research Evaluation 15, 2006)
≈ Grounds for evaluation suggested by researchers (Feller, Research Evaluation 15, 2006)

10. October 2010 - Slide 10
6th WIS & 11th COLLNET
Program evaluation
o Analysis of APPLICATION DATA
o 7 generations: (Co)applicants, affiliations to teams & faculties
o -> Degree of interdisciplinarity, faculty participation
o BIBLIOMETRIC study
o First 4 generations: Co-publications & co-citations in Web of Science
o -> Measures for output & impact and evolution over time *
o SURVEY
o Funded applications followed for 4 full years (full project length)
o Validity of data & indicators used, opinions
* Not used in this context:
interdisciplinarity of references, breath of journal subject categories

11. October 2010 - Slide 11
6th WIS & 11th COLLNET
Faculty participation
o ALL FACULTIES are represented among the applicants
Faculties: MEdicine
and pharmacy,
SCience and bio-
engineering sciences,
ENgineering sciences,
ARts and philosophy,
EConomics, social and
political sciences, LAw
and criminology,
PHysical education
and physiotherapy,
PSychology and
educational science;
CA: Central
academic; linear
correlation coefficient
r=0,910 for N=9 (i.e.
8 faculties + central
academic staff).

12. October 2010 - Slide 12
6th WIS & 11th COLLNET
Degree of interdisciplinarity
o A majority of BROAD INTERDISCIPLINARITY
o The total sample of applications links each of the 8 faculties to at
least 5 others.
o Degree of interdisciplinarity: joining applicants from different faculty
clusters* (broad), from different faculties within one cluster
(medium), and from different departments within one faculty
(narrow). [* ‘Social Sciences and Humanities’, ‘Basic, Natural and
Applied Sciences’ or ‘Biomedical Sciences’]

13. October 2010 - Slide 13
6th WIS & 11th COLLNET
Output & impact
o BIBLIOMETRIC interdisciplinary OUTPUT & IMPACT
o Co-publications: joint publications by applicants from different
departments
o Co-citations: publications citing applicants from different departments
A: "Newly activated", B: Previously active", C: "Output only", D: "Citation impact only", E: "Not visible".
See also: Added value of funding visible after ±3 years (Rons, Proceedings 11th Conference on STI, 2010)

14. October 2010 - Slide 14
6th WIS & 11th COLLNET
Feedback & opinions
o Confirmed interdisciplinary SYNERGIES
o The large majority (82%) of co-publications concern a synergy (as
opposed to an application of expertise from one field in another)
o Confirmed OUTPUT & IMPACT representation
o A majority of co-publications (89%) and co-citations (64%) is related to
the network topic and contain the majority of related output and
impact of various forms (next to 30% other output items and 11%
other impact items reported: contributions in proceedings, Dutch
language journal articles and books for professional audience, PhD’s
obtained and in preparation, obtained research funding)
o Confirmed SPECIFIC NEEDS
o Stimulating and necessary funding channel
o Longer startup periods due to multiple differences in traditions

15. October 2010 - Slide 15
6th WIS & 11th COLLNET
Program Evaluation: conclusions
o DEGREE OF INTERDISCIPLINARITY and FACULTY
PARTICIPATION
o The program attracts a potential present in all faculties for -
mainly ‘broad’ - interdisciplinary collaborations
o OUTPUT & IMPACT
o The program successfully selects & supports interdisciplinary
research synergies, and activates funded as well as unfunded networks
o FEEDBACK & OPINIONS
o The program stimulates interdisciplinary collaborations and responds
to a need for a dedicated funding channel
Presentation of the REPORT to the Research Council -> Program continued

16. October 2010 - Slide 16
6th WIS & 11th COLLNET
Possibilities for future studies
Bibliometric characteristics of interdisciplinary research synergies
Feasibility of normalization in interdisciplinary domains
Indicators for ex-post evaluation of interdisciplinary research

17. October 2010 - Slide 17
6th WIS & 11th COLLNET
THANK YOU FOR YOUR ATTENTION!
Nadine Rons
Vrije Universiteit Brussel (VUB)
VUB R&D Dept, Research Coordination Unit
Pleinlaan 2, B-1050 Brussels, Belgium
Nadine.Rons@vub.ac.be
http://rd-ir.vub.ac.be/en_GB/people/show/id/554
http://www.vub.ac.be/studyinginbrussels/index.html