Explored possible measures for national indicators of return on investments in international scientific collaboration

1. Measure of National Return in International
Science Cooperation
Caroline Wagner, Travis Whetsell,
Koen Jonkers, Jeroen Baas

2. Purpose (observation)
• “…[t]he biggest challenge for policy makers is to
ascertain the costs and benefits for international
activities of foreign and of indigenous [researchers]
for their country or region….”
• Jakob Edler
• Nations pay for science, expect to show a return
• > one-fourth of science is internationally coauthored
• Most indicators focus on national activities or counts

3. Hypothesis
•Nations gain from engagement
•Gain is measureable

4. Variables (test)
• Isolate the national contribution: FracPubs
• Isolate the national return: FracFWCI
• Add: Mobility of workforce (inflows/outflows)
• Percentage of internationally coauthored work
• Add: Government for R&D spending (GBARD)

5. Building on other work
• Earlier work showed promise for this approach
• Nature reported that FracFWCI correlated to openness
• Government spending benefitted from ‘openness’
• Follow-on article in Frontiers showed similar finding
• JASIST - openness positively correlated engagement
• +GBARD was negatively correlated

6. Tested Further
• Limitations to the original approach
• One year
• Cross sectional data
• OECD countries only
• Highly aggregated
• Wished to take it further by adding years, countries,
additional measures

7. Conducted Analysis
• Added years 2007 – 2015
• FracFWCI - dependent variable
• Conducted cross sectional analysis
• Added: Panel data regression analysis
• Still Limited to OECD 36 countries

8. Unexpected Findings
• Year on year analysis supported hypothesis
• Cross sectional showed positive correlation O=I
• The estimate grew stronger over the time period
• But…analysis of whole did not fully support
• Fixed Effects Panel data analysis did not support the findings
• Openness reversed and showed a negative sign
• Compared between countries to explore further
• Again, had a unexpected finding

9. Negative Relationship
Countries - a negative relationship with openness
USA
Russia
Poland
New Zealand
Netherlands
Japan
France
Germany
Switzerland
Canada
Austria

10. Results
• Cross-sectional - hypothesis strongly supported
• Panel data - some countries: negative relationship
• Not clear what is being shown by negative slope
• Would like feedback and ideas about this work
• Suggestions for other ways to measure benefit

11. Backup slides

14. Principal Component Loadings for Openness
Measures

15. Cross Sectional Regressions, 2007-2015

16. Controlled across countries and across time…
P-value<0.05 = *, <0.01=**, <0.001***
– Two Way Fixed Effects Model, 2007-2015

20. Table 7 – Two Way Fixed Effects, 2007-2015

21. The relative influences of government funding and international collaboration on
citation impact
Loet Leydesdorff, Lutz Bornmann, Caroline S. Wagner
(Submitted on 13 Dec 2017)
In a recent publication in Nature, Wagner & Jonkers (2017) report that public R&D
funding is only weakly correlated with the citation impact of a nation's papers as
measured by the field-weighted citation index (FWCI; defined by Scopus). On the
basis of the supplementary data, we upscaled the design using Web-of-Science data for
the decade 2003-2013 and OECD funding data for the corresponding decade assuming
a two-year delay (2001-2011). Using negative binomial regression analysis, we find
very small coefficients, but the effects of international collaboration are positive and
statistically significant, whereas the effects of government funding are negative, an
order of magnitude smaller, and statistically non-significant (in two of three analyses).
In other words, international collaboration improves the impact of average research
papers, whereas more government funding tends to have a small adverse effect when
comparing OECD countries.