Research evaluation in the Netherlands : a library perspective Wouter Gerritsma, Library Wageningen UR
Introduction <ul><li>Research evaluation in the Netherlands </li></ul><ul><li>Bibliometric evaluation </li></ul><ul><li>Im...
Research evaluation in the Netherlands <ul><li>Based on a 6 year cycle </li></ul><ul><ul><li>Supervised by  Quality Assura...
SEP criteria <ul><li>Quality </li></ul><ul><li>Productivity </li></ul><ul><li>Relevance </li></ul><ul><li>Vitality </li></ul>
Procedures <ul><li>External reviews are internally prepared </li></ul><ul><ul><li>Productivity is extracted from publicati...
Bibliometrics analyses <ul><li>Bibliometric analysis is not stipulated by the SEP </li></ul><ul><li>It is valued by the pr...
Bibliometric analysis
A word of warning : Impact factors <ul><li>Measure for the quality of journals </li></ul><ul><ul><li>“…  it is also used f...
50 % of articles generate 90% of citations  Seglen, P. O. (1997). Why the impact factor of journals should not be used for...
Bibliometric analysis <ul><li>Citation data can be derived from many resources </li></ul><ul><li>The main question is: </l...
Baseline mathematics
Baseline Molecular Biology
Essential Science Indicators (ESI) <ul><li>Database that presents analyses of the most recent (10 years + year building) d...
Steps in citation analysis <ul><li>On the basis of authors names, all publications are checked for citations in WoS, downl...
Relations in Access
Graduate School in Environment/Ecology
Evaluation of a research institute  1,84 2,26 2,08 2,06 Overall impact 1,73 0.57 1,70 Microbiology 1,11 1,81 1,73 Clinical...
Evaluation of candidates  0 3 0,83 0,58 0,75 346 57 e 3 16 1,94 1,69 1,86 1886 88 d 0 8 0,9 1,39 1,15 972 93 c 1 17 1,95 1...
Tools at hand <ul><li>Subscription citation products  </li></ul><ul><ul><li>Web of Science (WOS) (= Science Citation Index...
Scopus <ul><li>Scopus has some important advantages </li></ul><ul><ul><li>Comprehensive citation reports </li></ul></ul><u...
Thank you for your attention © Wageningen UR
Upcoming SlideShare
Loading in...5
×

Research evaluation in the Netherlands : a library perspective

2,138

Published on

My presentation during the 2006 Keris symposium

Published in: Economy & Finance, Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,138
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
32
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Research evaluation in the Netherlands : a library perspective

  1. 1. Research evaluation in the Netherlands : a library perspective Wouter Gerritsma, Library Wageningen UR
  2. 2. Introduction <ul><li>Research evaluation in the Netherlands </li></ul><ul><li>Bibliometric evaluation </li></ul><ul><li>Impact factors </li></ul><ul><li>Citation analysis </li></ul>
  3. 3. Research evaluation in the Netherlands <ul><li>Based on a 6 year cycle </li></ul><ul><ul><li>Supervised by Quality Assurance Netherlands Universities (QANU). </li></ul></ul><ul><ul><li>Standard Evaluation Protocol (SEP) ( http://www.vsnu.nl/web/show/id=53923/langid=43/ ; in English) </li></ul></ul><ul><ul><li>Self assessments and external reviews </li></ul></ul>
  4. 4. SEP criteria <ul><li>Quality </li></ul><ul><li>Productivity </li></ul><ul><li>Relevance </li></ul><ul><li>Vitality </li></ul>
  5. 5. Procedures <ul><li>External reviews are internally prepared </li></ul><ul><ul><li>Productivity is extracted from publication databases (repositories play an important role) </li></ul></ul><ul><ul><li>Relevance, bibliometric analyses do play a role </li></ul></ul><ul><ul><li>Vitality, SWOT analyses are popular. </li></ul></ul>
  6. 6. Bibliometrics analyses <ul><li>Bibliometric analysis is not stipulated by the SEP </li></ul><ul><li>It is valued by the preparing committee </li></ul><ul><ul><li>Used internally to judge researchers and research groups </li></ul></ul><ul><ul><li>Not always used by the peer review committee </li></ul></ul>
  7. 7. Bibliometric analysis
  8. 8. A word of warning : Impact factors <ul><li>Measure for the quality of journals </li></ul><ul><ul><li>“… it is also used for assessment of the quality of individual papers, scientists and departments. For the latter a scientific basis is lacking, as we will demonstrate in this contribution” (Opthof, 1997) </li></ul></ul><ul><ul><li>Opthof, T. (1997). Sense and nonsense about the impact factor. Cardiovascular Research 33 (1): 1-7. http:// dx.doi.org /10.1016/S0008-6363(96)00215-5 </li></ul></ul>
  9. 9. 50 % of articles generate 90% of citations Seglen, P. O. (1997). Why the impact factor of journals should not be used for evaluating research. BMJ 314 (7079): 497-502. http://bmj.bmjjournals.com/cgi/content/full/314/7079/497
  10. 10. Bibliometric analysis <ul><li>Citation data can be derived from many resources </li></ul><ul><li>The main question is: </li></ul><ul><li>How do we compare numbers? </li></ul><ul><li>Scientist Z. Math has a publication from 1996 with 17 citations </li></ul><ul><li>Scientist M. Biology has a publication from 2003 with 24 citations </li></ul>
  11. 11. Baseline mathematics
  12. 12. Baseline Molecular Biology
  13. 13. Essential Science Indicators (ESI) <ul><li>Database that presents analyses of the most recent (10 years + year building) data from SCI </li></ul><ul><li>Comparisons between countries, institutes, researchers en journals </li></ul><ul><li>Hot papers </li></ul><ul><li>Research fronts </li></ul><ul><li>Baselines </li></ul>
  14. 14. Steps in citation analysis <ul><li>On the basis of authors names, all publications are checked for citations in WoS, downloaded to EndNote, subsequently to Access </li></ul><ul><li>Baselines are retrieved from ESI </li></ul><ul><li>Journals categories are checked in ESI </li></ul><ul><li>The three tables are linked by ISSN and category names </li></ul><ul><li>Analyses are made for authors, research groups and Institutes </li></ul>
  15. 15. Relations in Access
  16. 16. Graduate School in Environment/Ecology
  17. 17. Evaluation of a research institute 1,84 2,26 2,08 2,06 Overall impact 1,73 0.57 1,70 Microbiology 1,11 1,81 1,73 Clinical medicine 1,76 1,76 Chemistry 1,09 0,44 1,55 0,91 Biology & biochemistry 3,60 3,87 3,86 3,82 Agricultural Sciences Group 3 Group 2 Group 1 All groups
  18. 18. Evaluation of candidates 0 3 0,83 0,58 0,75 346 57 e 3 16 1,94 1,69 1,86 1886 88 d 0 8 0,9 1,39 1,15 972 93 c 1 17 1,95 1,84 1,93 498 65 b 2 4 1,52 1,76 1,64 1565 80 a #papers top 1% #papers top 10% RI 1999-2003 RI 1994-1998 Relative Impact #Citations # Papers 1994-2003 Author
  19. 19. Tools at hand <ul><li>Subscription citation products </li></ul><ul><ul><li>Web of Science (WOS) (= Science Citation Index) </li></ul></ul><ul><ul><li>Essential Science Indicators (ESI) </li></ul></ul><ul><ul><li>Scopus (new, Elsevier product) </li></ul></ul><ul><li>Free available Web services </li></ul><ul><ul><li>Google Scholar http://scholar.google.com/ </li></ul></ul><ul><ul><li>Citeseer http://citeseer.ist.psu.edu/ </li></ul></ul><ul><ul><li>Smealsearch http://smealsearch2.psu.edu/ </li></ul></ul>
  20. 20. Scopus <ul><li>Scopus has some important advantages </li></ul><ul><ul><li>Comprehensive citation reports </li></ul></ul><ul><ul><li>Substantially larger journal base </li></ul></ul><ul><ul><li>Author disambiguation </li></ul></ul><ul><ul><li>Cooperation with third parties </li></ul></ul><ul><ul><li>Institute disambiguation </li></ul></ul>
  21. 21. Thank you for your attention © Wageningen UR
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×