Your SlideShare is downloading. ×
Bibliometrics and scientometrics
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Bibliometrics and scientometrics

5,301

Published on

Pablo Achard. Feb 26, 2009

Pablo Achard. Feb 26, 2009

Published in: Education, Technology, Business
0 Comments
11 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
5,301
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
564
Comments
0
Likes
11
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide














































































































  • Transcript

    • 1. Bibliometrics and scientometrics Part I: the classical performance indicators Pablo Achard - NCCR Affective Sciences Tuesday, February 24, 2009
    • 2. 1.What are we talking about? 2.Basic indicators 3. Performance indicators at the micro level 4.Performance indicators at the meso and macro level P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 3. 1.What are we talking about? 2.Basic indicators 3. Performance indicators at the micro level 4.Performance indicators at the meso and macro level P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 4. Definitions “Bibliometrics” is introduced by Pritchard in 1969: “the application of mathematical and statistical methods to books and other media of communication” “Scientometrics” is the science of measuring and analyzing science. As such, it includes the bibliometrics of scientific books and articles; but it also takes into account funding, demography, geography, etc. P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 5. What is it used for 1. Science and Technology Studies (epistemology, science sociology, science history,…) and the scientific study of idea spread 2. Biblioeconomics 3. Science management 4. Rankings (used by journalists, students, scientists,…) P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 6. What is it used for P. Achard “Bibliometrics and scientometrics” Lindberg, PhD Thesis Tuesday, February 24, 2009
    • 7. Why is it becoming so important 1. Availability of large databases 2. Increased use of management tools in the research administration 3. Globalization of the education market P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 8. What is this talk focusing on? The managerial use of performance indicators at the micro level (individuals, groups), and at the meso and macro levels (from departments to continents) P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 9. What is this talk focusing on? The managerial use of performance indicators at the micro level (individuals, groups), and at the meso and macro levels (from departments to continents) Therefore we will not deal with important topics like: - The study of idea spread - Demographics, education, collaborations - Links between Research and Development, public P. Achard “Bibliometrics and scientometrics” understanding of science Tuesday, February 24, 2009
    • 10. P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 11. 1.What are we talking about? 2.Basic indicators 3. Performance indicators at the micro level 4.Performance indicators at the meso and macro level P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 12. Direct input indicators Number of scientists • R&D budgets • P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 13. Direct output indicators Number of publications / books / abstracts • Number of patents • Number of PhDs • P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 14. Influence indicators Measure the “passive” effects Number of citations • Number of downloads • Pagerank of a website • P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 15. Number of citations ≠ quality The number of citations measures the reception “if a paper receives 5 or 10 citations a year throughout several years after its publication, it is very likely that its content will become integrated into the body of knowledge of the respective subject field; if, on the other hand, no reference is made at all to the paper during 5 to 10 years after publication, it is likely that the results involved do not contribute essentially to the contemporary scientific paradigm system of the subject field in question” P. Achard “Bibliometrics and scientometrics” (Braun et al. 1985) Tuesday, February 24, 2009
    • 16. Number of citations ≠ quality Book evaluation vs citations: “The J-shaped distribution of citedness” (Nicolaisen 2002) P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 17. Number of citations: issues P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 18. Number of citations: issues P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 19. Normalized indicators Number of citations / number of expected citations for similar publications The “Crown indicator” of Leiden University: Normalized by publication type, year and field P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 20. The myth of delayed recognition Papers highly cited only after a period of 5 years = 60 P. Achard “Bibliometrics and scientometrics” out of 450’000 published in 1980 or 0.013% Tuesday, February 24, 2009
    • 21. P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009 IF
    • 22. IF Statistics matter: the mean value is a very rough description of a distribution P. Achard “Bibliometrics and scientometrics” (W. Glänzel, 2003) Tuesday, February 24, 2009
    • 23. P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 24. Methodological pitfalls Field delineation Aggregation level Time frame Counting scheme Data quality P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 25. 1.What are we talking about? 2.Basic indicators 3. Performance indicators at the micro level 4.Performance indicators at the meso and macro level P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 26. Warning 1: Mating... You wouldn’t mary someone based on his/her picture on a mating website But if you look for partners on a mating website, you should care about the picture you upload (using Photoshop is a matter of personal ethics) P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 27. Warning 1: Mating... You wouldn’t mary someone based on his/her picture on a mating website But if you look for partners on a mating website, you should care about the picture you upload (using Photoshop is a matter of personal ethics) Apply the same rules with evaluating someone’s P. Achard “Bibliometrics and scientometrics” scientific value and bibliometrics Tuesday, February 24, 2009
    • 28. Warning 2: Distortion... 1. Funding agencies and science policy makers want to get the best science for their money. But there are more scientists than one can know and more fields than one can understand. 2. They fund / hire preferentially projects / scientists with the best score on a metric that correlates with the ‘quality’ they are looking for. 3. Scientists adapt their publication behavior to P. Achard “Bibliometrics and scientometrics” increase their score on this metric. Tuesday, February 24, 2009
    • 29. Individual performance indicators Number of publications / books / abstracts Number of patents Problem: no indication of the influence of the work Distortions: multiplication of small papers (Least P. Achard “Bibliometrics and scientometrics” Publishable Unit); “honorary” authors Tuesday, February 24, 2009
    • 30. Individual performance indicators Number of citations received Problems: one article can make it all (highly skewed); too long to build Distortions: self-citations; citation exchange P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 31. Individual performance indicators H-index: quot;A scientist has index h if h of his or her Np papers have at least h citations each and the other (Np - h) papers have fewer than h citations eachquot; P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 32. Individual performance indicators H-index: Problems: very strong quot;A scientist has correlation with the index h if h of his or number of publications; her Np papers have perfect to compare at least h citations scientists... at the end of each and the other their career! (Np - h) papers have fewer than h Distortions: same as citations eachquot; P. Achard “Bibliometrics and scientometrics” publication numbers Tuesday, February 24, 2009
    • 33. Individual performance indicators Many variants of the h-index P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 34. Individual performance indicators Number of publications x IF of the journal in which they where published Problems: highly noisy; very field-dependent; IF doesn’t determine future citation P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 35. The field dependency of IF P. Achard “Bibliometrics and scientometrics” (Leydesdorff, 2008) Tuesday, February 24, 2009
    • 36. IF-citation correlation? Number of citations vs IF of the journal in which articles were published for each and every article of 4 researchers P. Achard “Bibliometrics and scientometrics” (Seglen, 1997) Tuesday, February 24, 2009
    • 37. Individual performance indicators Number of publications x IF of the journal in which they where published Problems: highly noisy; very field-dependent; IF doesn’t determine future citation Distortions: editors “hot topics”; worse at the P. Achard “Bibliometrics and scientometrics” macroscopic level Tuesday, February 24, 2009
    • 38. How to increase the IF of your university? P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 39. How to increase the IF of your university? Just cut the departments with low citation rates! P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 40. How to increase the IF of your university? Just cut the departments with low citation rates! P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 41. How to increase the IF of your university? Just cut the departments with low citation rates! P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 42. How to increase the IF of your university? Just cut the departments with low citation rates! Do you think I’m kidding? P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 43. How to increase the IF of your university? Just cut the departments with low citation rates! Do you think I’m kidding? P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 44. How to increase the IF of your university? Just cut the departments with low citation rates! Do you think I’m kidding? P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 45. Individual performance indicators P. Achard “Bibliometrics and scientometrics” www.phdcomics.com Tuesday, February 24, 2009
    • 46. 1.What are we talking about? 2.Basic indicators 3. Performance indicators at the micro level 4.Performance indicators at the meso and macro level P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 47. University rankings Shanghai = alumni awards score + staff awards score + highly cited score + Nature and Science articles + articles + a mixture of all this/number of faculty Times Higher Education = peer-review score + employer-review score + staff/student score + citation/ staff score + international student score UNIGE ranking 2007 = 105 ranking 2008 = 68 P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 48. University rankings Shanghai = alumni awards score + staff awards score + highly cited score + Nature and Science articles + articles + a mixture of all this/number of faculty Times Higher Education = peer-review score + employer-review score + staff/student score + citation/ staff score + international student score UNIGE ranking 2007 = 105 ranking 2008 = 68 P. Achard “Bibliometrics and scientometrics” Let’s have a look at more serious indicators! Tuesday, February 24, 2009
    • 49. Publications per country P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 50. Publications per country P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 51. Publications per country P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 52. Publications vs GDP P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 53. Citations vs GDP (normalized) P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 54. Field dependency (again) Clinical medicine (MED) Biomedical research (BRE) Biology (BIO) Chemistry (CHE) Physics (PHY) Mathematics (MAT) Engineering (ENG) Earth and space sciences (ESS) P. Achard “Bibliometrics and scientometrics” (Glänzel, 2003) Tuesday, February 24, 2009
    • 55. Citations vs GDP (normalized) P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009
    • 56. P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009 Conclusions?
    • 57. “The use of a single index crashes the multidimensional space of bibliometrics into one single dimension.” (Wolfgang Glänzel) “Not everything that can be counted counts, and not everything that counts can be counted.” (Albert Einstein) P. Achard “Bibliometrics and scientometrics” Tuesday, February 24, 2009

    ×