Overall WoS coverage by main field EXCELLENT (> 80%) VERY GOOD (60-80%) GOOD(40-60%) Biochem & Mol Biol Appl Phys & Chem Mathematics & Statistical sciences Biol Sci – Humans Biol Sci – Anim & Plants Economics Chemistry Psychol & Psychiat Engineering Clin Medicine Geosciences MODERATE (<40 %) Phys & Astron Soc Sci ~ Medicine Other Soc Sci Humanities & Arts
Conclusions on adequacy issue
We can clearly conclude that the application of bibliometric techniques, solely based on WoS (but very likely also Scopus) will not be valid for some of the ‘soft’ fields in the social sciences and the humanities.
That is why the tool box has to be extended !
The H-Index and its limitations
The H-Index, defined as …
The H-Index is the score that indicates the position at which a publication in a set, the number of received citations is equal to the ranking position of that publication.
Idea of an American physicist, J. Hirsch, who published about this index in the Proc. NAS USA.
Examples of Hirsch-index values
Environmental biologist, output of 188 papers, cited 4,788 times in the period 80-04.
Hirsch-index value of 31
Clinical psychologist, output of 72 papers, cited 760 time sin the period 80-04.
Hirsch-index value of 14
Problems with the H-Index
For serious evaluation of scientific performance, the H-Index is as indicator not suitable, as the index:
Is insensitive to field specific characteristics (e.g., difference in citation cultures between medicine and other disciplines).
Does not take into account age and career length of scientists, a small oeuvre leads necessarily to a low H-Index value.
Is inconsistent in its ‘behaviour’.
Actual versus field normalized impact (CPP/FCSm) displayed against the output.
Large output can be combined with a relatively low impact
H-Index displayed against the output.
Larger output is strongly correlated with a high H-Index value.
Definition. A scientific performance measure is said to be consistent if and only if for any two actors A and B and for any number n ≥ 0 the ranking of A and B given by the performance measure does not change when A and B both have a new publication with n citations.
Consistency ensures that if the publishing behavior of two actors does not change over time, their ranking relative to each other also does not change
Consistency ensures that if the individual researchers in one research group X outperform the individual researchers in another research group Y , the former research group X as a whole outperforms the latter research group Y.
Inconsistency of the h-index Actor A Actor B h = 4 h = 6 h = 6 h = 8
ISI Impact Factors: calculation and validity
Methodology: ISI’s classical IF
The ISI Impact Factor (IF) is defined as the number of citations received by a journal in year t, divided by the number of citeable documents in that same journal in the years t-1 and t-2,
Or, as a Formula:
Citations in year t Number of ‘citeable documents’ in t -1 & t -2
Share ‘citations-for-free’ for The Lancet
Article 784 2986
Note 144 593
Review 29 232
Sub-total 957 (a) 7959 (b)
Letter 4181 (d) 4264 (e)
Editorial 1313 905
Other 1421 909
Total 7872 14037 (c)
ISI Method :
Citations in 2000 .
Citeable documents in ‘98 and ‘99
CWTS Method :
Citations to Art/Not/Rev in 2000 .
Art/Not/Rev in ‘98 and ‘99
Citations to Art/Let/Not/Rev in 2000 .
Art/Let/Not/Rev in ‘98 and ‘99
ISI Impact Factors
From 1995 onwards CWTS has analyzed the uses and validity ISI Journal Impact Factor (IF).
Most important points of criticism were:
Calculated erroneously .
Not sensitive for the composition of the journal in terms of the document types.
Not sensitive for the science fields a journal is attached to …
Based on too short ‘citation windows’ .
Distribution of citations used for the calculationof the IF value of The Lancet
The IF-score of The Lancet is seriously ‘overrated’ by the scientific ‘audience’ of the journal.
The red area indicates citations ‘for free’, while the blue area indicates ‘correct citations ’
Impact Factors for Br. J. Clin. Pharm. and Clin. Pharm. & Ther.
The graph shows the correct and erroneous impact factors of BJCP and CPT
In the case of CPT , citations to published meeting abstracts are included, while BJCP has stopped publishing of meeting abstracts !
Document types and fields Field Journal IF JFIS The IF is for ‘02, JFIS covers ‘98-‘02 IMMUNOLOGY ANN REV IMMUNOL 50.49 1 5.18 1 BIOCHEM & MOLECULAR BIOL ANN REV BIOCHEM 34.61 1 4.10 3 PHARMACOL & PHARMACY PHARMACOLOGICAL REV 27.74 1 4.75 1 CELL BIOL ANN REV CELL & DEVELOPM BIOL 27.53 1 1.72 13 DEVELOPMENTAL BIOL ANN REV CELL & DEVELOPM BIOL 27.53 1 1.72 3 PHYSIOLOGY PHYSIOLOGICAL REV 24.82 1 3.18 1 CELL BIOLOGY NATURE REV MOL CELL BIOL 22.21 4 2.76 8 ENDOCRINOL & METABOLISM ENDOCRINE REV 21.98 1 2.87 1 NEUROSCIENCES ANN REV NEUROSCIENCE 21.89 1 3.12 4 PHYSICS REV MODERN PHYSICS 20.14 1 5.02 1 CHEMISTRY CHEMICAL REV 19.67 1 2.89 2