What is wrong with Journals' Impact Factor


Published on

It’s important to remember that the impact factor only looks at an average citation and that a journal may have a few highly cited papers that greatly increase its impact factor, while other papers in that same journal may not be cited at all. Therefore, there is no direct correlation between an individual article’s citation frequency or quality and the journal impact factor.

Published in: Education
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

What is wrong with Journals' Impact Factor

  1. 1. The Impact Factor of Journals ليس ب IF وحدة تقيم الابحاث Dr.-Ahmed-Refat-AG 12-01-2009
  2. 2. In This presentation 1 2 3 1 4 <ul><li>Raising Star H-Index !!! </li></ul><ul><li>What is your H-index ?? </li></ul><ul><li>Impact Factor Fever </li></ul><ul><li>History of IF </li></ul><ul><li>Calculation of IF </li></ul><ul><li>Uses and Misuses of IF </li></ul><ul><li>IF Alternatives </li></ul><ul><li>H-index </li></ul><ul><li>GS </li></ul><ul><li>SJR </li></ul><ul><li>Ranking of Universities and its relation to Quality of journals & Publishing </li></ul>
  3. 3. Moed HF, Van Leeuwen TN (2005) Citation analysis in research evaluation. Springer, New York <ul><li>‘ It was no longer enough just to publish in bulk: </li></ul><ul><li>it had to be demonstrable that your publications were also heavily used, </li></ul><ul><li>hence useful and important . </li></ul><ul><li>A direct indicator of usage was the fact that your research was cited by subsequent research .’ </li></ul>
  4. 4. Citation Fever & Impacities !!!??? <ul><li>Just Only Two Examples: </li></ul><ul><li><<<<<<<<< Impact fever > >>>>>>>>> </li></ul><ul><li>Oxford Journals </li></ul><ul><li>BioMed Central “BMC” Journals </li></ul>
  5. 5. Citation Fever & Impacities !!!??? <ul><li><<<<<<<<< Impact fever > >>>>>>>>> </li></ul><ul><li>Oxford University Journals </li></ul><ul><li>mini survey ~ to know some features of IF as: </li></ul><ul><li>Magnitude , Variation, Ranking of specific J, </li></ul><ul><li>Position of IF “ culture” in editorial policy and more…. Just have a look , take a note , hold your questions , and wait for the second example BMC .. </li></ul>
  6. 6. Citation Fever & Impacities !!!??? <ul><li>Do journals published by BioMed Central have Impact Factors and are their citations tracked? </li></ul><ul><li>Yes; for any journal to have an Impact Factor , however, it must be tracked by Thomson Reuters (ISI) for three years.. The following list shows those journals that are already tracked by Thomson Reuters (ISI) and so already have, or will soon have, Impact Factors . </li></ul>
  7. 7. Breaking News …Good News ???
  8. 9. It is a great honor to be indexed in Pub-Med ???????
  9. 10. Please Recommend me to ISI ??? <ul><li>Additional BioMed Central journals are currently being considered for tracking by Thomson Reuters (ISI). If the journal you are interested in is not currently tracked, you can recommend it to Thomson Reuters (ISI) for tracking </li></ul>
  10. 13. Approaches to Ranking Journals
  11. 14. Impact Factor <ul><li>An Impact Factor is one measure of the relative importance of a journal , individual article or scientist to science and social science literature and research. </li></ul><ul><li>Each index or database used to create an impact factor uses a different methodology and produces slightly different results, revealing the importance of using several sources to judge the true impact of a journal's or scientist’s work </li></ul>
  12. 15. Impact Factor <ul><li>Journals Ranking </li></ul><ul><li>Author Ranking </li></ul>
  13. 16. Journal Impact Factor <ul><li>Journal Impact Factor is frequently used as a proxy for the importance of a journal to its field. </li></ul>
  14. 17. Approaches to Ranking Journals <ul><li>In general we can distinguish two broad approaches to ranking journals: </li></ul><ul><li>Stated preference (or peer review) and </li></ul><ul><li>Revealed preference </li></ul>
  15. 18. Approaches to Ranking Journals <ul><li>Stated preference </li></ul><ul><li>involves members of particular academic community ranking journals on the basis of their own expert judgements </li></ul>
  16. 19. Stated Preference Ranking <ul><li>e.g. the British ABS Quality Journal Guide and Journal Quality List ). </li></ul><ul><li>Opinions might be based on anything from a large-scale worldwide survey of academics to a small group of individuals with decision-making power, but will always contain some element of subjectivity </li></ul>
  17. 20. Revealed Preference Ranking <ul><li>Revealed preference rankings are based on actual publication behaviour and generally </li></ul><ul><li>measure the </li></ul><ul><li>citation rates of journals </li></ul>
  18. 21. <ul><li>Several indicators of productivity or quality of Journals have been proposed, which correlate with different aspects of productivity and quality. </li></ul><ul><li>The first of these indicators, overall citation counts (‘citedness’), will remain with us as an index of the overall scientific impact (Garfield 1977–1993), while second order indicators can be used to explore subtly-grained aspects of this impact. </li></ul>Revealed Preference First Order Ranking
  19. 22. <ul><li>A well-known second-order indicator </li></ul><ul><li>is the </li></ul><ul><li>J ournal I mpact F actor (JIF) </li></ul>Revealed Preference 2 nd order Ranking
  20. 23. What is Impact Factor? <ul><li>Impact Factor IF is one of the quantitative tools for ranking, evaluating, categorizing, and comparing journals.  .  </li></ul>
  21. 24. What is Impact Factor? <ul><li>It is a measure of the frequency with which the &quot;average article&quot; in a journal has been cited in a particular year or period. </li></ul>
  22. 25. What is Impact Factor? <ul><li>The annual impact factor is a ratio between citations and recent citable items published. </li></ul>
  23. 26. What is Impact Factor? <ul><li>Thus, the impact factor of a journal is calculated by dividing the number of current year citations to the source items published in that journal during the previous two years </li></ul>
  24. 27. What is Impact Factor? <ul><li>Impact factor was first proposed </li></ul><ul><li>by Eugene Garfield </li></ul><ul><li>who is a chemist, librarian, and linguist </li></ul><ul><li>by training in a 1955 </li></ul><ul><li>article in the journal Science. </li></ul><ul><li>Chairman Emeritus, Institute for Scientific Information, ISI Philadelphia. </li></ul><ul><li>     </li></ul>
  25. 28. Calculation of IF
  26. 29. <ul><li>Example The specific calculations for OMJ 's 2007 impact factor are displayed below. (Numerator) Articles published in 200 6 that were cited in 2007: 98 Articles published in 200 5 that were cited in 2007 : 103 98+103= 201 </li></ul>Calculation of IF
  27. 30. <ul><li>(Deneminator) </li></ul><ul><li>Total Number of articles </li></ul><ul><li>published in 200 6 : 67 Total number of articles </li></ul><ul><li>published in 200 5 : 48 67+48=115 </li></ul><ul><li>IF =201/115= 1.748 </li></ul>Calculation of IF
  28. 31. <ul><li>201 (articles published in 2006 and 2005 that were cited in 2007) 115 ( total number of articles published in 2006 and 2005) </li></ul><ul><li>= 1.748 </li></ul><ul><li>The 2007 IF for the OM journal means that, </li></ul><ul><li>on average, articles published in this journal from one or two years ago have been cited around 1 and three-quarter times. </li></ul>Calculation of IF
  29. 32. Calculation of IF
  30. 34. <ul><li>. </li></ul>
  31. 35. Origin of Impact Factor
  32. 41. Impact Factor Debate <ul><li>Impact factors have been much debated in the literature in terms of their value for evaluating research quality. </li></ul>
  33. 42. What is Impact Factor? <ul><li>The impact factor is a citation measure produced by </li></ul><ul><li>Thompson Scientific's ISI </li></ul><ul><li>Web of Knowledge database . </li></ul>
  34. 43. What is Impact Factor? <ul><li>Impact factors are published annually in ISI's Journal Citation Reports Database. </li></ul>
  35. 44. What is Impact Factor? <ul><li>Impact factors are only available for journals that are indexed in ISI databases. </li></ul>
  36. 45. What is Impact Factor? <ul><li>One journal's impact factor on its own doesn't mean much . Instead, it's important to look at impact factors of multiple journals in the same subject area. This way, one can determine if the impact factor of the journal of interest is high or low compared to other journals in a subject area. </li></ul>
  37. 47. Using the Impact Factor Wisely <ul><li>The Institute for Scientific Information® (ISI®) does not depend on the impact factor alone in assessing the usefulness of a journal, and neither should anyone else. The impact factor should not be used without careful attention to the many phenomena that influence citation rates, </li></ul>
  38. 48. Using the Impact Factor Wisely <ul><li>The impact factor should be used with informed peer review. In the case of academic evaluation for tenure it is sometimes inappropriate to use the impact of the source journal to estimate the expected frequency of a recently published article. Again, the impact factor should be used with informed peer review. Citation frequencies for individual articles are quite varied. . </li></ul>
  39. 49. Factors that Influence Impact Factors 1- Date of Publication <ul><li>The impact factor is based solely on citation data and only looks at the citation frequency of articles from a journal in their first couple years of publication. Journals with articles that are steadily cited for a long period of time (say, 10 years) rather than only immediately lose out with this calculation. </li></ul>
  40. 51. Factors that Influence Impact Factors Large vs. Small Journals <ul><li>Large and small journals are compared equally. Large journals tend to have higher impact factors--nothing to do with their quality </li></ul>
  41. 52. Factors that Influence Impact Factors Average Citation <ul><li>It’s important to remember that the impact factor only looks at an average citation and that a journal may have a few highly cited papers that greatly increase its impact factor, while other papers in that same journal may not be cited at all. Therefore, there is no direct correlation between an individual article’s citation frequency or quality and the journal impact factor. </li></ul>
  42. 53. Factors that Influence Impact Factors Review Articles <ul><li>Impact factors are calculated using citations not only from research articles but also review articles (which tend to receive more citations), editorials, letters, meeting abstracts, and notes. The inclusion of these publications provides the opportunity for editors and publishers to manipulate the ratio used to calculate impact factor and falsely try to increase their number. </li></ul>
  43. 54. Factors that Influence Impact Factors Changing / Growing Fields <ul><li>Rapidly changing and growing fields (e.g. biochemistry and molecular biology) have much higher immediate citation rates, so those journals will always have higher impact factors than nursing, for instance. </li></ul>
  44. 55. Factors that Influence Impact Factors ISI's Indexing / Citation Focus <ul><li>There is unequal depth of coverage in different disciplines. In the health sciences, the Institute for Scientific Information (ISI), the company which publishes impact factors, has focused much of their attention on indexing and citation data from journals in clinical medicine and biomedical research and has not focused on nursing as much. </li></ul>
  45. 56. Factors that Influence Impact Factors Research vs. Clinical Journals <ul><li>In some disciplines such as some areas of clinical medicine where there is not a distinct separation between clinical/practitioner versus research journals, research journals tend to have higher citation rates. This may also apply to nursing. </li></ul>
  46. 60. Journal impact factor Impactitis <ul><li>P J Van Diest, H Holzel, D Burnett, and J Crocker Impactitis: new cures for an old disease J. Clin. Pathol., November 1, 2001; 54(11): 817 - 819. </li></ul>
  47. 61. Proper and improper uses of impact factor <ul><li>Evaluating individual scientists </li></ul><ul><li>Awarding higher academic positions </li></ul><ul><li>Evaluating research groups </li></ul><ul><li>Evaluating institutions </li></ul><ul><li>Resource allocation </li></ul><ul><li>Evaluating journals </li></ul>
  48. 62. Some examples <ul><li>In Japan people walk around boasting of their own individual impact factor </li></ul><ul><li>In the UK (and elsewhere) university funding is awarded on the basis of impact factors (research assessment exercise) </li></ul><ul><li>Dean s instruct authors to publish in highest impact factors, though may not be the best fit </li></ul>
  49. 63. 1 What is wrong with impact factors? <ul><li>IF not statistically representative of of individual journal articles </li></ul><ul><li>IF correlates poorly with actual citations of individual articles </li></ul><ul><li>Authors not only use IF when submitting to journals </li></ul><ul><li>The ISI database is imperfect and contains citations to “non-citable” articles </li></ul><ul><li>Per Seglen BMJ 1997;314:497 </li></ul>
  50. 64. 2 What is wrong with impact factors? <ul><li>Self citations are allowed </li></ul><ul><li>Review articles are heavily cited and inflate the impact factor </li></ul><ul><li>Long articles collect many citations </li></ul><ul><li>Short publication lag allows short term journal self citations </li></ul><ul><li>Same language citations are preferred by authors </li></ul>
  51. 65. 3 What is wrong with impact factors? <ul><li>Selective journal self citation </li></ul><ul><li>Coverage of the database is not complete (3,200 out of 126,000 journals) </li></ul><ul><li>Database has an English language bias </li></ul><ul><li>Database is dominated by American publications </li></ul><ul><li>Journals in database may vary from year to year </li></ul><ul><li>IF is a function of the number of articles in the research field </li></ul>
  52. 66. 4 What is wrong with impact factors? <ul><li>Research fields with literature that rapidly becomes obsolete are favoured </li></ul><ul><li>Impact factor depends on expansion or contraction of research field </li></ul><ul><li>Small research fields lack journals with high impact </li></ul><ul><li>Relations between fields (eg clinical v basic science) strongly determine IF </li></ul><ul><li>Citation rate of article determines journal impact, but not vice versa </li></ul>
  53. 67. Why the IF of journals should not be used for evaluating research
  54. 68. What are the alternatives?
  55. 69. IF-Variant
  56. 70. Webmetrics
  57. 71. Google Scholar (Beta) <ul><li>Google Scholar (Beta) Cited References </li></ul><ul><li>(GS) covers peer-reviewed papers, theses, books, abstracts, and other scholarly literature from all broad areas of research and from a wide variety of academic publishers and professional societies, as well as scholarly articles available across the web. </li></ul>
  58. 72. Google Scholar (Beta) Cited References <ul><li>Each Google Scholar search result contains bibliographic information, such as the title, author names, and source of publication. At the end of the search result is a “Cited by” link, which will display a list of articles and documents that have cited the document originally retrieved in the search. Note that this only includes resources indexed by Google Scholar . </li></ul>
  59. 73. P&P
  60. 74. Eigenfactor <ul><li>Ranks journals similar to Google ranking of websites. It uses the structure of the entire network (instead of purely local citation information) to evaluate the importance of each journal. </li></ul><ul><li>Measures journal price as well as citation influence. The Cost-Effectiveness Search orders journals by a measure of the value of the dollar they provide. </li></ul>
  61. 75. Eigenfactor <ul><li>Ranks scholarly journals as well as newspapers, theses, popular magazines, etc. </li></ul><ul><li>Adjusts for citation differences across disciplines, allowing for better comparison across research areas. </li></ul><ul><li>Calculations are based on the citations received over a 5-year period vs. 2 years in JCR . </li></ul><ul><li>Available free of charge on the web. [ eigenfactor .org, 2007 ] </li></ul>
  62. 76. Eigenfactor
  63. 77. H-Index
  64. 78. H-Index <ul><li>The h-index quantifies the actual scientific productivity and the apparent impact of the scientist. The h-index is based on the author’s most cited papers and the number of citations they have received from other articles. </li></ul><ul><li>&quot;A scientist has index h if h of his/her Np papers have at least h citations each, and the other (Np − h) papers have no more than h citations each.&quot; </li></ul>
  65. 79. H-Index <ul><li>An h-index of 16 means, for example, that a researcher has published 16 papers that each had at least 16 citations. Therefore, the h-index reflects both the number of articles as well as the number of citations per article. </li></ul>
  66. 80. H-Index
  67. 86. More & More about IF
  68. 87. ISI
  69. 88. H-Index <ul><li>GS Google search engine~~~~~~~ </li></ul><ul><li>Scopus Database SJR </li></ul>
  70. 89. H-index of Garfielf E The Founder of IF <ul><li>Eugene Garfield, PhD </li></ul><ul><li>Dr. Garfield is Chairman Emeritus, Institute for Scientific Information, ISI Philadelphia. </li></ul>
  71. 90. Eugene Garfield, PhD
  72. 91. H Index Garfield E
  73. 92. Zewail AH <ul><li>. </li></ul>
  74. 93. H-Index Zewail
  75. 94. GS of Ahmed-Refat
  76. 96. H-index of Ahmed-Refat
  77. 97. H index Ahmed-Refat
  78. 98. Any Question ???? Thank You