Why You Should Not Use The Journal Impact Factor To Evaluate Research

Uploaded on


More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Why you should not use the journal impact factor to evaluate research Helping you get published
  • 2. Eugene Garfield,1 the founder of the Journal Impact Factor (JIF), had originally designed it as a means to help choose journals. Unfortunately, the JIF is now often used inappropriately, for example, to evaluate the influence of individual pieces of research or even the prestige of researchers. This metric has recently come under considerable criticism owing to its inherent limitations and misuse.2-4 Why you should not use the journal impact factor to evaluate research
  • 3. The impact factor of a journal is a simple average obtained by considering the number of citations that articles in the journal have received within a specific time frame.5 A previous article, “The impact factor and other measures of journal prestige” touched upon its calculation and features. This article delves a little deeper into the fallacies of the impact factor and points that you should consider when using it. Why you should not use the journal impact factor to evaluate research
  • 4. Why you should not use the journal impact factor to evaluate research How the JIF should be used •As a measure of journal prestige and impact •To compare the influence of journals within a specific subject area •By librarians, to manage institutional subscriptions •By researchers, to identify prestigious field-specific journals to follow and possibly submit to •By journals, to compare expected and actual citation frequency and compare themselves with other journals within their field •By publishers, to conduct market research6 How the JIF should not be used •To evaluate the impact of individual articles and researchers •To compare journals from different disciplines •By funding agencies, as a basis for grant allocation •By authors, as a singular criterion of consideration for journal selection •By hiring and promotion committees, as a basis for predicting a researcher’s standing •By authors, to compare themselves against their peers3,7
  • 5. Characteristics of the JIF Using citation analysis to measure research impact
  • 6. Using citation analysis to measure research impact 1.To direct readers to an authentic source of relevant information 2.To help other researchers trace the genealogy of your ideas 3.To acknowledge pioneers and peers 4.To direct readers to previously used methods, and equipment 5.To criticize or correct previous work 6.To substantiate your claims and arguments with evidence 7.To show that you have considered various opinions in framing your arguments 8.To highlight the originality of your work in the context of previous work 9.To guide other researchers in their work 10.To build your credibility as an author 11.Finally, because not citing sources can amount to plagiarism4 11 reasons to cite previous work
  • 7. Using citation analysis to measure research impact Citation-based metrics Ranking journals Ranking researchers Ranking articles Ranking universities and countries Citation analyses can be grouped according to some broad types based on who/what is being evaluated.
  • 8. What are the various citation-based metrics? Ranking journals: Journals are ranked by counting the number of times their papers are cited in other journals. Journal-level metrics are generally meant to serve as an indicator of journal prestige. The most well known of these is the journal impact factor, from Journal Citation Reports®(a product of Thomson Reuters). The journal impact factor is calculated as the average number of citations all articles in a journal receive over a specific period of time.5 Using citation analysis to measure research impact
  • 9. What are the various citation-based metrics? Ranking researchers: Various citation metrics are now used for this purpose. Researchers are ranked by counting the number of times their individual papers are cited in other published studies. These metrics are also used to evaluate researchers for hiring, tenure, and grant decisions. A researcher-level metric that is gaining popularity is the h index,6 which is calculated by considering a combination of the number of papers published by a researcher and the number of citations these papers have received. Using citation analysis to measure research impact
  • 10. What are the various citation-based metrics? Ranking articles: Article-level citation counts may provide an accurate evaluation of the quality and impact of a specific piece of work, regardless of the author. Unfortunately though, such metrics are rarely considered because obtaining these data is tedious and time-consuming.7 Using citation analysis to measure research impact
  • 11. What are the various citation-based metrics? Ranking universities and countries: There are databases that rank universities and countries by considering their overall research output through criteria such as citable documents, citations per document, and total citations. These metrics help determine which universities and countries have the most and/or best scientific output. For example, Scimago Research Group (http://www.scimago.es/ ) releases annual reports of institution- and country-wise rankings. Using citation analysis to measure research impact
  • 12. How can citation analysis help you? Researchers today are faced with increasing pressure to get published. Academic departments are expected to meet specific levels of publication output. Clearly, there is a lot at stake in the assessment of research quality for both individuals and institutions. Given this, governments, funding agencies, and tenure and promotion committees are looking toward simple and objective methods to assess increasing research volumes in the least possible time. To this end, they are turning more and more to citation analysis for objective parameters of impact assessment. Using citation analysis to measure research impact
  • 13. Using citation analysis to measure research impact Citation analysis helps researchers to •understand the reach of their research •identify patterns in the way their work is used •benchmark themselves against their peers •lend credibility to their resumes when applying for grants and promotions •set objective targets for themselves and their publications
  • 14. Pitfalls of citation analysis When using citation analysis, it is important to bear in mind some of its limitations3,7: It overlooks the disparity in discipline-wise citation rates, that is, the fact that citation patterns differ among disciplines and over time. It ignores the fact that certain manuscript types such as letters and case reports offer inadequate scope for citation and typically have short reference lists. The sentiment of the citation is not considered; that is a negative citation (one used to refute a prior claim) is given as much merit as a positive citation (one used to further the claim being made). So even a paper that has been cited simply to discredit it can work to the author’s advantage in citation analysis. It does not account for author contribution on papers with multiple authors: such citations are as meritorious as those to single-author papers. Citation analysis attributes equal importance to all authors of a paper, regardless of their individual contribution. Using citation analysis to measure research impact
  • 15. Conclusion:- Thus, sole reliance on citation data provides an incomplete understanding of research. Although citation analysis may be simple to apply, it should be used with caution to avoid it coming under disrepute through uncritical use.3 Ideally, citation analysis should be performed to supplement, not replace, a robust system of expert review to determine the actual quality and impact of published research.8 Using citation analysis to measure research impact
  • 16. Using citation analysis to measure research impact Bibliography 1.Dodson MV (2008). Research paper citation record keeping: It is not for wimps. Journal of Animal Science, 86: 2795-2796. 2.Thomson Reuters. History of citation indexing. Essay in Free Scientific Resources. [http://thomsonreuters.com/products_services/science/free/essays/history_of_citation_indexing/] 3.Smith L (1981). Citation analysis. Library Trends, 30: 83-106. 4.Garfield E. Citation indexing-Its Theory and Application in Science, Technology, and Humanities. New York: Wiley, 1979. 5.Garfield E (2006). The history and meaning of the journal impact factor. The Journal of the American Medical Association, 295: 90-93. 6.Hirsch JE (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences USA, 102: 16569-16573. 7.Neylon C and Wu S (2009). Article-level metrics and the evolution of scientific impact. PLoS Biology, 7: 1-6. 8.Moed HF (2007) The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review. Science and Public Policy, 34: 575-583.
  • 17. Connect http://www.facebook.com/Editage http://www.twitter.com/Editage http://www.linkedin.com/company/cactus-communications