Journal Metrics: The Impact Factor and Everything Else


Published on

What the latest journal metrics are, what you need to know, how people, including your customers and your authors use them, and where’s it all going.

Published in: Education
1 Like
  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Inadequate benchmarks - 10.1002/asi.21330
  • Journal Metrics: The Impact Factor and Everything Else

    1. 1. Journal Metrics: The Impact Factor and Everything Else. What the latest journal metrics are, what you need to know, how people, including your customers and your authors use them, and where’s it all going. Iain Craig Market Research & Analysis Wiley-Blackwell
    2. 2. In 1976….the Impact Factor
    3. 3. Today Impact Factor SJR Indicator SNIP ERA - A*/A/B/C Audience Factor h-index Article Influence Author Superiority Index Total Citations Immediacy Index Eigenfactor 5-year Impact Factor Cited Half-life Impact Factor g-index Rank normalised Impact Factor AR Index
    4. 4. Before You Start <ul><li>Define your Citation Universe </li></ul><ul><ul><li>Two main commercial, multidisciplinary, products </li></ul></ul><ul><ul><ul><li>Web of Science (Thomson Reuters) </li></ul></ul></ul><ul><ul><ul><li>Scopus (Elsevier) </li></ul></ul></ul><ul><ul><li>Other services include: </li></ul></ul><ul><ul><ul><li>GoogleScholar (+ Publish or Perish) </li></ul></ul></ul><ul><ul><ul><li>PubMed Central </li></ul></ul></ul><ul><ul><ul><li>CrossRef </li></ul></ul></ul><ul><ul><ul><li>RePEc/CitEc. </li></ul></ul></ul>
    5. 5. Citation Universes Scopus ~ 16,500 WoS ~11,500
    6. 6. Types of Journal Metrics <ul><li>Two Broad Classes </li></ul><ul><ul><li>Un-weighted. All citations treated equally </li></ul></ul><ul><ul><ul><li>Impact Factor, 5-Year Impact Factor, Total Citations </li></ul></ul></ul><ul><ul><li>Weighted. Some citations worth more than others </li></ul></ul><ul><ul><ul><li>SJR Indicator, Eigenfactor, Article Influence </li></ul></ul></ul>
    7. 7. Impact Factor <ul><li>Numerator </li></ul><ul><ul><li>Citations in a given census period (e.g. 2009), to all items from a Journal X published in a given target period (e.g. 2007-2008) </li></ul></ul><ul><li>Denominator </li></ul><ul><ul><li>Number of citable items published in Journal X in a given target period (e.g. 2007-2008) </li></ul></ul>
    8. 8. Problems…? Just a few <ul><li>Target period too short </li></ul><ul><li>Differences in referencing behaviour between subjects </li></ul><ul><li>Free citations to ‘non citable’ items </li></ul><ul><li>Differences in database coverage between subjects </li></ul>
    9. 9. Source Normalized Impact per Paper <ul><li>SNIP = RIP / RDCP </li></ul><ul><ul><li>RIP = Raw Impact per Paper </li></ul></ul><ul><ul><li>RDCP = Relative Database Citation Potential </li></ul></ul>
    10. 10. Some Definitions <ul><li>Raw Impact per paper (RIP) </li></ul><ul><ul><li>Average cites per article (similar to the IF) </li></ul></ul><ul><ul><ul><li>One year census period, 3 year target period </li></ul></ul></ul><ul><ul><ul><li>Only citations to/from citable items counted </li></ul></ul></ul><ul><ul><ul><li>Only citations from journals counted </li></ul></ul></ul><ul><ul><ul><li>Includes all self-citations </li></ul></ul></ul>
    11. 11. More Definitions <ul><li>Citation Potential (CP) </li></ul><ul><ul><li>Average no. references of a given age in the articles citing a given journal </li></ul></ul><ul><ul><ul><li>Corrects for differences in reference list length </li></ul></ul></ul><ul><li>Relative Database Citation Potential (RDCP) </li></ul><ul><ul><li>The CP divided by the median CP of all other journals in the database </li></ul></ul><ul><ul><ul><li>Establishes a database specific benchmark </li></ul></ul></ul>
    12. 12. Eigenfactor & SJR <ul><li>Not all citations are equal </li></ul><ul><ul><li>Citations from NEJM (IF = 50.017) are given more weight than citations from Naval Architect (IF = 0.001) </li></ul></ul><ul><li>Differences in target period </li></ul><ul><ul><li>Impact Factor = 2 year </li></ul></ul><ul><ul><li>SJR = 3 year </li></ul></ul><ul><ul><li>Eigenfactor/Article Influence = 5 year </li></ul></ul><ul><li>Differences in handling of Self-Citations </li></ul><ul><ul><li>Impact Factor includes all </li></ul></ul><ul><ul><li>SJR excludes some (max allowed =33%) </li></ul></ul><ul><ul><li>Eigenfactor/Article Influence excludes all </li></ul></ul>
    13. 13. How They Do It <ul><li>From the entire citation network: </li></ul><ul><ul><li>Iterate successive metrics, and compare how each new generation compares to the last </li></ul></ul><ul><ul><li>End point reached when no more change is apparent </li></ul></ul><ul><li>When all the dust has settled, rankings by: </li></ul><ul><ul><li>Eigenfactor ≅ Total Citations </li></ul></ul><ul><ul><li>Article Influence ≅ Impact Factor ≅ SJR </li></ul></ul>
    14. 14. Practical Necessities <ul><li>SNIP data are available at: </li></ul><ul><li> </li></ul><ul><li> </li></ul><ul><li>SJR data are available at: </li></ul><ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul><ul><li>Eigenfactor & Article Influence available at: </li></ul><ul><ul><li> </li></ul></ul><ul><ul><li> </li></ul></ul>
    15. 15. Author Metrics. Why? <ul><li>Increasing desire to assess performance of individuals </li></ul><ul><li>Recognition that journal measures are inappropriate </li></ul><ul><li>Logical extension of increasing availability of citation data </li></ul>
    16. 16. H-index <ul><li>“ I propose the index h , defined as the number of papers with citation number ≥ h , as a useful index to characterize the scientific output of a researcher.” </li></ul><ul><li>Hirsch, J. E. (2005). An index to quantify an individual's scientific research output. Proceedings of the National Academy of Sciences USA, 102 (46), 16569-16572 </li></ul><ul><li>h=5 : 5 papers each cited at least 5 times </li></ul><ul><li>h=10 : 10 papers each cited at least 10 times </li></ul><ul><li>h=100 : 100 papers each cited at least 100 times </li></ul>
    17. 17. Pros and Cons <ul><li>Conceptually very simple </li></ul><ul><li>Easy to calculate from most citation databases inc. WoS, Scopus </li></ul><ul><li>Captures elements of quantity and quality </li></ul><ul><li>Reduces the effect of single, highly cited, publications </li></ul><ul><li>Requires accurate list of output </li></ul><ul><li>Max h is max number of papers – favours prolific/older authors </li></ul><ul><li>Can never decrease but can increase even if author is inactive </li></ul><ul><li>Inter-field comparison problems remain </li></ul>
    18. 18. a-, r, and ar-Indices <ul><li>First identify the ‘h-core’ then based on that sub-set of papers, calculate: </li></ul><ul><ul><li>a: the average number of citations </li></ul></ul><ul><ul><li>r: the square root of the sums of the citations </li></ul></ul><ul><ul><li>ar: the square root of the sums of the average citations per year </li></ul></ul>
    19. 19. Which Index to Choose? <ul><li>Some evidence to support two classes of metric: </li></ul><ul><ul><li>Quantity of Productive Core </li></ul></ul><ul><ul><ul><li>h-index, q-index, m quotient, h(2) index </li></ul></ul></ul><ul><ul><ul><li>≅ Number of publications </li></ul></ul></ul><ul><ul><li>Impact of Productive Core </li></ul></ul><ul><ul><ul><li>a-index, m-index, r-index, ar-index </li></ul></ul></ul><ul><ul><ul><li>≅ Total citation counts </li></ul></ul></ul>
    20. 20. Observations <ul><li>Problems with author identification </li></ul><ul><li>Time consuming to update </li></ul><ul><li>Citation Index dependent </li></ul><ul><li>Inadequate benchmarks </li></ul><ul><li>Broadly two sets measuring: </li></ul><ul><ul><li>quantity </li></ul></ul><ul><ul><li>quality </li></ul></ul>
    21. 21. Where are we Going? <ul><li>Usage based measurements </li></ul><ul><ul><li>PIRUS2 </li></ul></ul><ul><li>Post or Public Peer-review / feedback </li></ul><ul><ul><li>PLOS journals </li></ul></ul><ul><li>Composite measures </li></ul><ul><ul><li>Usage + citations + (post) peer review? </li></ul></ul><ul><li>Researcher Profiles </li></ul><ul><ul><li>ResearcherID => Orcid </li></ul></ul><ul><ul><li>Collexis </li></ul></ul><ul><ul><li>BiomedExperts, Scholarometer, ScholarUniverse </li></ul></ul>
    22. 22. Thank You!