This document discusses various metrics for measuring the impact and importance of academic journals, articles, and authors. It describes journal impact factors, citation analysis tools like Web of Knowledge and Google Scholar, metrics for individual researchers like the h-index, and newer altmetric tools that analyze social media mentions. Limitations of different metrics are also outlined.

1. Journal Impact Factors
and Citation Analysis
The Ranking of Journals and
Articles
Rebecca Payne, Memorial Library Ariel Neff, Chemistry Library
University of Wisconsin-Madison Libraries • Madison, WI 53706
Phone: (608) 262-3193 • www.library.wisc.edu

2. Overview
• Introductions
• Measuring the importance of journals,
articles, and authors – Why?
• Types of Measurements & Tools
– Purpose, usage, and bias
• Altmetrics

3. Why Measure Impact?
• Hiring, advancement, tenure
• Grant applications
• Locating publications to publish in
• Identifying seminal research
• Identifying researchers whose work is
getting attention

4. Article Metrics
• Number of times an article has been cited.
• Cited Reference Search - Search for journal
articles, books, proceedings, and other published
works that have cited a previously published work.
• Discover who is citing your research and the
impact of your work.
• Discover which journals have appropriate focus for
your research.
• “Cited by” links

6. Tools for Article Impact
• Web of Knowledge
– Library resource
• Google Scholar
• Scopus
– Library does not have subscription.
– Released in 2004; Citation tracking from 1996.
• Other Databases

7. Web of Knowledge
• Science, Social Science and Humanities
Citation Indexes in one database; coverage
better for sciences
• Citations from 10,000+ journals
• Books and conference proceedings too
• One of the first to offer Cited Reference
Searching and most widely known

8. Web of Knowledge: Tips
• Use first cited author
• Search author last name initials
• Use preferred journal title abbreviations and
potential variants of journal title or publication
• Use volume, issue, and page fields with caution
• Look for cited reference variations (sometimes
different pages of the same article are cited or
papers are cited incorrectly).

9. Google Scholar
• Articles, theses, books, preprints, abstracts,
and technical reports
• If article has been cited by others, Cited by
link will be part of the record. Link to other
articles in Google Scholar.
• Better international coverage than Web of
Knowledge

10. Other Databases
• Certain disciplines, journals, and document types may not
be well represented in major tools for citation analysis.
• Examples of other databases:
– ACM Digital Library
– ProQuest databases (e.g., EconLit, Sociological
Abstracts)
– EBSCOhost databases (e.g., CINAHL, PsycINFO) –
Cited References in navigation bar at top
– Full Text Journal Collections (e.g., JSTOR)
– PubMed
– SciFinder Scholar

11. Cautions/Limitations
• No one database is comprehensive.
• No one database covers all journals.
• No one database includes all publication
formats.
• Publication dates affect your results.
• Results can be misleading.
• All impact measures are controversial.
Source: http://researchguides.library.wisc.edu/publishingresearch

12. Individual Metrics
• H-Index
– Variations

13. H-Index
A scientist has index h if h of his/her Np
papers have at least h citations each, and
the other (Np − h) papers have no more
than h citations each.
J. E. Hirsch. Proc Natl Acad Sci. 2005 November 15; 102(46):
16569–16572.

14. Calculating H-Index
Article # Times Cited
1 87
2 70 8 articles have been
3 46 cited 8 or more
4 30 times, and the
5 12 remaining articles
6 11 have been cited less
7 11 than 8 times
8 9
9 7 H index = 8
10 3
11 0

15. H-Index
Larger h-index
values mean that
a research has
published many
papers which
have been
heavily cited

16. Tools for H-Index
• Web of Knowledge databases
– Library resource
• Google Scholar
– Depends on if the researcher has a Google
account linked with their publications
• Scopus
– Library does not have a subscription
– Author impact search is free online

17. Other Tools
• Publish or Perish
– Calculates many different metrics based on
Google Scholar data
– Best for researchers in the humanities, social
science, business, engineering, math,
computer science, economics
• ResearcherID
– Uses Web of Science data
– Calculates individual metrics

18. Problems with H-Index
• Does not account for coauthors
– Favors large groups of authors (usually
experimental science over theoretical)
• Can be influenced by self-citation
• Cannot account for „first authors‟
• Dependant on career length
• All types of citations are weighted the
same
– Reviews, books, primary research

19. H-Index Alternates
• M-Quotient = h/n
– n is the individual‟s number of active years
– Better for young researchers
• G-Index = h-index for an averaged citations
count
– Weights heavily cited papers
– Egghe L., Theory and practice of the g-index,
Scientometrics, 69(2006),No 1,p.131–52
• Many other variations – see handout for
citations

20. Journal Impact
• ISI Journal Impact Factor
• Eigenfactor
• SJR: SCImago Journal Rank

21. Journal Impact Factor
• Average number of times articles from the
journal published in the past two years have
been cited in the Journal Citation Report
(JCR) year.
• Calculated by dividing the number of
citations in the JCR year by the total number
of articles published in the two previous
years. An Impact Factor of 1.0 means that, on
average, the articles published one or two
year ago have been cited one time.
Source: http://admin-apps.webofknowledge.com.ezproxy.library.wisc.edu/JCR/help/h_impfact.htm

22. Journal Citation Reports (JCR)
• Compares and evaluates 10,600 journals in
science, technology and social sciences.
• Two editions available:
– Science Edition
– Social Sciences Edition
• No Arts and Humanities edition
• Compare impact factors of journals within
the same discipline; impact factors vary by
discipline.
• Provides a means of determining the core
journals of a subject.

23. Immediacy Index
• Average number of times an article is cited
in the year it is published.
• Indicates how quickly articles in a journal
are cited.
• Calculated by dividing the number of
citations to articles published in a given
year by the number of articles published in
that year.
Source: http://admin-apps.webofknowledge.com.ezproxy.library.wisc.edu/JCR/help/h_immedindex.htm

24. Cited Half-Life
• Median age of the articles that were cited in the
JCR year. Half of a journal's cited articles were
published more recently than the cited half-life.
• For example, in JCR 2001 the journal Crystal
Research and Technology has a cited half-life of
7.0. That means that articles published in Crystal
Research and Technology between 1995-2001
(inclusive) account for 50% of all citations to
articles from that journal in 2001.
• Only journals cited 100 or more times in the JCR
year have a cited half-life.
Source: http://admin-apps.webofknowledge.com.ezproxy.library.wisc.edu/JCR/help/h_ctdhl.htm

25. 5-Year Impact Factor
• Average number of times articles from the
journal published in the past five years
have been cited in the JCR year.
• Calculated by dividing the number of
citations in the JCR year by the total
number of articles published in the five
previous years.
Source: http://admin-apps.webofknowledge.com.ezproxy.library.wisc.edu/JCR/help/h_impfact.htm

26. Cautions/Limitations
• Do not rely only on JCR for journal evaluations.
• Not intended to replace informed peer review.
• Careful attention should be paid to the conditions that
can influence citation rates: language, journal history
and format, publication schedule, and subject specialty.
• Articles include mostly original research and review
articles; other kinds of articles excluded.
• Journals published in non-English languages or using
non-Roman alphabets may be less accessible to
researchers worldwide.
• Other factors to consider when evaluating journals:
audience, peer review, journal indexing, circulation
count, acceptance rate, editor and editorial board
Source: http://admin-apps.webofknowledge.com.ezproxy.library.wisc.edu/JCR/help/h_using.htm

27. Eigenfactor
• Developed by University of Washington
– Uses data from journals in ISI Web of Science
– Free to use
– Cuts across science/social science
• Calculation is based on the number of times
articles from the journal published in the past
five years have been cited in the JCR year
– highly cited journals will influence the network
more than lesser cited journals
– Journal self-citations are removed

28. Problems with Eigenfactor
• Studies have shown that Eigenfactor
values usually correlate with IF numbers
– So why have another metric?
• Has similar biases as IF

29. SJR: SCImago Journal Rank
• Based on the PageRank algorithm used by
Google
– Accounts for both the number of citations
received by a journal and the prestige of the
journals where such citations originated
• Data drawn from Scopus
• Free and open source
– Great alternative to IF for those without
access

30. Problems with SJR
• Uses Scopus – citations only from 1996-
present
• Divides the prestige gained by a journal,
through the citations of its articles, to the
total number of articles included, rather
than to the number of citable articles
– Plays down letters to editor, news, etc.

31. Altmetrics
• Systems which use social media data to
rank authors and journals rather than just
traditional citation methods
– Tracking “likes,” comments, tweets, and other
postings

32. Altmetrics
• ImpactStory
– Compiles data on how scholars and the public use
articles in social media; includes citation manager
saves, Facebook and Twitter mentions, F1000
recommendations, and more
• PLOS Article-Level Metrics
– Altmetrics collected and compiled for every
article published in PLOS
• This is a very new field – expect growth!