By Leena Shah Managing Editor, Ambassador for DOAJ 5th Annual Conference of Asian Council of Science Editors [ACSE]   Dubai, 21-22 March 2018

1. Understanding the Basics of
Journal Metrics
By Leena Shah
Managing Editor, Ambassador for DOAJ
5th Annual Conference of Asian Council of Science Editors [ACSE]
Dubai, 21-22 March 2018 [https://theacse.com/acseconference.php]

2. Agenda
What are journal metrics?
Where are journal metrics used?
Overview of popular citation journal metrics
Source : Journal Citation Reports (JCR) - WoS
Scopus
Google Scholar
Alternate metrics – Altmetrics
Fake metrics
Criticism of IF & key developments – DORA, Leiden
manifesto & Metric Tide report

3. What are Journal Metrics?
• Ranking of journals within a specific discipline
according to their impact and influence
• Proxy measure used to evaluate quality of a
journal
• Most citation metrics are journal level metrics.
Popular e.g. Journal Impact Factor

4. Where are journal metrics used?
- Grant applications
- Staff applying for promotion & tenure
- Make informed decision when you want to
submit your article for publication
- Make a comparison of journals within a field
- Libraries use rankings to make decisions on
journal retention or acquisitions
- Journals use it to attract editors, sponsorship,
author submissions

5. Overview of popular journal metrics based
on JCR
Source: Web of Science. Published annually in Journal
Citation Reports (JCR)
• Journal Impact Factor
• 5-year Journal IF

6. Overview of popular journal metrics based
on JCR
Source: Web of Science. Published annually in Journal
Citation Reports (JCR)
• Journal Immediacy Index
Immediacy Index refers to the number of times a paper
published in a year is cited within that same year.
• Eigenfactor
Number of times articles published in a journal over the past 5 years
have been cited, with citations from more influential journals
weighted more than citations from less influential journals.

7. Source: Scopus
• SJR – SCImago Journal Rank
weighted citations in Year X to papers published in the previous 3
years. Free website:
http://www.scimagojr.com/index.php
• SNIP – Source Normalized Impact per Paper
ratio of a source's average citation count per paper, and the
'citation potential' of its subject field. It aims to allow direct
comparison of sources in different subject fields.
Overview of popular journal metrics based
on Scopus

8. h – index [Source: WoS, Google Scholar or Scopus ]
• Attempts to measure the productivity and citation impact of the
published body of work of an author.
• Definition of h-index: a researcher with an index of h has
published h papers, each of which has been cited at least htimes.
Google Scholar Metrics [ Source: Google Scholar]
• A list of top publications based on the citations within Google,
ordered by their five-year h-index and h-median metrics.
• h5-index is the h-index for articles published in the last 5 complete
years. It is the largest number h such that h articles published in
2011-2015 have at least hcitations each.
Overview of popular journal metrics based
on Scopus/Google Scholar/WoS

9. Alternative metrics (Altmetrics)
• Bibliometrics measure research impact by analysing
citations
• Altmetrics monitor social visibility around scientific
articles e.g. tweets, blog mentions, news media,
social bookmarking, article views, and downloads.
Includes web citations in digitised scholarly
documents
• Applied to journal articles, books/book chapters,
datasets
• Real-time usage

10. Where can I find altmetrics?
• Impactstory
free author profile service that allows users with an ORCID identifier to find and
compile their altmetrics into a single profile
http://www.impactstory.org/
• Plum Analytics (Elsevier)
Plum Analytics offers altmetrics alongside citation metrics and usage statistics in
their PlumX suite of products, aimed primarily at universities and funders.
http://plumanalytics.com/
• Altmetric.com (Digital Science, Springer Nature)
• Individual journals and publishers e.g PLOS
• Scopus

11. Fake metrics
Some examples of bogus Impact factor sites:
Scientific Journal Impact Factor Info Base Index Factor
Cosmos Impact Factor Universal Impact Factor
Global Impact Factor Citefactor
How to identify fake Impact Factors?
- Research the company
- Are they asking for upfront payment?
- Can you validate the mathematical formula for their
metric?
- Data source?

12. Criticism of Journal IF
• Journal level metrics which cannot be used to evaluate
the impact of individual articles
• Scientific field that the journal belongs influences IF
• Self citation – authors cite papers in the same journal to
manipulate IF
• Artificially high IFs in journals that mainly publish review
articles
• Not reproducible and the data source is not public
• Based on a narrow 2-year time frame which is
inappropriate for many disciplines
• Biased towards English language journals

13. A call for change – DORA
The transformation of the declaration from a statement of intent to a facilitator of policy change
has begun
• San Francisco Declaration on Research Assessment [DORA]
http://www.ascb.org/dora/
- Declaration was conceived in 2012 Annual Meeting of the American
Society for Cell Biology in San Francisco and was announced in May 2013.
- International declaration (18 recommendations) calling on the world
scientific community to eliminate the role of the Journal Impact Factor
(JIF) in evaluating research for funding, hiring, promotion, or institutional
effectiveness
- Over 11800 individual signatories, 463 organizations have signed including
Nature Research, BioMed Central, SpringerOpen, British Library and UK
research councils. Recently OASPA, the Brazilian Institute of Information in
Science and Technology, and the IRCCS Ospedale San Raffaele have also
signed the declaration.

14. Leiden Manifesto and others..
• Leiden Manifesto
http://www.leidenmanifesto.org/
- Named after the conference at which it was conceived
- Released in 2015, it offers best practice (10 principles) in metrics-based research
assessment
- The article published in Nature in 2015 , “has been viewed over 60,000 times on
Nature’s website, and has accumulated 259 citations in Google Scholar, 128 in
Scopus and 55 in Web of Science.”
• ‘The Metric Tide’ Report
https://responsiblemetrics.org/the-metric-tide/
- Published in 2015 by Independent Review of the Role of Metrics in Research
Assessment and Management
- Review looked in detail at the potential uses and limitations of research metrics
and indicators, exploring the use of metrics within institutions and across
disciplines.

15. Summary
• Journal-level metrics based on citations:
JIF, 5-year IF, SNIP, SJR, Eigenfactor
source: Scopus or JCR/Web of Science
• Article-level metrics :
Altmetrics
source: social media/ web
• Author-level metrics:
h-index and Google Scholar
Criticism of JIF & its over-reliance as a measure of researcher
assessment
call for action :
- DORA
- Metric Tide
- Leiden Manifesto

16. References
LSE Impact Blog
• Beyond Impact Factors: an Academy of Management report on measuring scholarly impact. (2018, March 02). Retrieved
March 14, 2018, from http://blogs.lse.ac.uk/impactofsocialsciences/2018/03/02/beyond-impact-factors-an-academy-of-
management-report-on-measuring-scholarly-impact/
Libguides
• http://aut.ac.nz.libguides.com/impact/jif-rankings
• http://libguides.lib.umanitoba.ca/facultyhelp/journalmetrics
Blogs
• Neophytou, J. (2014, May 14). How to navigate the world of citation metrics [Web blog post]. Retrieved March 14, 2018, from
https://hub.wiley.com/community/exchanges/discover/blog/2014/05/14/how-to-navigate-the-world-of-citation-
metrics?referrer=exchanges
• Neophytou, J. (2015, April 9). Fake metrics and how to spot them [Web blog post]. Retrieved March 14, 2018, from
https://hub.wiley.com/community/exchanges/discover/blog/2015/04/09/fake-metrics-and-how-to-spot-
them?referrer=exchanges
• Curry, S. (2018, February 7). Let’s move beyond the rhetoric: it’s time to change how we judge research [Web post].
Retrieved March 14, 2018, from https://www.nature.com/articles/d41586-018-01642-w
Websites
• http://www.metrics-toolkit.org/
• https://sfdora.org/
Articles/Reports
• Scully, C., & Lodge, H. (2005). Impact factors and their significance; overrated or misused? British Dental Journal,198(7), 391-
393. doi:10.1038/sj.bdj.4812185
• Wilsdon, J., et al. (2015). The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment
and Management. DOI: 10.13140/RG.2.1.4929.1363

17. Thanks to
All the Library Consortia, Universities and Publishers and our Sponsors for the
financial support to DOAJ!

18. Thank you for your attention!
leena@doaj.org