ImpactPresentation

1. Demonstrating Scholarly
Impact: Metrics, Tools and
Trends
Jane Strudwick, Director of Scholarly Communication
June 23, 2017

2. Topics Covered
• Why assess scholarly impact of research
• Overview of metrics and tools
• Journal Metrics
• Article Metrics
• Author Metrics
• What are altmetrics
• Current issues and future considerations

3. What are we talking about when
we talk about impact? (Typically)
Extent to which research is:
• Read
• Discussed
• Used
….And
• Disseminated
Inside and outside academe

4. Why we measure
• For Promotion and Tenure
• Determine research Quality
• Assess potential for grant funding
• Job Market
• Prove productivity
• Why else?

5. What are we measuring?
• Journal Quality
• Article reach
• Article impact/Influence
• Dissemination
• Researcher Impact

6. How are we measuring?
• Journal-Level Metrics
• Measure quality of Journal using citation formulas
• Article-Level Metrics
• Citation Based or altmetrics
• Author-Level Metrics
• Measure bibliographic impact of individual authors
• Altmetrics
• Measures and monitors reach and impact of research
through online interactions

7. Journal-Level Metrics
• Most journal-level metrics are calculated from the
pool of journals indexed in two citation indexing
databases
• Web of Science(WOS) (Clarivate Analytics)
Over 11,549 journals from the Science Citation Index Expanded
(SCIE) and Social Science Citation Index (SSCI). The Library has a
subscription to Web of Science, does not include access to
Journal Citation Reports (JCR)
• SCOPUS (Elsevier)
Covers nearly 22,000 titles in the scientific, technical, medical and
social sciences (including arts and humanities. The Library does
not have a subscription to SCOPUS. Journal Metrics are freely
available.

8. https://www.journals.elsevier.com/marine-pollution-
bulletin
http://journalinsights.elsevier.com/journals/0025-
326X
https://www.journals.elsevier.com/marine-pollution-
bulletin/altmetric-articles

9. Journal Impact Factor (JIF)
• Most well known and especially influential in STEM
disciplines
• JIF is a measure of the frequency with which the
"average article" published in a given scholarly
journal has been cited in a particular year or period
• JIF is not normalized for discipline. Can use Quartile
position of title in category: Q1, Q2, Q3, Q4
• 5-year indicators are also available
• The JIF can be found independently of Journal
Citation Reports, usually on the journal home page

10. Eigenfactor Scores® and Article
Influence ®
• The Eigenfactor Project is sponsored by the West
Lab at the Information School and the Bergstrom
Lab in the Department of Biology at the University
of Washington
• It uses information from the entire citation network
to measure the importance of each journal, much
as Google's PageRank algorithm measures the
importance of websites on the world wide web
• Find Eigenfactor metrics at Eigenfactor.Org

11. Scopus-Based Metrics
• CiteScore (Elsevier)
• Calculates the average number of citations received in a
calendar year by all items published in that journal in
the preceding three years
• CiteScore counts all documents since they all have the
potential to attract citations, and the Impact Factor
counts the documents considered most likely to attract
citations. CiteScore is independent of the document-
type classification
• See About CiteScore and its derivative metrics
• Find CiteScore at https://journalmetrics.scopus.com/

12. Scopus-Based Metrics
• SNIP (Source Normalized Impact per paper)
• Defined as the ratio of the raw Impact per Publication divided
by the Relative Database Citation Potential
• Corrects for subject-specific characteristics of the field
someone is publishing in so any two journal can be compared
• Find at CWTS Journal Indicators
http://www.journalindicators.com/
• SJR Indictor (SciMago)
• Citations are weighted, depending on the rank of the citing
journal
• A citation from an important journal will count as more than
one citation; a citation coming from a less important journal
will count as less than one citation.
• Find at www.scimagojr.com

13. Article-Level Metrics
Citation-based and altmetric measures can show
impact of individual research publication
• How many times was an article cited
• How is it tracking in social media
• What is the geographic distribution of citing papers
• What is the disciplinary distribution of citing papers
• What is the impact outside of the scholarly
Community

14. Web of Science Citations
http://ezproxy.fau.edu/login?url=http://isiknowledge.com/wos

15. Google Scholar
• Useful for authors and publications in disciplines
less well covered by the commercial services
• Author search in Google Scholar will produce a list
of publications with citation and a link to Google
Scholar Profile
• Indexes varied content
• Pro – Includes useful cited by literature such as policies
• Con – Can inflate citation counts

16. Google Scholar Author Search
https://scholar.google.com/citations?view_op=search_authors

17. Author Metrics
• H-Index is the best known. Attempts to measure
both productivity and impact of the published
work.
• A scientist has an index h if h of his/her Np papers
has at least h citations each, and the other (Np h)
papers have no more than h citations each
• To have an h-index of 5, an author has to have
5 publications, each receiving at least 5 citations
• Variants include g-index and m-index
• Account for highly cited papers or author career span

18. Where to Find H-Index
Value of the index may vary depending on the source of
information (number of indexed publications, time span, etc.)
• Web of Science
• To identify all publications by an author you can use Author Analyze
search function or enter author's ORCID or ResearcherID identifier
(if known)
• Google Scholar
• Requires creation of Google Scholar profile before providing metrics
• H-index tends to be higher than what is calculated by Web of
Science
• Publish or Perish http://www.harzing.com/pop.htm
• Freely accessible software program
• Uses Google Scholar data and includes h-index, g-index, m-index
and other metrics

19. Altmetrics
“Altmetrics expand our view of what impact looks
like, but also of what’s making the impact. This
matters because expressions of scholarship are
becoming more diverse.”
From http://altmetrics.org/manifesto/
Simply, altmetrics are metrics beyond traditional
citations.

20. What altmetrics address
• How many times something is downloaded
• Who is reading the work
• Has is it been covered by news outlets
• Who is commenting on the work
• How is it being shared
• Which countries are looking at my work
http://pitt.libguides.com/altmetrics/introduction
From University of Pittsburgh Library System

21. https://springerlink.altmetric.com/details/10218936

22. https://www.nature.com/nclimate/journal/vaop/ncurrent/nclimate3316/metrics

23. What’s Next?
• The end of the Journal Impact Factor?
• Maybe…slowly
• 2012 San Francisco Declaration on Research Assessment
(DORA)
• New metrics such as the Relative Citation Ratio
• Changing expectations of funding agencies
• Acceptance of altmetrics
• Social Impact

24. http://biorxiv.org/content/early/2017/05/19/139170