This document discusses various approaches for assessing research impact, including reputation, bibliometric, and altmetric approaches. It notes that reputation approaches are subjective but bibliometric and altmetric approaches can provide more quantifiable measures of impact. However, all approaches have caveats and limitations. The document recommends following best practices such as using transparent methodologies, contextualizing metrics for specific needs, and using metrics supplementally rather than relying solely on quantitative measures.

1. From Reputation to
Citation: Varying Roles
for Scholarly Metrics
Elaine M. Lasda, MLS CAS
Associate Librarian, University at Albany
Special Libraries Association
July 18, 2017

2. What I’m Going to Say:
• Types of metrics/approaches for assessing research impact
• Reputation
• Bibliometric
• Altmetric
• Appropriate Usage and Best Practices
• Grassroots Movements
• Some common sense objectives
https://www.amazon.com/Make-
Your-Bed-Little-
Things/dp/1455570249

3. Why “Impact”?
https://flic.kr/p/SHujQt

4. Maximizing Research Impact Workshop
• Author Identifiers (ORCID)
•Bibliometrics (oldies, goodies and freebies)
• Open Access Scholarship
• Altmetrics
• Three Quick Hits
•ORCID
•Scholars Archive (IR)
•Author Profile in WoS/Scopus/GS

5. “Reputation” approach
https://flic.kr/p/JTndV5

6. “Squishy” Reputation Approaches
• Colleagues – word of mouth
• Colleagues – published surveys
• Ulrich’s – content indexed in what dbs
• Editorial Board composition
• Peer researchers have published there
• Published scholarly surveys of disciplinary experts

7. Caveats of Reputation Approach
• Subjective
• Overlook new journals/output platforms
• Rely heavily on/misuse JIF metric
• Survey methodologies vary & don’t always exist
• Info can be outdated

8. Traditional citation metrics
https://flic.kr/p/9XtEKC

9. Bibliometrics: Context and Background
Pritchard, A. (1969). Statistical bibliography or bibliometrics. Journal of Documentation, 25, 348-9.

10. Traditional-ish
• Citation count
• JIF
• SNIP/SJR
• h-index/e-index, g-index, etc.
• i-10

11. New(er) Citation-Based Tools
• Cross disciplinary metrics
• InCites
• CiteScore
• Viz!
• Manuscript Matcher
• PoP

12. Caveats of bibliometrics
• Self citations
• “Cartels” of scholars, journals
• Significant variation in source datasets
• Quality control vs. Inclusivity
• Static or rigid disciplinary categories
•emerging fields
•inter- and multidisciplinary fields

13. Altmetrics (with a small “a”)

14. What are altmetrics?
• ImpactStory
https://impactstory.org/u/0000-0002-9498-7074
• Altmetric
• Bookmarklet: https://www.altmetric.com/products/free-tools/bookmarklet
• Scholars Archive “donut”
http://scholarsarchive.library.albany.edu/ulib_fac_scholar/11/
http://scholarsarchive.library.albany.edu/ulib_fac_scholar/18/
• PlumX
In Ebscohost databases
http://libproxy.albany.edu/login?url=http://search.ebscohost.com/login.aspx?direct=true
&bquery=(altmetrics+AND+are+AND+awesome)&type=0&site=eds-live&scope=site
• Bought by Elseiver

15. Altmetrics:
More of a Reputation or Citation Approach?
https://flic.kr/p/5xgQBY

16. If a Tweet mentions your paper,
does it make an impact?
https://flic.kr/p/5xgQBY

17. Role of Altmetrics
More of a quantifiable “reputation” indicator than empirical evidence
of use
Can overcome caveats of more “squishy” reputation approaches
• Trendsetting/Immediacy
• Transcends disciplinary “silos”
• Can generate mainstream traction/impact
• Leads (hopefully) to increased impact in scholarly peer reviewed
literature (ie, boosts one’s citation count)

18. Caveats of altmetric approaches
•Not Standardized
•Transparency varies
• Which scholars are on what platforms?
•Who else is looking where our scholars look?
•Proprietary academic social media
•ResearchGate, Academia.edu, Mendeley, etc.

19. Whither “Gaming”?
https://flic.kr/p/6mzCJk
https://flic.kr/p/6mzCJk
https://flic.kr/p/6mzCJk

20. Appropriate Usage of Scholarly Metrics

22. Rubber-Meets-the-Road Impact Schemas

23. Scholarly Metrics Best Practices
• Use transparent methodologies
• Supplemental to expert qualitative evaluation
• Altmetrics serve as a reputation measure (for now)
• Citation Metrics remain a more empirical measure
• Contextualize the metric to your need
• (institution, discipline, peer groups, country/region, journal, other?)

24. What I just talked about
• Workshop to address researcher/scholar needs on campus
• Reputation and empirical (citation) impact assessments are
complementary
• Gaming happens with all measures, as does dirty data
• New approaches measure tangible outcomes of scholarly activity
• Best practices include transparent methods, contextualized
interpretation, and usage that is supplemental to expert qualitative
evaluation (no magic numbers!!!)

25. Heeeere’s
Richard!
Elaine M. Lasda
University at Albany
elasda@albany.edu
@ElaineLibrarian
http://slideshare.net/librarian68