The document discusses the altmetrics movement which aims to supplement traditional metrics like citations with new metrics based on social media to measure the broader impact of scholarly works. It outlines various altmetric tools that track metrics from news stories, blogs, tweets and other sources to provide more comprehensive data on the attention and influence of research. However, issues remain regarding what altmetrics measure and how to properly contextualize and explain alternative metrics.

1. Altmetrics: the movement, the
tools, and the implications
Kimberley R. Barker, MLIS
Andrea H. Denton, MILS
January 27, 2016

2. Defining altmetrics
• J. Priem (@jasonpriem), I like the term
#articlelevelmetrics, but it fails to imply
*diversity* of measures. Lately, I'm liking
#altmetrics., 4:28 AM - 29 Sep 10, Tweet
• “…the creation and study of new metrics
based on the Social Web for analyzing, and
informing scholarship.”
– http://altmetrics.org/about/

3. Defining altmetrics
• Metrics that supplement or complement
traditional metrics

4. Awareness,
not
promotion!

5. From metrics to altmetrics
Measures
Traditional
Research
Products
Traditional
- Article
- Chapter
- Book
Times Cited
Impact Factor + Rank
H-index

6. From metrics to altmetrics
Measures
Traditional New
Research
Products
Traditional
- Article
- Chapter
- Book
Times Cited
Impact Factor +
Rank
H-index
Page Views
Downloads

7. From metrics to altmetrics
Measures
Traditional New
Research
Products
Traditional
- Article
- Chapter
- Books
Times Cited
Impact
Factor +
Rank
H-index
Page Views
New
- Dataset
- Blog post
- More
None

8. From metrics to altmetrics
Measures
Traditional New
Research
Products
Traditional
- Article
- Chapter
- Books
Times Cited
Impact Factor
+ Rank
H-index
Page Views
New
- Datasets
- Blog post
- More
None News stories
Blog mentions
Tweets

9. From metrics to altmetrics
Measures
Traditional New
Research
Products
Traditional
- Article
- Chapter
- Books
Times Cited
Impact Factor
+ Rank
H-index
Page Views
News stories
Blog mentions
Tweets
New
- Datasets
- Blog post
- More
None News stories
Blog mentions
Tweets

10. Additional scholarly contributions
• Blogs
• Invited Interviews
• Twitter
• Facebook postings
• Reddit
• Datasets
• Patents
• Software
• Copyrights

11. Examples of “altmeasuring”
• Downloads and page views
• Track-backs
• Tweets and retweets
• Links from review services (e.g. Facultyof1000)
• Sharing, social bookmarking
• News media

12. New metrics for traditional products

13. Newer metrics for traditional products
Br J Sports Med doi:10.1136/bjsports-2013-092417

14. New metrics for new products

15. Other influences
NSF “Publications” broadened to “Products
of Research” (Jan 2013)
• “citable and accessible including but not
limited to publications, data sets,
software, patents, and copyrights.“

16. Other influences
NIH Biosketch new format (Jan 2015)
• other non-publication research products,
including audio or video products; patents; data
and research materials; databases; educational
aids or curricula; instruments or equipment;
models; protocols; and software or netware…

17. And yet another influence…

18. Tools

19. Early altmetric tools
• Measure web views and downloads
–Google Analytics
–Bit.ly
• Measure views and reads of articles
• Google Profiles
• ResearchGate

20. Journal-level tools
• Each publisher does it a slightly different
way

21. Emerging Tools
• ImpactStory
• Altmetric.com
• PlumX

22. Impactstory
• Create an online profile
– Discover and share how your research is read,
cited, tweeted, bookmarked, and more
– Help colleagues find and read your preprints,
articles, slides and other work by uploading
research products straight your profile
• Jason Priem and Heather Piwowar
• Free for 30 days, then $60 a year.

23. Sample Impactstory Profile

24. Altmetric.com
• London-based start-up
• Funding from Digital Science (LabGuru, FigShare)

25. Altmetric’s widget (“donut”)
• Used by publishers/journals
–Nature Publishing
–Cell Press
–Wiley
–BioMed Central
–BMJ Specialty journals

26. What sources does Altmetric track?
News outlets
• Over 1,300 sites
• Manually curated list
• Text mining
• Global coverage
Social media
and blogs
• Twitter, Facebook,
Google+, Sina Weibo
• Public posts only
• Manually curated list
Reference
managers
• Mendeley, CiteULike
• Reader counts
• Don’t count towards the
Altmetric score
Other sources
• Wikipedia
• YouTube
• Reddit
• F1000
• Pinterest
• Q&A
Post-publication
peer review
• Publons
• PubPeer
Policy documents
• NICE Evidence
• Intergovernmental Panel
on Climate Change
• Many more…

27. Altmetric Score
Volume Sources Authors
The score for an article
rises as more people
mention it.
Each source category
contributes a different base
amount to the final score.
How often the author of
each mention talks about
scholarly articles influences
the contribution of the
mention.
The Altmetric score provides an indicator of the
attention surrounding a research output.
It represents a weighted approximation of all the
attention picked up for a research output and is
calculated according to three facets:

28. Cochrane Library paper investigated use of probiotics to
treat eczema: There is not enough evidence to recommend
using probiotics for the treatment of eczema.
The paper has a relatively low score of attention but
received mentions across policy documents and
Wikipedia:
• Royal College of Paediatrics and Child Health -
Allergy Care Pathways for Children: core
competency for health professionals treating
children with eczema
Discovering policy references

29. Nature Neuroscience

30. Altmetric Bookmarklet
• Free
• Reading a paper and
want to find out its
Altmetric details?
• Install the
bookmarklet in your
browser
• When viewing the
paper, “Altmetric it”

31. Altmetric Bookmarklet

32. Altmetric Bookmarklet

33. Plum Analytics
• PlumX is an institutional “impact dashboard”
that provides information on how research
output is being utilized, interacted with, and
talked about around the world
• Gathers metrics (altmetrics) about research
from more than thirty sources including PLOS,
PubMed and YouTube, and categorizes them

34. However…

35. • Standards aren’t fully defined
– Definitions, calculations, etc.
– NISO effort
• Are altmetrics important for discovery?
For evaluation? Both?
Issues

36. Issues
• Impact vs. attention
–David C.’s Improbable Science… “Why you
should ignore altmetrics and other
bibliometric nightmares”
http://www.dcscience.net/?p=6369
• Popularity
–Popular topics get higher counts, quickly,
but then fade. How does this reflect
quality?

37. Issues
• Too much concern on metrics (“culture of
measurement” “yelpification”)
• Does social media help promote good
science? Or not? (e.g. anti-vaccine)

38. Altmetrics: where to start?
Altmetrics for Researchers (Duke University Medical Library)

39. What are your products?
• Paper, chapter, book?
• A clinical protocol?
• Software code?
• Conference poster?
• Teaching material?
• White paper?
• Data set

40. Where are your products?
• A repository?
• Website?
• Profile?
Are they well-described (findable)?
Are they accessible by others?
Are they citable?
Are they downloadable?
Are there metrics to tell you?

41. What metrics match those products?
Product Metric
Clinical protocol Adoption
Software code Downloads or forks
Conference poster Views
Teaching materials Adoption/adaptation
White paper Views, Tweets

42. What systems or tools can provide
those metrics?
• Journal’s website
–Views, downloads, comparisons
• Repository
–Views, downloads
• Altmetric.com; Impactstory

43. How will you explain these metrics?
• Contextualize
– “This paper was in the top 10% of all papers
downloaded in 2015.”
• Describe “broader impact”
– “This work was picked up by over 100 news
sources.”

44. Pulling it all together
• Online identity
• Personal branding
• Reputation management

45. Additional Readings
• Altmetrics for Researchers (Duke University
Medical Center Library & Archives)
• How to Track the Impact of Research Data
with Metrics (Digital Curation Centre)

46. Thank You!
Andrea Denton andrea@virginia.edu
Kimberley Barker kimberley@virginia.edu