altmetrics (medical library perspective)

ALTMETRICS | Tara Brigham | 1

ALTMETRICS
NNLM Southeastern/Atlantic Region webinar by Tara Brigham June 17th 2015
Presentation Outline

1. ALTMETRICS
ALTERNATIVE TO TRADITIONAL METRICS

“Altmetrics is the study and use of scholarly impact measures based on activity in online tools and
environments. The term has also been used to describe the metrics themselves – one could propose in
plural a ‘set of new altmetrics.’ Altmetrics is in most cases a subset of both scientometrics and
webometrics; it is a subset of the latter in that it focuses more narrowly on scholarly influence as
measured in online tools and environments, rather than on the Web more generally.”

Priem, J., P. Groth, and D. Taraborelli. “The Altmetrics Collection.” PLoS ONE 7, no.11: e48753.
doi:10.1371/journal.pone.0048753.

Photo by Evil Erin

2. "Altmetrics are measures of scholarly impact mined from activity in online tools and environments.”
Jason Priem, author of “Altmetrics: a manifesto”

Mr. Priem also is acknowledged as the person that first coined the term ‘altmetrics’, in which he
mentioned in a 2010 tweet.

Priem, J. (jasonpriem). “I like the term #articlelevelmetrics, but it fails to imply *diversity* of measures.
Lately, I’m liking #altmetrics.” September 28, 2010, 7:28pm,
https://twitter.com/jasonpriem/status/25844968813.

3. WHAT CAN BE MEASURED?
Articles, Books, Posters
Plus non‐traditional scholarly work or research ‘products’
Examples: lab notes, data sets or software

Article‐level metrics + Includes research ‘products’ like lab notes, data sets or software.

Photo by thisisbossi

4. SOURCES OF ALTMETRICS
 PubMed
 Mendeley
 Twitter
 Facebook
 research blogs
 & many others


* Depends on aggregator or altmetric tool (More to come later in the presentation).

Photo by mkhmarketing

5. SLIDE 5
http://altmetrics.org/manifesto/

6. TRADITIONAL METRICS
The best way for us to explore the new field of altmetrics is to identify how altmetrics differ from
traditional metrics:
 number of peer reviewed article published by an author
 number of citations accumulated from those published articles
 ranking of the journals in which those articles were published (journal impact factor)

7. Number of peer reviewed articles published

You can search for the number of peer‐reviewed articles a clinician or researcher has published by
looking at their CV, or more reliably might be searching for the articles in PubMed, Scopus, Web of
Science (or a respective database for their particular discipline).

Interestingly, some reviewers of journal articles are now being credited for their work as a reviewer.
 See this article: http://www.nature.com/news/the‐scientists‐who‐get‐credit‐for‐peer‐review‐
1.16102
 Publons: https://publons.com/

Photo by Jason A. Howie

8. Number of citations accumulated from those published articles

Can be found on some journal websites, in the Scopus database, Web of Science database &
GoogleScholar.
 Scopus: http://www.elsevier.com/solutions/scopus/features
 Web of Science: http://interest.science.thomsonreuters.com/content/WOKUserTips‐201011‐IN
 GoogleScholar: http://scholar.google.com/intl/en‐US/scholar/citations.html

Photo by Wiertz Sébastien

9. Ranking of the journals in which those articles were published (journal impact factor)

http://wokinfo.com/essays/impact‐factor/
https://en.wikipedia.org/wiki/Impact_factor


Alberts, B. (May 17, 2013). "Impact Factor Distortions". Science 340 (6134): 787.
doi:10.1126/science.1240319. ‐ http://www.sciencemag.org/content/340/6134/787.full
Archambault, Éric; Vincent Larivière (2009). "History of the journal impact factor: Contingencies and
consequences" (PDF). Scientometrics 79 (3): 635–649. doi:10.1007/s11192‐007‐2036‐x

Simons, K. (2008). "The Misused Impact Factor". Science 322 (5899): 165. doi:10.1126/science.1165316.
PMID 18845714

Photo by taberandrew

10. EVOLUTION
So traditional metrics have been the status quo ‐ why not stick to this model? Well ‐ as with everything ‐
the scholarly environment is changing and constantly evolving.

11. Explosion of social media (and the ability to track it)
 https://en.wikipedia.org/wiki/Social_media
 http://www.pewinternet.org/data‐trend/social‐media/social‐media‐use‐by‐age‐group/
 http://www.pewinternet.org/fact‐sheets/social‐networking‐fact‐sheet/
http://altmetrics.org/altmetrics12/priem/

Photo by kdonovangaddy

12. The San Francisco Declaration on Research Assessment (DORA) 2012
Eliminate the use of journal‐based metrics, such as Journal Impact Factors, in funding, appointment, and
promotion considerations.
http://am.ascb.org/dora/
http://www.ascb.org/dora‐old/files/SFDeclarationFINAL.pdf

Initiated by the American Society for Cell Biology (ASCB) together with a group of editors and publishers
of scholarly journals. A number of themes run through these recommendations:
 the need to eliminate the use of journal‐based metrics, such as Journal Impact Factors, in
funding, appointment, and promotion considerations;
 the need to assess research on its own merits rather than on the basis of the journal in which
the research is published; and
 the need to capitalize on the opportunities provided by online publication (such as relaxing
unnecessary limits on the number of words, figures, and references in articles, and exploring
new indicators of significance and impact).

Photo by Keoki Seu

13. NIH Grant Applications


Whereas before 2014, the NIH only wanted to see what peer‐reviewed publications you had produced
to establish whether or not you were a qualified candidate, now they are allowing investigators to
include "non‐publication research products".
Non‐publication research products include:
 audio or video products
 patents
 data and research materials
 databases
 educational aids or curricula
 instruments or equipment
 models
 protocols
 software or netware

The biosketch directions now state: "Briefly describe up to five of your most significant contributions to
science. For each of these contributions, reference up to four peer‐reviewed publications or other non‐
publication research products."
https://nexus.od.nih.gov/all/2014/05/22/changes‐to‐the‐biosketch/
http://grants.nih.gov/grants/funding/424/index.htm
http://grants.nih.gov/grants/funding/424/SF424R‐R_biosketchsample_VerC.docx

This change first happened for National Science Foundation grant applications:
National Science Foundation. NSF 13‐1, Grant Proposal Guide Summary of Changes. January 2013.
http://www.nsf.gov/pubs/policydocs/pappguide/nsf13001/gpg_sigchanges.jsp

14. DIFFERENCES/ BENEFITS

This article, from one of the co‐founders of ImpactStory, lists many of the benefits of altmetrics:
Piwowar, H. “Altmetrics: What, Why and Where? Introduction.” ASIS&T Bulletin 39, no. 4 (April/May
2013): 8‐9. http://asis.org/Bulletin/Apr‐13/

15. BROADNESS
BEYOND JUST ARTICLES

 Altmetrics can measure the impact of scholarly products other than articles.
 Reflect some of the impact of web‐native scholarly products like blog posts, software, data sets,
videos and much more. http://www.slideshare.net/eservice/altmetrics‐aishe‐seminar‐may‐2015

Photo by NASA's Marshall Space Flight Center

16. IMPACT FLAVORS
CAPTURES NUANCES IN HOW ARTICLE/ PRODUCT IS BEING USED

 altmetrics can provide a greater understanding of how a publication or product is being used.


 They can disclose which scholarly products are being read, discussed, saved, and recommended
as well as cited.
 Tracking the variance in usage has led to the concept in altmetric literature called “impact
flavors.”
 Impact flavors are a way to “understand the distinctive patterns in the diverse impacts of
individual products.”
 The impact flavor of a scholarly or research product featured in the mainstream media is going
to be much different than one heavily saved in an online citation manager or cited in a research
paper.

Piwowar, H. “Altmetrics: What, Why and Where? Introduction.” ASIS&T Bulletin 39, no. 4 (April/May
2013): 8‐9. Lapinski, S., H. Piwowar, and J. Priem. “Riding the Crest of the Altmetrics Wave.” C&RL
News 74, no. 6 (June 2013): 292‐300.

Photo by brianjmatis

17. SPEED
QUICKER TURN AROUND TIME TO SEE INFLUENCE

 altmetrics aggregators are pulling data daily or weekly, so they can produce a quicker turn‐
around time to reflect the influence an article or research product is exerting on a particular
field.
 altmetrics permit impact to be measured shortly after the publication of a paper or the
completion of other products.

Photo by medically_irrelevant

18. DIVERSITY
REACHING A WIDER AUDIENCE

 "Indications of impacts on diverse audiences including scholars but also practitioners, clinicians,
patients, educators and the general public."
 For scholars and researchers who would like to show how far their research stretches into the
mainstream, altmetrics can be that link. ‐ As a rule, it is easy for most to obtain altmetrics data.

Piwowar, H. “Altmetrics: What, Why and Where?” ASIS&T Bulletin 39, no. 4 (April/May 2013): 8‐9.

Photo by KellyCDB

19. CAVEATS
As with traditional citation metrics, altmetrics can succumb to the similar problems. And these issues
have caused many scholars and researchers to be hesitant to use altmetrics in grant applications or
promotion material.


20. Open to manipulation or gaming
 Open to manipulation and gaming.
 Jeffrey Beall made this argument that altmetrics could be easily gamed.
 Beall is well‐known for his blog, which he uses to expose predatory journals and publishers that
abuse Open Access publishing.
 However, this is true for traditional metrics as well.
 Many of the altmetrics aggregators are working on this issue and making it harder to do so
example, Altmetric noticing fake Twitter accounts (new, never cited scholarly items before).
Beall blog posting about issues with altmetrics:

Beall’s blog post: http://scholarlyoa.com/2013/08/01/article‐level‐metrics/

Rebuttal to Beall's blog post: http://blog.impactstory.org/beall‐altmetrics/

Photo by elPadawan

21. Author disambiguation

 True for traditional metrics as well.
 ORCID exists, but still somewhat difficult to get individuals / researchers to sign up since it is
voluntary.
 Attended a presentation session at MLA 2015 where libraries were working with other areas to
get their users signed up for ORCID.
* Encourage our patrons to get an ORCID
http://orcid.org/
http://blog.impactstory.org/ten‐things‐you‐need‐to‐know‐about‐orcid‐right‐now/
What about at conferences? For posters?
http://betterposters.blogspot.com/2013/12/identifying‐poster‐authors‐conference.html

Photo by db Photography | Demi‐Brooke

22. Users choose different online tools, tools which come and go

 Use of online tools differ by individual researcher, discipline, and can change over time.
 Whereas one discipline has adopted Twitter as a platform for discussing new and relevant
information, another might have a stronger following on Mendeley.
 Data sources come and go:
‐ The popularity evolution among the social media platforms can also be a limitation of
altmetrics.
‐ Social media tools like MySpace and Friendster were much more active eight years ago,
but have since diminished in popularity and have been replaced with other tools like
Twitter and Facebook.

Photo by Ludwig Gatzke


23. Lack of authority or regulations

Lack of standards and oversight.
 Unlike the regulated computations of scholarly influence such as the journal impact factor, it is
unclear how altmetrics would distinguish between a poorly written article receiving a high
number of mentions in the mainstream media and an article of higher quality discussed among
leaders in that particular field.
 “[A] new paper that receives Facebook likes from a dozen Nobel laureates may be of higher
impact than another paper that receives 50 likes from undergraduates at the local junior
college.”
Tananbaum, G. “Article‐Level Metrics: A SPARC Primer.” SPARC. April 2013.

 Important distinctions in the quality of feedback and recognition are currently lacking in most
altmetric calculations.
 Many of the altmetrics tools have incorporated a ranking system that makes comparisons
between other articles within the same journal, written during the same time frame, and across
the whole database.

Photo by Nicolas Hoizey

24. Popularity or attention ≠ quality
 Popularity (attention) does not always equal quality of research.
 For example, an article or other artifact might receive more attention because it’s poorly
constructed, scandalous, or controversial.

Photo by Siebbi

25. ALTMETRICS TOOLS

What does altmetrics measure?
API will capture all types of data:
 Usage, they are working with the publishers (how many downloads)
 Captures, the basis for captures is DOI (bookmarks, shares)
 Mentions (blogs, news, Wikipedia) ‐ Social Media (Facebook likes, tweets)
 Citations

26. SLIDE 26
Major product vendors for altmetric data are Altmetric, Impactstory, and Plum Analytics.

27. ALTMETRIC.COM
http://www.altmetric.com/


28. ALTMETRIC
Track anything with a digital object identifier (DOI)
Trademark Altmetric “donuts”
Altmetric is a fee‐based, subscription tool
Geared toward publishers and institutions

 It can find mentions of papers, books, datasets in social media sites (e.g., Twitter, Facebook,
Pinterest, Google+), science blogs, many mainstream media outlets (including the NY Times, The
Guardian, and special interest publications like Scientific American and New Scientist).
‐ Now close to 80,000 news outlets, government policy documents, and reference
managers for mentions of academic papers are tracked.
 Altmetric “donuts” are a very recognizable feature of this service.
‐ The colors that make up the donut reflect the mix of sources mentioning the Altmetric
score: blue for Twitter, yellow for blogs, red for mainstream media sources, and so on.
http://support.altmetric.com/knowledgebase/articles/83335‐which‐data‐sources‐does‐
altmetric‐track.
 The Altmetric score is an aggregate metric that is created by using the count from all the
relevant mentions from a set of online sources, along with the relative influence of each source.
http://support.altmetric.com/knowledgebase/articles/83337‐how‐is‐the‐altmetric‐score‐
calculated.
 Altmetric is a fee‐based, subscription tool. It is geared toward publishers and institutions with
various plans and pricing.

29. Screen shot of Nature article.
 Nature Publishing Group (nature.com) and Scopus are two examples that have incorporated
Altmetric’s information into their website.
 Note the Altmetric 'donut'.

30. Screen shot of article in Scopus database.
 Nature Publishing Group (nature.com) and Scopus are two examples that have incorporated
Altmetric’s information into their website.
 Note the Altmetric 'donut'.

31. ALTMETRIC‐ FREE TOOLS
Bookmarklet
Altmetric Explorer (for academic librarians & repository managers)
Embeddable badges for all institutional repositories

 Anyone can use a free Bookmarklet to instantly view the altmetrics of certain papers. There are
a few restrictions to using the Altmetric Bookmarklet, but it can produce altmetric figures if used
in PubMed or any web page with a DOI.
 Altmetric offers free embeddable badges for all institutional repositories.


 Librarians can get free access to Altmetric Explorer.
 I have an account and while limited in capabilities, it has helped me answer some research
questions.
‐ For example, there was an anesthesiologist that was giving a presentation titled: "Best
critical care papers of the past 12 months that you may have missed.” The audience was
going to be a group of critical care physicians who are also anesthesiologists. The
physician wanted to know if it was possible to come up with a list of the most viewed
articles that fit these criteria. He wanted a list of about 100 papers from the past year
that fit the criteria and were the most READ (not cited).

32. Screen shot of Altmetric Explorer.

33. IMPACTSTORY
https://impactstory.org/

34. IMPACTSTORY
Co‐founded by Heather Piwowar and Jason Priem
Metrics are computed based on a variety of sources
Geared toward individuals
1 month free, $60/ month

 Impactstory.org is a non‐profit.
 Originally called “total‐impact” and was a hackathon project developed at the UK Beyond
Impact workshop in 2011.
 Impactstory tracks the impact of articles, data sets, software, videos, presentations, posters, and
lab websites by monitoring a wide variety of data sources.
 At present, metrics are computed based on the following sources:
‐ Altmetric.com (which includes, blog posts, Facebook public posts, Google+ posts,
Twitter tweets)
‐ arXiv.org
‐ CiteULike
‐ crossref
‐ delicious
‐ Dryad
‐ figshare
‐ GitHub
‐ Mendeley
‐ PLoS search and Article‐Level Metrics
‐ PubMed
‐ Scopus
‐ SlideShare
‐ Vimeo
‐ Web pages
‐ Wikipedia


‐ YouTube

35. Screen shot of my Impactstory profile.

 Scholars or researchers can create a collection by uploading articles or products using Google
Scholar or Open Researcher and Contributor ID (ORCID), or by providing PubMed IDs, DOI, or
URL identifiers.
 Impactstory then gathers and provides information on the author’s profile that conveys the
altmetrics of each article or research product, detailing the influence within the context of the
year it was published or created, and based on engagement type and audience.
 Over all Impactstory is useful because you can get some context. i.e. You’ve reached 5,000 views
which is better than 75% of all slide decks in SlideShare.

36. Screen shot of my Impactstory profile with world map.
 Impactstory also provides context on the reach your research has had globally.

37. Screen shot of my Impactstory profile.

38. Screen shot of an Impactstory Advisor profile.
An example of a 'super user' of Impactstory.

39. PLUM ANALYTICS
http://www.plumanalytics.com/

40. PLUM ANALYTICS
Founded in 2011, bought by EBSCO in 2014
PlumX is an analysis tool which tracks about 20 different types of what it calls ”artifacts”
Collects impact metrics in five categories: usage, captures, mentions, social media, and citations

 Plum Analytics was founded in 2011 by former Microsoft librarian, Mike Buschman and
entrepreneur Andrea Michalek.
 In January 2014, Plum Analytics became a part of EBSCO Information Services.
 Artifacts includes: journal articles, books, videos, presentations, conference proceedings,
datasets, source code, cases, and more.

41. Screen shot of my PlumX profile.
https://plu.mx/u/tbrigham


42. Screen shot of the different impact metrics it collects.
 Plum Analytics collects impact metrics in five major categories: usage, captures, mentions, social
media, and citations.

43. PLUM ANALYTICS
 PlumX is a fee‐based, tiered subscription tool
 Geared mainly towards institutions and pricing is based on the number of researchers tracked
by the system
 Offers free trial of PlumX profile for individual researchers
‐ Contact them for more information or to ask for a free trial:
http://www.plumanalytics.com/contact.html

44. OTHER altmetrics
Most journals also provide some basic alternative metrics
Times downloaded or viewed

45. Screen shot of PLoS article.

 For example, PLoS is well known for providing metrics on things such as total article views, total
times cited, and where it has been discussed (on Twitter, Facebook, etc.)

 Article in the screen shot is:
 Koffel JB. Use of recommended search strategies in systematic reviews and the impact of
librarian involvement: a cross‐sectional survey of recent authors. PLoS One. 2015 May
4;10(5):e0125931. PMID: 25938454.
http://www.plosone.org/article/Metrics/info:doi/10.1371/journal.pone.0125931

46. Your users are social
Believe it or not!

47. Screen shot of Nature article.

 Take a look at this article from Nature: Online collaboration:
Scientists and the social network Giant academic social networks have taken off to a degree that no
one expected even a few years ago. A Nature survey explores why. Richard Van Noorden 13 August
2014 Corrected: 15 August 2014.
http://www.nature.com/news/online‐collaboration‐scientists‐and‐the‐social‐network‐1.15711.

 This survey reflects that scholars are using social media and using it connect, share and discover
other papers and peers.

48. ALTMETRIC DATA USERS


Who would be interested in altmetrics?

49. RESEARCHERS, CLINICIANS, ETC.
ANYONE CREATING OR WRITING

 Anyone interested in promotion or tenure.
 Those who are more actively producing videos, blogs, books, & even posters might be very
interested in altmetrics.
‐ For example: a local physician produces a YouTube video regarding a public health issue
and they are curious how useful it is in the wider community.

Photo by Alex Barth

50. PUBLIC AFFAIRS/ MARKETING

 The Public Affairs or Marketing deparment might be interested to know which articles, videos,
or 'products' are popular. Then they can highlight articles or research that is making a wider
impact on social media and perhaps in society/ general public.

Photo by Stuck in Customs

51. ADMINISTRATORS

 Administrators want to track publications of their department or organization/ hospital/ medical
center.
 altmetrics can possible determine or influence how much dedicated time a researcher/ clinician
is given for research or academic ventures.
‐ For example: altmetric numbers might justify research time a clinician or researcher
uses).

Photo by Tyler Merbler

52. FUNDERS
 Funders are interested and want researchers to document their outreach and engagement
initiatives, as well as the "broader impacts" of their work.

Photo by rafael‐castillo

53. LIBRARIANS CAN HELP:
Explain pros and cons
Educate individuals on how to use altmetrics
Provide assistance in tracking altmetrics


 Librarians can help by doing what we do best!
 Need more encouragement/ ideas? Check out these blog posts:
‐ http://blog.impactstory.org/better‐than‐a‐free‐ferrari/
‐ http://blog.impactstory.org/4‐things‐librarians‐altmetrics/

Photo by University of the Fraser Valley

54. QUIZ

55. WHAT DO ALTMETRICS NOT MEASURE?
1. Attention
2. Quantity of impact
3. Quality
4. Diversity of impact

Correct answer: #3 Quality

56.
“The point of altmetrics isn’t to measure quality. It’s to better understand impact: both the quantity of
impact and the diverse types of impact.” – Stacy Konkiel
http://blog.impactstory.org/beall‐altmetrics/

Photo by Evil Erin

57. LEARN MORE

Brigham TJ. An introduction to altmetrics. Med Ref Serv Q. 2014;33(4):438‐47. PMID: 25316077. doi:
10.1080/02763869.2014.957093.
Freely available here: http://www.tandfonline.com/eprint/3a5IzE3fPQIJs6BUkVht/full

“Altmetrics in Context” from the Canadian Association of Research Libraries:
http://www.carl‐abrc.ca/uploads/SCC/CARL2013‐altmetrics‐EN‐FA.pdf

Altmetrics: What, Why and Where? http://asis.org/Bulletin/Apr‐13/

Ladisch, M. Altmetrics and Social Media: Publicising, Discovering, Engaging Published on Jun 11, 2015.
http://www.slideshare.net/eservice/altmetrics‐aishe‐seminar‐may‐2015

Eysenbach G. Can tweets predict citations? Metrics of social impact based on Twitter and correlation
with traditional metrics of scientific impact. J Med Internet Res. 2011 Dec 19;13(4):e123. doi:
10.2196/jmir.2012.

Steal these altmetrics LibGuides! http://blog.impactstory.org/steal‐altmetrics‐libguide/
[NOTE! Please pay attention to the comment section. You have to download them, not get them from
the Springshare website.]


Duke created their guide from the Impactstory guide:
http://guides.mclibrary.duke.edu/c.php?g=217135&p=1434254

Utrecht University Library
http://libguides.library.uu.nl/researchimpact/altmetrics

If on Twitter follow:
#altmetrics
@jasonpriem [Impactstory]
@researchremix [Heather Piwowar‐ Impactstory]
@Stew [Euan Adie‐ Altmetric.com]

@altmetric
@Impactstory
@PlumAnalytics

Photo by Pink Sherbet Photography

58. Questions?
Feel free to contact me if you have questions:
brigham.tara at gmail.com
Twitter: @TBrigham
LinkedIn: https://www.linkedin.com/in/tarabrigham

Photo by pareeerica

altmetrics (medical library perspective)

Recommended

Recommended

More Related Content

Similar to altmetrics (medical library perspective)

Similar to altmetrics (medical library perspective) (20)

Recently uploaded

Recently uploaded (20)

altmetrics (medical library perspective)