This document discusses practical applications of altmetrics data. It begins by outlining different altmetrics data providers and how they may provide differing results. It then discusses how to design altmetric solutions by selecting appropriate indicators and approaches based on goals and target audiences. Several examples are given of how altmetrics can be used for monitoring attention to research and evaluating impact through case studies and researcher profiling. Interpreting altmetrics data requires consideration of proxy alignment, changes in data collection, and errors.

1. Practical applications of altmetrics
Nicolas Robinson-Garcia

2. Objectives
➢ Analyze differences by altmetric data provider and
consequences from using one or the other
➢ Discuss critically which information of interest can be
obtained from the different altmetrics
➢ Develop creative solutions to respond to current demands
making use of altmetric data

3. Agenda
I. The altmetric toolbox
➢ Indicators
➢ Data collection and processing
II. Design of altmetric solutions
➢ Demands that can/not be addressed
➢ Interpreting altmetrics
III. Altmetrics in practice
➢ Monitoring attention
➢ Supporting case studies

4. The altmetric toolbox
WHICH ALTMETRIC PROVIDER SHOULD WE USE?
• Altmetric providers cover a wide and heterogenenous set of
indicators
• This set of indicators varies by altmetric provider
• Altmetric providers offer differing results for same indicators
• These differences are due to the data collection process

5. The altmetric toolbox
“Altmetrics is the study and use of scholarly
impact measures based on activity in online
tools and environments”
Priem, Groth & Taraborelli, 2012

6. The altmetric toolbox

7. Citations
News media
Policy documents
Open peer review
(F1000, Publons)
YouTube
Open Syllabus
Patents
…
The altmetric toolbox

8. Downloads-Usage
Citations
Library holdings
Video plays
Code forks
News media
Amazon reviews
Goodreads
…
Citations
News media
Policy documents
Open peer review
(F1000, Publons)
YouTube
Open Syllabus
Patents
…
The altmetric toolbox

9. The altmetric toolbox
TYPES OF ALTMETRIC PROVIDERS
• ALL IN ONE
− Almetric.com ← SPRINGER NATURE
− PlumX Analytics ← ELSEVIER
− CrossRef Event Data
− Lagotto ← PLOS
• BY DATA SOURCE
− Mendeley ← ELSEVIER
− Twitter
− Facebook
− ResearchGate

10. The altmetric toolbox
Social media
• Twitter, Facebook, Google Plus… Tweets, retweets, likes, users
Reference managers
• Mendeley, Zotero, ReadCube… Bookmarks, collections, readers
Research blogging
• Publishers, institutional, personal… Mentions, comments, links
OA content
• PLOS journals, DataCite, Github… Downloads, comments, views
SOURCES INDICATORS

11. ALTMETRIC.COM
SOURCES AND INDICATORS
• Public policy documents
• Blogs
• Mainstream media
• Citations (web of science / scopus)
• Online reference managers (mendeley)
• Research highlights (F1000)
• Post-publication peer-review platforms (pubpeer, publons)
• Social Media (Facebook public pages, Twitter, Google+, LinkedIn, Sina,
Weibo and Pinterest)
• Open Syllabus Project
• Multimedia y otros (YouTube, Reddit y Q&A)

12. PLUMX ANALYTICS
SOURCES AND INDICATORS

13. LAGOTTO
SOURCES AND INDICATORS
Viewed
• PubMed Central Usage Stats
• Copernicus
Discussed
• Facebook, Twitter
• Blogs
• Wikipedia
• Reddit
• OpenEdition
Saved
• CiteULike
• Mendeley
Cited
• CrossRef, PubMed
• Scopus
• WoS
• DataCite
• Europe PMC
Recommended
• F1000 Prime
PLOS only
• Usage
• Figshare usage
• Comments

14. CROSSREF EVENT DATA
SOURCES AND INDICATORS

15. The altmetric toolbox
Zahedi & Costas, 2018

16. The altmetric toolbox
REASONS BEHIND THESE DIFFERENCES
• Data collection. Use of different APIs to access primary sources
• Aggregation and reporting. Differing strategies to identify mentions
and aggregating identifiers (i.e., DOIs, URLs)
• Data update. Update of data daily/weekly…
ALTMETRIC INDICATORS ARE UNSTABLE!!
Zahedi & Costas, 2018

17. The altmetric toolbox
Three tweets linking to three
different versions of the
same publication

18. The altmetric toolbox
Different versions of a publication
merged by Altmetric.com

19. Design of altmetric solutions
SOME CONSIDERATIONS
1. Goal
2. Altmetric sources ≈ Phenomenon analyzed
3. Selection of indicators
4. Selection of approach

20. Design of altmetric solutions
MONITORING
• The goal is to learn what is going
on
• Use of metrics should not ignore
uncertainty
• No direct consequences of its
results
• Superficial and exploratory
EVALUATING
• The goal is to assess if an
expected behavior/results is
taking place
• Altmetrics used as support to
other approaches
• Useful when combining with
qualitative analysis in case
studies

21. Design of altmetric solutions
INTERPRETING ALTMETRICS
CASE I. PROXY AND INDICATOR
ARE NOT ALIGNED
Twitter mentions as a proxy for
engagement with or consumption of
scientific literature
Robinson-Garcia et al., 2017

22. Design of altmetric solutions
INTERPRETING ALTMETRICS
CASE II. UNREPORTED CHANGES
IN ALTMETRIC PROVIDERS
Torres-Salinas, Robinson-Garcia & Gorraiz, Forthcoming

23. Design of altmetric solutions
INTERPRETING ALTMETRICS
CASE III. PROXY ALIGNED BUT
FOR ALL THE WRONG REASONS
Costas @ ESSS Granada 2016

24. Design of altmetric solutions
INTERPRETING ALTMETRICS
CASE IV. ERRORS IN DATA
RETRIEVAL

25. Design of altmetric solutions
INTERPRETING ALTMETRICS
CASE V. MANY FALSE NEGATIVES
Paper is not linked in the tweet
but news media covering the
paper, therefore, this mention
is not tracked

26. Design of altmetric solutions
SELECTING INDICATORS
Bibliometric approach Altmetric approach

27. Design of altmetric solutions
SELECTING INDICATORS
Academics
SMEs, Non-profits,
Hospital,
Gov. Agencies,
Associations, Large
firms
SocietyAltmetrics

28. Design of altmetric solutions
SELECTING INDICATORS
Altmetric approach Relational approach

29. Design of altmetric solutions
APPROACHES
• Selection of data sources -> Primary sources vs. Aggregators
• Reporting -> Unidimensional vs. Relational visualizations
• Interpreting -> Numbers speak vs. Contextualization

30. Altmetrics in practice
MONITORING
http://knowmetrics.org/

31. Altmetrics in practice
MONITORING
Torres-Salinas
et al. 2016

32. Altmetrics in practice
MONITORING
Wouters, Zahedi & Costas, 2018

33. Altmetrics in practice
MONITORING

34. Altmetrics in practice
EVALUATING
PROFILING SCHOLARS
HIGH SOC | HIGH SCI
D’Este, Ramos-Vielba &
Robinson-Garcia, 2018
Research area (topics) Nutrition and food science
(food technology, viticulture & enology,
Mediterranean-style diet)
Publication record 74 pubs – 46% Top 10% Highly cited
Altmetric record 63 pubs in social media – 21 policy briefs | 411
news | 186 blogs
Her work appears on scientific news (e.g.):
- Mediterranean diet
- Effects of cooking time of tomato sauce on health

35. Altmetrics in practice
EVALUATING
PROFILING SCHOLARS
Robinson-Garcia et al.,
2018
Public sector
NGO
Unknown
Academia
Private sector
Politicians
Social spheres of
interest of researchers

36. Altmetrics in practice
EVALUATING
ANALYZING RECEPTION OF
RESEARCH INITIATIVES

37. Altmetrics in practice
EVALUATING
ANALYZING RECEPTION OF
RESEARCH INITIATIVES

38. Practical applications of altmetrics
Nicolas Robinson-Garcia @nrobinsongarcia

39. Design of altmetric solutions
OTHER EXAMPLES
http://almreports.plos.org

40. References
D’Este, P., Ramos-Vielba, I., Robinson-Garcia, N. Reconciling scientific impact and societal relevance: The role of pro-social motivation
and productive interactions. XVI Triple Helix Conference, Sept. 5-8, Manchester, United Kingdom
Priem, J., Groth, P., & Taraborelli, D. (2012). The Altmetrics Collection. PLOS ONE, 7(11), e48753.
https://doi.org/10.1371/journal.pone.0048753
Robinson-Garcia, N., van Leeuwen, T. N., & Rafols, I. (2018). Using Almetrics for Contextualised Mapping of Societal Impact: From Hits
to Networks. Science and Public Policy. https://doi.org/10.1093/scipol/scy024
Robinson-Garcia, N., Costas, R., Isett, K., Melkers, J., & Hicks, D. (2017). The unbearable emptiness of tweeting—About journal articles.
PLOS ONE, 12(8), e0183551. https://doi.org/10.1371/journal.pone.0183551
Torres-Salinas, D., Gumpenberger, C., & Gorraiz, J. (2017). PlumX As a Potential Tool to Assess the Macroscopic Multidimensional
Impact of Books. Frontiers in Research Metrics and Analytics, 2. https://doi.org/10.3389/frma.2017.00005
Torres-Salinas, D., Robinson-Garcia, N., & Gorraiz, J. (2017). Filling the citation gap: measuring the multidimensional impact of the
academic book at institutional level with PlumX. Scientometrics, 113(3), 1371–1384.
Torres-Salinas, D., Robinson-Garcia, N., & Jiménez-Contreras, E. (2016). Can we use altmetrics at the institutional level? A case study
analysing the coverage by research areas of four Spanish universities. In STI 2016 Conference: Peripheries, frontiers and beyond.
Valencia. Retrieved from http://arxiv.org/abs/1606.00232
Wouters, P., Zahedi, Z., & Costas, R. (2018). Social media metrics for new research evaluation. In W. Glänzel, H. F. Moed, U. Schmoch, &
M. Thelwall (Eds.), Handbook of Quantitative Science and Technology Research. Springer. Retrieved from
https://arxiv.org/abs/1806.10541
Zahedi, Z., & Costas, R. (2018). General discussion of data quality challenges in social media metrics: Extensive comparison of four
major altmetric data aggregators. PLOS ONE, 13(5), e0197326. https://doi.org/10.1371/journal.pone.0197326