Presented at the “Science evaluation in the 21st century and its impact on scientific formation” session of the Biased Science and Alternatives for the Publication System Symposium - Rio de Janeiro, September 15 2014.

1. Alternative metrics
to evaluate scientific impact
Prof. Dra. Nanci Oddone
Programa de Pós-Graduação em Biblioteconomia/UNIRIO
With Iara Vidal & Andréa do Nascimento
Simpósio “Biased Science and Alternatives for the Publication System”
15 de setembro de 2014 ● UFRJ, Rio de Janeiro, Brasil

2. ALTMETRICS
“The creation and study of
new metrics based on the
Social Web for analyzing
and informing scholarship”

3. ORIGINS

4. ORIGINS
• Altmetrics: a manifesto / Jason
Priem, Dario Taraborelli, Paul Groth,
Cameron Neylon (2010)
• Scholarship’s main filters for
importance – peer review, citation
counts, JIF – are failing. Altmetrics
offers possible solutions

5. ALTERNATIVE WAYS
• Alternative Metrics: shares, tags, notes,
comments, bookmarks, social mentions...
• Alternative Products: blogs, raw data,
code, grey literature...
• Alternative Audiences: organizations,
doctors, patients, educators, students,
citizen scientists...

6. MULTIPLE DIMENSIONS
IMPACT
USAGE
Downloads
Views
CITATIONS
PEER
REVIEW
Expert opinion
ALTMETRICS
Storage
Links
Bookmarks
Conversations
Adapted from altmetrics.org

7. ALTMETRIC INDICATORS
Dado Descrição Exemplos
Shares / Mentions
A public post to share news about an
article or research output
Extraído de: KONKIEL, S. Altmetrics: A 21st-Century Solution to Determining Research Quality. Online searcher, v. 37, n. 4, p. 10-15, 2013.
Twitter, Topsy, Facebook, reddit,
news articles, blog posts, Google+,
YouTube, Figshare, Mendeley
Saves / Captures
Social bookmarks, favorites, online
reference managers
Mendeley, CiteULike, Delicious,
Github, Twitter, and Slideshare
Reviews / Social Media
Discusion, including additional
commentary
Faculty of 1000 (F1000), blog posts,
comentários de artigos, comentários
no Facebook
Adaptations
Creation of derivative work using an
article or other research output
Github
Views / Usage
Downloads or views in web services
and social media sites
Figshare, Slideshare, Dryad,
Facebook, YouTube, LinkedIn
Cites Citations in academic literature Pubmed, Scopus, Web of Science
Adaptado de: KONKIEL, S. Altmetrics: A 21st-Century Solution to Determining Research Quality. Online searcher, v. 37, n. 4, p. 10-15, 2013; e
TANAMBAUM, G. Article-Level metrics: a SPARC primer. 2013.

8. ADVANTAGES
• Diversity: diverse metrics, for diverse
products, capturing multiple dimensions
of scientific impact
• Complementing traditional metrics (San
Francisco Declaration on Research
Assessment – DORA, 2012)
• Speed: almost real-time data
• Openness: open data and tools

9. DISADVANTAGES
• Gaming
• Lack of credibility: a measure of
impact or popularity?
• Altmetric sources and indicators are
heterogeneous and ephemerous
• CONTEXT IS KEY!

10. ARTICLE-LEVEL METRICS (ALMS)
• The combination of traditional and
alternative data to capture the
impact of a particular article
• A clearer picture of how articles are
being discussed, shared, and used

11. ALMS VS. ALTMETRICS
ALMs =
attempts to
measure impact
at the article-level
Altmetrics =
refers to data
sources (social
media tools)

12. IMPACT MEASURES
From TANAMBAUM, G. Article-Level metrics: a SPARC primer. 2013.

13. PUBLISHERS & ALTMETRICS
•Elsevier
•Wiley
•EBSCO
•Emerald
•PLOS
•High Wire Press
•BioMed Central
•OJS/PKP
•Nature
•Rede SciELO
•Springer
•(…)

14. TOOLS
Four Main Providers:
•Altmetric
•ImpactStory
•Plum Analytics
•PLOS ALM

19. CHALLENGES
• Need for standards/best practices to
guide data collection and treatment
• Altmetrics Initiative, National
Information Standards Organization
(NISO)
• Need for ALM/altmetrics studies in
different contexts – including
peripheric countries.

20. THANK YOU
Iara Vidal
@iaravps / iaravidalps@gmail.com
Andréa do Nascimento
@deiafix / aandreafg@gmail.com
Prof. Dra. Nanci Oddone
neoddone@gmail.com