Tweets and Mendeley readers: Two different types of article level metrics
Tweets and Mendeley readers
Two different types of article level metrics
• increasing use
• Aim of the studies
• Data sets and methods
• Conclusions & outlook
Altmetrics: increasing use
• social media activity around scholarly articles growing by
5% to 10% per month (Adie & Roe, 2013)
• Mendeley and Twitter largest altmetrics sources
• 521 million bookmarks
• 2.7 million users
• 32% increase of users from 09/2012 to 09/2013
• 500 million tweets per day
• 230 million active users
• 39% increase of users from 09/2012 to 09/2013
Adie, E. & Roe, W. (2013). Altmetric: Enriching Scholarly Content with Article-level Discussion and Metrics. Learned Publishing, 26(1), 11-17.
Mendeley statistics based on monthly user counts from 10/2010 to 01/2014 on the Mendeley website accessed through the Internet Archive
Twitter statistics: https://business.twitter.com/whos-twitter and http://www.sec.gov/Archives/edgar/data/1418091/000119312513400028/d564001ds1a.htm
• ultimate goals
• similar to but more timely than citations
Ø predicting scientific impact
• different, broader impact than captured by citations
Ø measuring societal impact
• impact of various outputs
Ø “value all research products”
Piwowar, H. (2013). Value all research products. Nature, 493(7431), 159.
• Altmetrics are “representing very different things”
(Lin & Fenner, 2013)
• unclear what exactly they measure:
• scientific impact
• social impact
Lin, J. & Fenner, M. (2013). Altmetrics in evolution: Defining and redefining the ontology of article-level metrics. Information Standards
Quarterly, 25(2), 20-26.
need to be
• scientific impact?
• social impact?
Aim of the studies
• providing empirical evidence of Mendeley reader counts
and tweets of scholarly documents for a large data set
• generate knowledge about factors influencing popularity of
scholarly documents on Mendeley and Twitter
• analyzing the following research questions:
What is the relationship between social-media and citation counts?
How do social-media metrics differ?
Which papers are highly tweeted or highly bookmarked?
How do these aspects differ across scientific disciplines?
Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (2014).
Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature.
Journal of the Association for Information Sciences and Technology.
Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (submitted). Tweets vs.
Mendeley readers: How do these two social media metrics differ? IT-Information Technology.
Aim of the studies
• large-scale analysis of tweets and Mendeley readers of
• Twitter and Mendeley coverage
• Twitter and Mendeley user rates
• correlation with citations
• discovering differences between:
• disciplines & specialties
Ø providing an empirical framework to compare coverage,
correlations and user rates
Data sets & methods
• 1.4 million PubMed papers covered by WoS
• publication years: 2010-2012
• document types: articles & reviews
• matching of WoS and PubMed
• tweet counts collected by Altmetric.com
• collection based on PMID, DOI, URL
• matching WoS via PMID
• Mendeley readership data collected via API
• matching title and author names
• journal-based matching of NSF classification
Data sets & methods: age biases
Current biases influencing correlation coefficients
Ø compare documents of similar age
Ø normalize for age differences
• Twitter coverage is quite low but increasing
• correlation between tweets and citations is very low
Top 10 tweeted documents:
catastrophe & topical / web & social media / curious story
scientific discovery / health implication / scholarly community
Hess et al. (2011). Gain of chromosome band 7q11 in papillary thyroid carcinomas of young patients
is associated with exposure to low-dose irradiation
Yasunari et al. (2011). Cesium-137 deposition and contamination of Japanese soils due to the
Fukushima nuclear accident
Sparrow et al. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at
Onuma et al. (2011). Rebirth of a Dead Belousov–Zhabotinsky Oscillator
Journal of Physical
Silverberg (2012). Whey protein precipitating moderate to severe acne flares in 5 teenaged athletes
Wen et al. (2011). Minimum amount of physical activity for reduced mortality and extended life
expectancy: a prospective cohort study
Kramer (2011). Penile Fracture Seems More Likely During Sex Under Stressful Situations
Journal of Sexual
Newman & Feldman (2011). Copyright and Open Access at the Bedside
Journal of Medicine
Reaves et al. (2012). Absence of Detectable Arsenate in DNA from Arsenate-Grown GFAJ-1 Cells
Bravo et al. (2011). Ingestion of Lactobacillus strain regulates emotional behavior and central GABA
receptor expression in a mouse via the vagus nerve
PubMed papers covered by Web of Science (PY=2011)
Spearman correlations between citations (C), Mendeley readers (R) and tweets (T) for all papers published in
2011 (A, n=586,600), for papers with respectively at least one citation (B, n=410,722), one Mendeley reader (C,
n=390,190) or one tweet (D, n=63,800), one Mendeley reader and one tweet (E, n=45,229) and one citation, one
Mendeley reader and one tweet (F, n=36,068). All results are significant at the 0.01 level (two-tailed).
PubMed papers covered by Web of Science 2010-2012
Altmetrics: disciplinary biases
intensity of use
based on mean
General Biomedical Research papers 2011
Scatterplot of number of citations and number of tweets (A, ρ=0.181**) and Mendeley readers (B, ρ=0.677**),
bubble size represents number of Mendeley readers (A) and tweets (B). The respective three most tweeted (A)
and read (B) papers are labeled showing the first author.
Public Health papers 2011
Scatterplot of number of citations and number of tweets (A, ρ=0.074**) and Mendeley readers (B, ρ=0.351**)
for papers published in Public Health in 2011. The respective three most tweeted (A) and read (B) papers are
labeled showing the first author.
Conclusions & outlook
• uptake, usage intensity and correlations differ between
disciplines and research fields
Ø social media counts of papers from different fields are not
• citations, Mendeley readers and tweets reflect different
kind of impact on different social groups
• Mendeley seems to mirror use of a broader but still academic
audience, largely students and postdocs
• Twitter seems to reflect the popularity among a general public
and represents a mix of societal impact, scientific discussion
Ø the number of Mendeley readers and tweets are two
distinct social media metrics
Conclusions & outlook
• before applying social media counts in information
retrieval and research evaluation further research is
Ø identifying different factors influencing popularity of
scholarly documents on social media
Ø analyzing uptake and usage intensity in various disciplines
Ø differentiating between audiences and engagements
Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (in press). Tweeting
biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the
Association for Information Sciences and Technology.
Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (submitted). Tweets vs. Mendeley
readers: How do these two social media metrics differ? IT-Information Technology.
Thank you for your attention!