Your SlideShare is downloading. ×
Tweets and Mendeley readers: Two different types of article level metrics
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Tweets and Mendeley readers: Two different types of article level metrics

2,476

Published on

Presentation at APE2014

Presentation at APE2014

Published in: Education, Technology, Business
0 Comments
14 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,476
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
35
Comments
0
Likes
14
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Tweets and Mendeley readers Two different types of article level metrics Stefanie Haustein stefanie.haustein@umontreal.ca @stefhaustein
  • 2. Overview •  Altmetrics •  increasing use •  meaning? •  Aim of the studies •  Data sets and methods •  Results •  documents •  correlations •  disciplines •  Conclusions & outlook
  • 3. Altmetrics: increasing use •  social media activity around scholarly articles growing by 5% to 10% per month (Adie & Roe, 2013) •  Mendeley and Twitter largest altmetrics sources •  Mendeley •  521 million bookmarks •  2.7 million users •  32% increase of users from 09/2012 to 09/2013 •  Twitter •  500 million tweets per day •  230 million active users •  39% increase of users from 09/2012 to 09/2013 Adie, E. & Roe, W. (2013). Altmetric: Enriching Scholarly Content with Article-level Discussion and Metrics. Learned Publishing, 26(1), 11-17. Mendeley statistics based on monthly user counts from 10/2010 to 01/2014 on the Mendeley website accessed through the Internet Archive Twitter statistics: https://business.twitter.com/whos-twitter and http://www.sec.gov/Archives/edgar/data/1418091/000119312513400028/d564001ds1a.htm
  • 4. Altmetrics: meaning? •  ultimate goals •  similar to but more timely than citations Ø  predicting scientific impact •  different, broader impact than captured by citations Ø  measuring societal impact •  impact of various outputs Ø  “value all research products” Piwowar (2013) Piwowar, H. (2013). Value all research products. Nature, 493(7431), 159.
  • 5. Altmetrics: meaning? •  Altmetrics are “representing very different things” (Lin & Fenner, 2013) •  unclear what exactly they measure: •  scientific impact •  social impact •  “buzz” Lin, J. & Fenner, M. (2013). Altmetrics in evolution: Defining and redefining the ontology of article-level metrics. Information Standards Quarterly, 25(2), 20-26.
  • 6. Altmetrics: meaning? ad-hoc classifications need to be supported by research
  • 7. Altmetrics: meaning? scientist on Twitter tweeting scientific paper in non-scholarly manner: •  scientific impact? •  social impact? •  buzz?
  • 8. Altmetrics: meaning?
  • 9. Aim of the studies •  providing empirical evidence of Mendeley reader counts and tweets of scholarly documents for a large data set •  generate knowledge about factors influencing popularity of scholarly documents on Mendeley and Twitter •  analyzing the following research questions: •  •  •  •  What is the relationship between social-media and citation counts? How do social-media metrics differ? Which papers are highly tweeted or highly bookmarked? How do these aspects differ across scientific disciplines? Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (2014). Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature. Journal of the Association for Information Sciences and Technology. Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (submitted). Tweets vs. Mendeley readers: How do these two social media metrics differ? IT-Information Technology.
  • 10. Aim of the studies •  large-scale analysis of tweets and Mendeley readers of biomedical papers •  Twitter and Mendeley coverage •  Twitter and Mendeley user rates •  correlation with citations •  discovering differences between: •  documents •  disciplines & specialties Ø  providing an empirical framework to compare coverage, correlations and user rates
  • 11. Data sets & methods •  1.4 million PubMed papers covered by WoS •  publication years: 2010-2012 •  document types: articles & reviews •  matching of WoS and PubMed •  tweet counts collected by Altmetric.com •  collection based on PMID, DOI, URL •  matching WoS via PMID •  Mendeley readership data collected via API •  matching title and author names •  journal-based matching of NSF classification
  • 12. Data sets & methods: framework
  • 13. Data sets & methods: age biases Current biases influencing correlation coefficients Ø  compare documents of similar age Ø  normalize for age differences
  • 14. Results: documents •  Twitter coverage is quite low but increasing •  correlation between tweets and citations is very low Publication year Twitter coverage Papers (T≥1) Spearman's ρ Mean Median Maximum T2010 C2010 2.4% 13,763 .104** 2.1 18.3 1 7 237 3,922 T2011 C2011 10.9% 63,801 .183** 2.8 5.7 1 2 963 2,300 T2012 C2012 20.4% 57,365 .110** 2.3 1.3 1 0 477 234 9.4% 134,929 .114** 2.5 5.1 1 1 963 3,922 T2010-2012 C2010-2012
  • 15. Results: documents Top 10 tweeted documents: catastrophe & topical / web & social media / curious story scientific discovery / health implication / scholarly community Article Journal C T Hess et al. (2011). Gain of chromosome band 7q11 in papillary thyroid carcinomas of young patients is associated with exposure to low-dose irradiation PNAS 9 963 Yasunari et al. (2011). Cesium-137 deposition and contamination of Japanese soils due to the Fukushima nuclear accident PNAS 30 639 Sparrow et al. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips Science 11 558 Onuma et al. (2011). Rebirth of a Dead Belousov–Zhabotinsky Oscillator Journal of Physical Chemistry A -- 549 Silverberg (2012). Whey protein precipitating moderate to severe acne flares in 5 teenaged athletes Cutis -- 477 Wen et al. (2011). Minimum amount of physical activity for reduced mortality and extended life expectancy: a prospective cohort study Lancet 51 419 Kramer (2011). Penile Fracture Seems More Likely During Sex Under Stressful Situations Journal of Sexual Medicine -- 392 Newman & Feldman (2011). Copyright and Open Access at the Bedside New England Journal of Medicine 3 332 Reaves et al. (2012). Absence of Detectable Arsenate in DNA from Arsenate-Grown GFAJ-1 Cells Science 5 323 Bravo et al. (2011). Ingestion of Lactobacillus strain regulates emotional behavior and central GABA receptor expression in a mouse via the vagus nerve PNAS 31 297
  • 16. Results: correlations PubMed papers covered by Web of Science (PY=2011) Spearman correlations between citations (C), Mendeley readers (R) and tweets (T) for all papers published in 2011 (A, n=586,600), for papers with respectively at least one citation (B, n=410,722), one Mendeley reader (C, n=390,190) or one tweet (D, n=63,800), one Mendeley reader and one tweet (E, n=45,229) and one citation, one Mendeley reader and one tweet (F, n=36,068). All results are significant at the 0.01 level (two-tailed).
  • 17. Results: disciplines PubMed papers covered by Web of Science 2010-2012
  • 18. Altmetrics: disciplinary biases x-axis: coverage of specialty on platform y-axis: correlation between social media counts and citations bubble size: intensity of use based on mean social media count rate
  • 19. Results: disciplines General Biomedical Research papers 2011 Scatterplot of number of citations and number of tweets (A, ρ=0.181**) and Mendeley readers (B, ρ=0.677**), bubble size represents number of Mendeley readers (A) and tweets (B). The respective three most tweeted (A) and read (B) papers are labeled showing the first author.
  • 20. Results: disciplines Public Health papers 2011 Scatterplot of number of citations and number of tweets (A, ρ=0.074**) and Mendeley readers (B, ρ=0.351**) for papers published in Public Health in 2011. The respective three most tweeted (A) and read (B) papers are labeled showing the first author.
  • 21. Conclusions & outlook •  uptake, usage intensity and correlations differ between disciplines and research fields Ø  social media counts of papers from different fields are not directly comparable •  citations, Mendeley readers and tweets reflect different kind of impact on different social groups •  Mendeley seems to mirror use of a broader but still academic audience, largely students and postdocs •  Twitter seems to reflect the popularity among a general public and represents a mix of societal impact, scientific discussion and buzz Ø  the number of Mendeley readers and tweets are two distinct social media metrics
  • 22. Conclusions & outlook •  before applying social media counts in information retrieval and research evaluation further research is needed: Ø  identifying different factors influencing popularity of scholarly documents on social media Ø  analyzing uptake and usage intensity in various disciplines Ø  differentiating between audiences and engagements
  • 23. Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (in press). Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the Association for Information Sciences and Technology. Haustein, S., Larivière, V., Thelwall, M., Amyot, D., & Peters, I. (submitted). Tweets vs. Mendeley readers: How do these two social media metrics differ? IT-Information Technology. Thank you for your attention! Questions? Stefanie Haustein stefanie.haustein@umontreal.ca @stefhaustein

×