Measuring Twitter activity of
arXiv e-prints and published papers
Stefanie Haustein, Timothy D. Bowman, Benoît Macaluso,
C...
Introduction
•  increase of Twitter use
•  230 million active users, 500 million tweets per day
•  39% increase of users f...
Introduction
•  social media activity around scholarly articles grows
5% to 10% per month (Adie & Roe, 2013)
•  scholarly ...
Research questions
Twitter impact of e-prints and published versions
Research questions
Twitter impact of e-prints and published versions
•  How many times are e-prints and published versions...
Methods: direct match arXiv & Altmetric.com
Results: direct match arXiv & Altmetric.com
Methods: indirect match through WoS
exact&and&fuzzy&
•  DOI&
•  0tles&
•  author&names&
•  abstracts&
•  0tle&length&
Results: indirect match through WoS
5.8%&documents&&
w/&>1&Altmetric&id&
+1,507&papers&
w/&more&tweets&
+4,600&TwiKer&coun...
Results: indirect match through WoS
Preliminary findings
•  presence of DOIs in arXiv metadata influences ability to
match Twitter activity of e-print and pub...
Preliminary findings
•  differences between arXiv primary categories
•  high Twitter rate in Quantitative Biology (8.4) an...
Outlook: automated tweets
•  search for “arXiv” in Twitter user name and description:
•  many more automated accounts poss...
Outlook: automated tweets
25% 100%
27%
•  distinguishing type of tweet based on content
e.g., similarity with article titl...
Outlook: next steps
•  detecting bots and cyborgs
•  How much of Twitter activity in scholarly communication
do they accou...
Stefanie Haustein
Thank you for your attention!
Questions?
stefanie.haustein@umontreal.ca
@stefhaustein
Thanks to Euan Adi...
References
Adie, E. & Roe, W. (2013). Altmetric: Enriching Scholarly Content with Article-level Discussion and Metrics. Le...
Upcoming SlideShare
Loading in …5
×

Stefanie Haustein, Timothy D. Bowman, Benoît Macaluso, Cassidy R. Sugimoto & Vincent Larivière: Measuring Twitter activity of arXiv e-prints and published papers

1,575 views

Published on

Presentation at #altmetrics14, #WebSci14, Bloomington, IN
altmetrics.org/altmetrics14/

Published in: Technology
  • Be the first to comment

Stefanie Haustein, Timothy D. Bowman, Benoît Macaluso, Cassidy R. Sugimoto & Vincent Larivière: Measuring Twitter activity of arXiv e-prints and published papers

  1. 1. Measuring Twitter activity of arXiv e-prints and published papers Stefanie Haustein, Timothy D. Bowman, Benoît Macaluso, Cassidy R. Sugimoto & Vincent Larivière Canada Research Chair on the Transformations of Scholarly Communication École de bibliothéconomie et des sciences de l’information @stefhaustein
  2. 2. Introduction •  increase of Twitter use •  230 million active users, 500 million tweets per day •  39% increase of users from 09/2012 to 09/20131 •  16% of US, 3% of world population in 20131 •  19% of US internet users 01/20142 •  uptake by researchers •  1 in 40 university faculty member in US and UK have Twitter account (Priem & Costello, 2010) •  9% of researchers use Twitter for work (Rowlands et al., 2011) •  15% of German university faculty members, 70% of which at least occasionally in professional context (Pscheida et al., 2013) 1 Twitter statistics calculated based on data from: http://www.sec.gov/Archives/edgar/data/1418091/000119312513400028/d564001ds1a.htm and http://www.census.gov/population/international/data/ 2 Pew Research Center’s Internet Project surveys, 2010-2014 http://www.pewresearch.org/fact-tank/2014/06/11/can-twitter-survive-in-a-facebook-world-the-key-is-being-different/
  3. 3. Introduction •  social media activity around scholarly articles grows 5% to 10% per month (Adie & Roe, 2013) •  scholarly documents on Twitter 1.6% of WoS papers with DOIs 2005-2011 (Zahedi, Costas & Wouters, 2014) 13.3% of WoS papers with DOIs 07-12/2011 (Costas, Zahedi & Wouters, 2014) 20.4% of PubMed/WoS 2012 (Haustein et al., 2014) 21.5% of WoS papers with DOIs 2012 (Costas, Haustein & Larivière, in prep.) •  tweeting peaks shortly after publication •  of the published version in the journal of record (Eysenbach, 2011) •  of the e-print (Shuai, Pepe & Bollen, 2012)
  4. 4. Research questions Twitter impact of e-prints and published versions
  5. 5. Research questions Twitter impact of e-prints and published versions •  How many times are e-prints and published versions tweeted? •  When does the Twitter activity occur? •  Do Twitter audiences differ between the two versions? •  To what extent are both activities picked up by Altmetric.com? !  Should both kinds of activities been taken into account or are they equivalent?
  6. 6. Methods: direct match arXiv & Altmetric.com
  7. 7. Results: direct match arXiv & Altmetric.com
  8. 8. Methods: indirect match through WoS exact&and&fuzzy& •  DOI& •  0tles& •  author&names& •  abstracts& •  0tle&length&
  9. 9. Results: indirect match through WoS 5.8%&documents&& w/&>1&Altmetric&id& +1,507&papers& w/&more&tweets& +4,600&TwiKer&counts& +716&tweeted& papers&
  10. 10. Results: indirect match through WoS
  11. 11. Preliminary findings •  presence of DOIs in arXiv metadata influences ability to match Twitter activity of e-print and published paper !  enriching arXiv metadata with DOIs eliminates potential biases •  some improvement through indirect match, but actual differences not as large as expected •  +3.2% tweeted documents, +7.7% tweets •  +7.6% papers with increase in Twitter activity !  Altmetric.com finds most Twitter activity of papers with arXiv e-print !  most tweets refer to arXiv id, not DOI
  12. 12. Preliminary findings •  differences between arXiv primary categories •  high Twitter rate in Quantitative Biology (8.4) and Computer Science (5.0) •  high Twitter coverage in HEP Experiment (81.1%), HEP Theory (72.7%), HEP Phenomenology (69.6%), HEP Lattice (69.2%) •  particular high coverage (44.9%) if compared to other studies !  high presence of automated Twitter accounts! bots and cyborgs triggered by arXiv feeds: @hep_th @hep_ph @hep_ex @hep_lat
  13. 13. Outlook: automated tweets •  search for “arXiv” in Twitter user name and description: •  many more automated accounts possible: •  journals •  publishers !  not equally distributed !  distribution instead of impact account'type& number'(%)' of'accounts& tweets& mean' followers& mean' following& %'of'50,068' tweets& mean'Truthy' BotOrNot'score& arXiv&feed&(bot)& 43&(84.3%)& 87,389& 34.9& 0.6& 8.8%& 33%& topic&feed&(bot)& 4&(7.8%)& 10,040& 527.0& 491.5& 0.1%& 40%& selec0ve& 4&(7.8%)& 3,081& 361.8& 50.5& 1.0%& 46%& '& 51'(100%)& 100,510& 99.1& 43.0& 9.9%& 33%& •  societies / associations •  institutions •  authors
  14. 14. Outlook: automated tweets 25% 100% 27% •  distinguishing type of tweet based on content e.g., similarity with article title (%) engagement distribution 100% 86%
  15. 15. Outlook: next steps •  detecting bots and cyborgs •  How much of Twitter activity in scholarly communication do they account for? •  How are their tweets distributed? •  distinguishing between distribution and engagement •  answering our original research questions
  16. 16. Stefanie Haustein Thank you for your attention! Questions? stefanie.haustein@umontreal.ca @stefhaustein Thanks to Euan Adie and for access to their Twitter data!
  17. 17. References Adie, E. & Roe, W. (2013). Altmetric: Enriching Scholarly Content with Article-level Discussion and Metrics. Learned Publishing, 26(1), 11-17. Costas, R., Zahedi, Z. & Wouters, P. (2014). Do altmetrics correlate with citations? Extensive comparison of altmetric indicators with citations from a multidisciplinary perspective. Journal of the Association for Information Sciences and Technology. arxiv: 1401.4321 Eysenbach, G. (2011) Can tweets predict citations? Metrics of social impact based on twitter and correlation with traditional metrics of scientific impact. Journal of Medical Internet Research, 13, e123. doi: 10.2196/jmir.2012 Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (2014b). Tweeting Biomedicine: An Analysis of Tweets and Citations in the Biomedical Literature. Journal of the Association for Information Sciences and Technology, 65(4), 656-669. doi: 10.1002/asi.23101 Priem, J., & Costello, K. L. (2010). How and why scholars cite on Twitter. Proceedings of the 73th Annual Meeting of the American Society for Information Science and Technology, Pittsburgh, USA. Pscheida, D., Albrecht, S., Herbst, Minet, C. & Köhler, T. (2013). Nutzung von Social Media und onlinebasierten Anwendungen in der Wissenschaft. Erste Ergebnisse des Science 2.0-Survey 2013 des Leibniz-Forschungsverbunds „Science 2.0“ available from: http://www.qucosa.de/fileadmin/data/qucosa/documents/13296/ Science20_Datenreport_2013_PDF_A.pdf Rowlands, I., Nicholas, D., Russell, B., Canty, N., & Watkinson, A. (2011). Social media use in the research workflow. Learned Publishing, 24, 183–195. Shuai, X., Pepe, A., & Bollen, J. (2012). How the scientific community reacts to newly submitted preprints: article downloads, Twitter mentions, and citations. PLOS ONE, 7(11), e47523. doi:10.1371/journal.pone.0047523 Zahedi, Z., Costas, R. & Wouters, P. (2014). How well developed are altmetrics? cross-disciplinary analysis of the presence of 'alternative metrics' in scientific publications. Scientometrics. doi: 10.1007/s11192-014-1264-0

×