Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Empirical analyses of scientific papers and researchers on Twitter: Results of two studies

1,289 views

Published on

presentation held at PLoS ALM Workshop 2013 in San Francisco
http://article-level-metrics.plos.org/alm-workshop-2013-preliminary-program/
presenting results of two Twitter studies: 1.4 PubMed papers and 37 astrophysicists on Twitter

Published in: Technology
  • Be the first to comment

Empirical analyses of scientific papers and researchers on Twitter: Results of two studies

  1. 1. Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent Larivière, Isabella Peters, Cassidy R. Sugimoto, & Mike Thelwall
  2. 2. Background •  when Garfield created SCI, sociologists of science analyzed meaning of publications and citations (Merton, Zuckerman, Cole & Cole, etc.) •  sociological research •  What is it to publish a paper? •  What are the reasons to cite? •  empirical bibliometric research •  disciplinary differences in publication and citation behavior •  delay and obsolescence patterns
  3. 3. Background •  empirical studies helped sociologists to understand structure and norms of science •  for bibliometricians, studies provided a theoretical framework and legitimation to use citation analysis in research evaluation •  knowledge about disciplinary differences and obsolescence patterns helped to normalize statistics and create more appropriate indicators
  4. 4. Background •  recently social-media metrics have become important in the scholarly world •  suggestions to complement (or even replace) citation analysis by so-called ”altmetrics“ •  broader audience (not just citing authors) •  more timely •  however, similar to bibliometrics in the 1960s, little is known about the actual meaning of various social-media counts
  5. 5. Research questions •  What is the relationship between social-media and citation counts? •  How do various social-media metrics differ? •  Why are papers tweeted, bookmarked, liked…? •  Who tweets (bookmarks, likes…) scientific papers? •  How do these aspects differ across scientific disciplines? Two case studies on Twitter •  large-scale analysis of tweets of biomedical papers •  in-depth analysis of astrophysicists on Twitter
  6. 6. Aim of the study •  large-scale analysis of tweets of biomedical papers •  Twitter coverage •  Twitter citation rates (tweets per paper) •  correlation with citations •  discovering differences between: •  documents •  journals •  disciplines & specialties !  providing empirical framework to understand the extent to which biomedical journal articles are tweeted Study I: Tweeting biomedicine Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (in press). Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the American Society for Information Science and Technology, http://arxiv.org/abs/1308.1838.
  7. 7. Data sets & methods •  1.4 million PubMed papers covered by WoS •  publication years: 2010-2012 •  document types: articles & reviews •  matching of WoS and PubMed •  tweet counts collected by Altmetric.com •  collection based on PMID, DOI, URL •  matching WoS via PMID •  journal-based matching of NSF classification •  tweets per article, Twitter coverage and correlation with citations for: •  journals •  NSF disciplines and specialties Study I: Tweeting biomedicine
  8. 8. Data sets & methods: framework Study I: Tweeting biomedicine
  9. 9. Data sets & methods: correlations Study I: Tweeting biomedicine PY=2010 PY=2011 PY=2012
  10. 10. Results: documents Study I: Tweeting biomedicine Publication year Twitter coverage Papers (T≥1) Spearman's ρ Mean Median Maximum T2010 2.4% 13,763 .104** 2.1 1 237 C2010 18.3 7 3,922 T2011 10.9% 63,801 .183** 2.8 1 963 C2011 5.7 2 2,300 T2012 20.4% 57,365 .110** 2.3 1 477 C2012 1.3 0 234 T2010-2012 9.4% 134,929 .114** 2.5 1 963 C2010-2012 5.1 1 3,922 •  Twitter coverage is quite low but increasing •  correlation between tweets and citations is very low
  11. 11. Results: documents Study I: Tweeting biomedicine Article Journal C T Hess et al. (2011). Gain of chromosome band 7q11 in papillary thyroid carcinomas of young patients is associated with exposure to low-dose irradiation PNAS 9 963 Yasunari et al. (2011). Cesium-137 deposition and contamination of Japanese soils due to the Fukushima nuclear accident PNAS 30 639 Sparrow et al. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips Science 11 558 Onuma et al. (2011). Rebirth of a Dead Belousov–Zhabotinsky Oscillator Journal of Physical Chemistry A -- 549 Silverberg (2012). Whey protein precipitating moderate to severe acne flares in 5 teenaged athletes Cutis -- 477 Wen et al. (2011). Minimum amount of physical activity for reduced mortality and extended life expectancy: a prospective cohort study Lancet 51 419 Kramer (2011). Penile Fracture Seems More Likely During Sex Under Stressful Situations Journal of Sexual Medicine -- 392 Newman & Feldman (2011). Copyright and Open Access at the Bedside New England Journal of Medicine 3 332 Reaves et al. (2012). Absence of Detectable Arsenate in DNA from Arsenate-Grown GFAJ-1 Cells Science 5 323 Bravo et al. (2011). Ingestion of Lactobacillus strain regulates emotional behavior and central GABA receptor expression in a mouse via the vagus nerve PNAS 31 297 Top 10 tweeted documents: catastrophe & topical / web & social media / curious story scientific discovery / health implication / scholarly community
  12. 12. Results: journals • 97.7% of 3,812 journals at least tweeted once • two-thirds of journals have coverage below 20% and Twitter citation rate < 2.0 • high Twitter citation rates often caused by few papers • high coverage and Twitter citation rates for general journals Study I: Tweeting biomedicine
  13. 13. Results: disciplines Study I: Tweeting biomedicine
  14. 14. Results: specialties Study I: Tweeting biomedicine • specialties differ in terms of coverage, Twitter citation rate and correlations with citations • 47 of 61 specialties show low positive, 3 negative and 13 no correlation bubblesize=Twittercitationrate
  15. 15. Aim of the study •  in-depth analysis of astrophysicists on Twitter •  number of tweets, followers, retweets •  characteristics of tweets: RTs, @messages, #hashtags, URLs •  comparison with scientific output •  publications •  citations •  comparison of tweet and publication content !  provide evidence in how far astrophysicists on Twitter use Twitter for scholarly communiation Study II: Astrophysicists on Twitter Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I., (submitted). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. Aslib Proceedings.
  16. 16. Data sets & methods •  37 astrophysicists on Twitter identified by Holmberg & Thelwall (2013) •  web searches to identify person behind account •  publications in WoS journals •  publication years: 2008-2012 •  author disambiguation •  Twitter account information •  68,232 of 289,368 tweets downloaded and analyzed: •  number of RTs per tweet •  % of tweets that are RTs •  % of tweets containing #hashtags, @usernames, URLs Study II: Astrophysicists on Twitter Holmberg, K., & Thelwall, M. (2013). Disciplinary differences in Twitter scholarly communication. In: Proceedings of ISSI 2013 – 14th International Conference of the International Society for Scientometrics and Informetrics, Vienna, Austria (Vol. 1, pp. 567-582).
  17. 17. Data sets & methods •  grouping astrophysicists according to tweeting and publication behavior •  analyzing differences of tweeting characteristics between user groups Study II: Astrophysicists on Twitter Selected astrophysicists (N=37)! tweet rarely (0.0-0.1 tweets per day)! tweet occasionally (0.1-0.9)! tweet regularly (1.2-2.9)! tweet frequently (3.7-58.2)! total (publishing activity)! do not publish (0 publications 2008-2012)! --! --! 1! 5! 6! publish occasionally (1-9)! 4! 3! 4! 2! 13! publish regularly (14-37)! --! 5! 5! 3! 13! publish frequently (46-112)! 1! 3! 1! --! 5! total (tweeting activity)! 5! 11! 11! 10! 37!
  18. 18. Data sets & methods •  comparison of tweet and publication content •  extraction of noun phrases from tweets and abstracts •  limited to 18 most frequently publishing astrophysicists to ensure certain number of abstracts •  analyzing overlap of character strings •  calculating similarity with cosine per person and overall Study II: Astrophysicists on Twitter Selected astrophysicists (N=37)! tweet rarely (0.0-0.1 tweets per day)! tweet occasionally (0.1-0.9)! tweet regularly (1.2-2.9)! tweet frequently (3.7-58.2)! total (publishing activity)! publish regularly (14-37)! --! 5! 5! 3! 13! publish frequently (46-112)! 1! 3! 1! --! 5! total (tweeting activity)! 1! 8! 6! 3! 18!
  19. 19. Results: correlations •  comparison of Twitter and publication activity and impact •  publications and tweets per day: ρ=−0.339* •  citation rate and tweets per day: ρ=−0.457** •  citation rate and RT rate: ρ=0.077 Study II: Astrophysicists on Twitter
  20. 20. Results: characteristics Study II: Astrophysicists on Twitter Mean share of tweets containing at least one user name or URL per person per group
  21. 21. Results: content similarity Study II: Astrophysicists on Twitter •  overall similarity between abstracts and tweets is low •  cosine=0.081 •  4.1% of 50,854 tweet NPs in abstracts •  16.0% of 12,970 abstract NPs in tweets •  Twitter coverage among most frequent abstract terms is high, although this differs between users •  97,1% of 104 most frequent noun phrases on Twitter
  22. 22. Conclusions •  Twitter coverage of biomedical papers is low but increasing •  number of tweets per paper varies between journals, disciplines, specialties and from year to year !  tweet counts need to be normalized accordingly •  correlations between tweet and citation counts are low (biomedical papers) or even moderately negative (astrophysicists) !  tweets cannot replace citations as measures of scientific impact !  challenge is to differentiate between high tweet counts because of value (to scientists and/or the general public) and curiosity
  23. 23. Outlook •  user surveys and qualitative research to investigate who is using scholarly content on social media and why •  empirical large-scale studies on other metrics
  24. 24. Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (in press). Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the American Society for Information Science and Technology. Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I., (submitted). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. Aslib Proceedings. Stefanie Haustein Thank you for your attention! Questions? stefanie.haustein@umontreal.ca @stefhaustein

×