Empirical analyses of scientific
papers and researchers on Twitter:
Results of two studies
Stefanie Haustein, Timothy D. B...
Background
•  when Garfield created SCI, sociologists of science
analyzed meaning of publications and citations
(Merton, Z...
Background
•  empirical studies helped sociologists to understand
structure and norms of science
•  for bibliometricians, ...
Background
•  recently social-media metrics have become
important in the scholarly world
•  suggestions to complement (or ...
Research questions
•  What is the relationship between social-media and
citation counts?
•  How do various social-media me...
Aim of the study
•  large-scale analysis of tweets of biomedical papers
•  Twitter coverage
•  Twitter citation rates (twe...
Data sets & methods
•  1.4 million PubMed papers covered by WoS
•  publication years: 2010-2012
•  document types: article...
Data sets & methods: framework
Study I: Tweeting biomedicine
Data sets & methods: correlations
Study I: Tweeting biomedicine
PY=2010 PY=2011 PY=2012
Results: documents
Study I: Tweeting biomedicine
Publication
year
Twitter
coverage
Papers
(T≥1)
Spearman's ρ Mean Median M...
Results: documents
Study I: Tweeting biomedicine
Article Journal C T
Hess et al. (2011). Gain of chromosome band 7q11 in p...
Results: journals
• 97.7% of 3,812
journals at least
tweeted once
• two-thirds of
journals have
coverage below
20% and Twi...
Results: disciplines
Study I: Tweeting biomedicine
Results: specialties
Study I: Tweeting biomedicine
• specialties differ in
terms of coverage,
Twitter citation rate
and co...
Aim of the study
•  in-depth analysis of astrophysicists on Twitter
•  number of tweets, followers, retweets
•  characteri...
Data sets & methods
•  37 astrophysicists on Twitter identified by
Holmberg & Thelwall (2013)
•  web searches to identify ...
Data sets & methods
•  grouping astrophysicists according to tweeting and
publication behavior
•  analyzing differences of...
Data sets & methods
•  comparison of tweet and publication content
•  extraction of noun phrases from tweets and abstracts...
Results: correlations
•  comparison of Twitter and publication activity and impact
•  publications and tweets per day: ρ=−...
Results: characteristics
Study II: Astrophysicists on Twitter
Mean share of tweets containing at least one user name or
UR...
Results: content similarity
Study II: Astrophysicists on Twitter
•  overall similarity between abstracts and tweets is low...
Conclusions
•  Twitter coverage of biomedical papers is low but increasing
•  number of tweets per paper varies between jo...
Outlook
•  user surveys and qualitative research to investigate who is
using scholarly content on social media and why
•  ...
Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (in press). Tweeting
biomedicine: an analysis of t...
Upcoming SlideShare
Loading in...5
×

Empirical analyses of scientific papers and researchers on Twitter: Results of two studies

811

Published on

presentation held at PLoS ALM Workshop 2013 in San Francisco
http://article-level-metrics.plos.org/alm-workshop-2013-preliminary-program/
presenting results of two Twitter studies: 1.4 PubMed papers and 37 astrophysicists on Twitter

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
811
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
23
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Empirical analyses of scientific papers and researchers on Twitter: Results of two studies

  1. 1. Empirical analyses of scientific papers and researchers on Twitter: Results of two studies Stefanie Haustein, Timothy D. Bowman, Kim Holmberg, Vincent Larivière, Isabella Peters, Cassidy R. Sugimoto, & Mike Thelwall
  2. 2. Background •  when Garfield created SCI, sociologists of science analyzed meaning of publications and citations (Merton, Zuckerman, Cole & Cole, etc.) •  sociological research •  What is it to publish a paper? •  What are the reasons to cite? •  empirical bibliometric research •  disciplinary differences in publication and citation behavior •  delay and obsolescence patterns
  3. 3. Background •  empirical studies helped sociologists to understand structure and norms of science •  for bibliometricians, studies provided a theoretical framework and legitimation to use citation analysis in research evaluation •  knowledge about disciplinary differences and obsolescence patterns helped to normalize statistics and create more appropriate indicators
  4. 4. Background •  recently social-media metrics have become important in the scholarly world •  suggestions to complement (or even replace) citation analysis by so-called ”altmetrics“ •  broader audience (not just citing authors) •  more timely •  however, similar to bibliometrics in the 1960s, little is known about the actual meaning of various social-media counts
  5. 5. Research questions •  What is the relationship between social-media and citation counts? •  How do various social-media metrics differ? •  Why are papers tweeted, bookmarked, liked…? •  Who tweets (bookmarks, likes…) scientific papers? •  How do these aspects differ across scientific disciplines? Two case studies on Twitter •  large-scale analysis of tweets of biomedical papers •  in-depth analysis of astrophysicists on Twitter
  6. 6. Aim of the study •  large-scale analysis of tweets of biomedical papers •  Twitter coverage •  Twitter citation rates (tweets per paper) •  correlation with citations •  discovering differences between: •  documents •  journals •  disciplines & specialties !  providing empirical framework to understand the extent to which biomedical journal articles are tweeted Study I: Tweeting biomedicine Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (in press). Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the American Society for Information Science and Technology, http://arxiv.org/abs/1308.1838.
  7. 7. Data sets & methods •  1.4 million PubMed papers covered by WoS •  publication years: 2010-2012 •  document types: articles & reviews •  matching of WoS and PubMed •  tweet counts collected by Altmetric.com •  collection based on PMID, DOI, URL •  matching WoS via PMID •  journal-based matching of NSF classification •  tweets per article, Twitter coverage and correlation with citations for: •  journals •  NSF disciplines and specialties Study I: Tweeting biomedicine
  8. 8. Data sets & methods: framework Study I: Tweeting biomedicine
  9. 9. Data sets & methods: correlations Study I: Tweeting biomedicine PY=2010 PY=2011 PY=2012
  10. 10. Results: documents Study I: Tweeting biomedicine Publication year Twitter coverage Papers (T≥1) Spearman's ρ Mean Median Maximum T2010 2.4% 13,763 .104** 2.1 1 237 C2010 18.3 7 3,922 T2011 10.9% 63,801 .183** 2.8 1 963 C2011 5.7 2 2,300 T2012 20.4% 57,365 .110** 2.3 1 477 C2012 1.3 0 234 T2010-2012 9.4% 134,929 .114** 2.5 1 963 C2010-2012 5.1 1 3,922 •  Twitter coverage is quite low but increasing •  correlation between tweets and citations is very low
  11. 11. Results: documents Study I: Tweeting biomedicine Article Journal C T Hess et al. (2011). Gain of chromosome band 7q11 in papillary thyroid carcinomas of young patients is associated with exposure to low-dose irradiation PNAS 9 963 Yasunari et al. (2011). Cesium-137 deposition and contamination of Japanese soils due to the Fukushima nuclear accident PNAS 30 639 Sparrow et al. (2011). Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips Science 11 558 Onuma et al. (2011). Rebirth of a Dead Belousov–Zhabotinsky Oscillator Journal of Physical Chemistry A -- 549 Silverberg (2012). Whey protein precipitating moderate to severe acne flares in 5 teenaged athletes Cutis -- 477 Wen et al. (2011). Minimum amount of physical activity for reduced mortality and extended life expectancy: a prospective cohort study Lancet 51 419 Kramer (2011). Penile Fracture Seems More Likely During Sex Under Stressful Situations Journal of Sexual Medicine -- 392 Newman & Feldman (2011). Copyright and Open Access at the Bedside New England Journal of Medicine 3 332 Reaves et al. (2012). Absence of Detectable Arsenate in DNA from Arsenate-Grown GFAJ-1 Cells Science 5 323 Bravo et al. (2011). Ingestion of Lactobacillus strain regulates emotional behavior and central GABA receptor expression in a mouse via the vagus nerve PNAS 31 297 Top 10 tweeted documents: catastrophe & topical / web & social media / curious story scientific discovery / health implication / scholarly community
  12. 12. Results: journals • 97.7% of 3,812 journals at least tweeted once • two-thirds of journals have coverage below 20% and Twitter citation rate < 2.0 • high Twitter citation rates often caused by few papers • high coverage and Twitter citation rates for general journals Study I: Tweeting biomedicine
  13. 13. Results: disciplines Study I: Tweeting biomedicine
  14. 14. Results: specialties Study I: Tweeting biomedicine • specialties differ in terms of coverage, Twitter citation rate and correlations with citations • 47 of 61 specialties show low positive, 3 negative and 13 no correlation bubblesize=Twittercitationrate
  15. 15. Aim of the study •  in-depth analysis of astrophysicists on Twitter •  number of tweets, followers, retweets •  characteristics of tweets: RTs, @messages, #hashtags, URLs •  comparison with scientific output •  publications •  citations •  comparison of tweet and publication content !  provide evidence in how far astrophysicists on Twitter use Twitter for scholarly communiation Study II: Astrophysicists on Twitter Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I., (submitted). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. Aslib Proceedings.
  16. 16. Data sets & methods •  37 astrophysicists on Twitter identified by Holmberg & Thelwall (2013) •  web searches to identify person behind account •  publications in WoS journals •  publication years: 2008-2012 •  author disambiguation •  Twitter account information •  68,232 of 289,368 tweets downloaded and analyzed: •  number of RTs per tweet •  % of tweets that are RTs •  % of tweets containing #hashtags, @usernames, URLs Study II: Astrophysicists on Twitter Holmberg, K., & Thelwall, M. (2013). Disciplinary differences in Twitter scholarly communication. In: Proceedings of ISSI 2013 – 14th International Conference of the International Society for Scientometrics and Informetrics, Vienna, Austria (Vol. 1, pp. 567-582).
  17. 17. Data sets & methods •  grouping astrophysicists according to tweeting and publication behavior •  analyzing differences of tweeting characteristics between user groups Study II: Astrophysicists on Twitter Selected astrophysicists (N=37)! tweet rarely (0.0-0.1 tweets per day)! tweet occasionally (0.1-0.9)! tweet regularly (1.2-2.9)! tweet frequently (3.7-58.2)! total (publishing activity)! do not publish (0 publications 2008-2012)! --! --! 1! 5! 6! publish occasionally (1-9)! 4! 3! 4! 2! 13! publish regularly (14-37)! --! 5! 5! 3! 13! publish frequently (46-112)! 1! 3! 1! --! 5! total (tweeting activity)! 5! 11! 11! 10! 37!
  18. 18. Data sets & methods •  comparison of tweet and publication content •  extraction of noun phrases from tweets and abstracts •  limited to 18 most frequently publishing astrophysicists to ensure certain number of abstracts •  analyzing overlap of character strings •  calculating similarity with cosine per person and overall Study II: Astrophysicists on Twitter Selected astrophysicists (N=37)! tweet rarely (0.0-0.1 tweets per day)! tweet occasionally (0.1-0.9)! tweet regularly (1.2-2.9)! tweet frequently (3.7-58.2)! total (publishing activity)! publish regularly (14-37)! --! 5! 5! 3! 13! publish frequently (46-112)! 1! 3! 1! --! 5! total (tweeting activity)! 1! 8! 6! 3! 18!
  19. 19. Results: correlations •  comparison of Twitter and publication activity and impact •  publications and tweets per day: ρ=−0.339* •  citation rate and tweets per day: ρ=−0.457** •  citation rate and RT rate: ρ=0.077 Study II: Astrophysicists on Twitter
  20. 20. Results: characteristics Study II: Astrophysicists on Twitter Mean share of tweets containing at least one user name or URL per person per group
  21. 21. Results: content similarity Study II: Astrophysicists on Twitter •  overall similarity between abstracts and tweets is low •  cosine=0.081 •  4.1% of 50,854 tweet NPs in abstracts •  16.0% of 12,970 abstract NPs in tweets •  Twitter coverage among most frequent abstract terms is high, although this differs between users •  97,1% of 104 most frequent noun phrases on Twitter
  22. 22. Conclusions •  Twitter coverage of biomedical papers is low but increasing •  number of tweets per paper varies between journals, disciplines, specialties and from year to year !  tweet counts need to be normalized accordingly •  correlations between tweet and citation counts are low (biomedical papers) or even moderately negative (astrophysicists) !  tweets cannot replace citations as measures of scientific impact !  challenge is to differentiate between high tweet counts because of value (to scientists and/or the general public) and curiosity
  23. 23. Outlook •  user surveys and qualitative research to investigate who is using scholarly content on social media and why •  empirical large-scale studies on other metrics
  24. 24. Haustein, S., Peters, I., Sugimoto, C.R., Thelwall, M., & Larivière, V. (in press). Tweeting biomedicine: an analysis of tweets and citations in the biomedical literature. Journal of the American Society for Information Science and Technology. Haustein, S., Bowman, T.D., Holmberg, K., Larivière, V., & Peters, I., (submitted). Astrophysicists on Twitter: An in-depth analysis of tweeting and scientific publication behavior. Aslib Proceedings. Stefanie Haustein Thank you for your attention! Questions? stefanie.haustein@umontreal.ca @stefhaustein
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×