Twitter and research impact


Published on

A review of Eysenbach, G., 2011. Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact. Journal of Medical Internet Research, 13(4), p.e12

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Designed by Garfield to help research libraries choose journal subscriptions but has come into much criticism in recent years due to its perceived limitations and loopholes. Authors citing themselves to boost citation rate, cross-citation where journals purposely cite papers from the other to boost overall impact factor of both journals. If detected these journals are suspended. Has been variously called wrong or a “mis-measure” or as Imperial College London researcher Stephen Curry: “the stupid, it burns.”
  • Eysenbach’s study looks at one particular platform – Twitter – and is concerned with the correlation between citation of scholarly articles on this platform and traditional metrics of citation in peer-reviewed journals. He doesn’t deal with metrics outside article-level such as Slideshare views, Likes, blog entries etc.
  • roughly 80% of the effects come from 20% of the causes
  • Left: Zipf plot for JMIR articles 3/2000-12/2009 (n=405), with number of citations (y-axis) plotted against the ranked articles. Right: Zipf plot showing the number of tweetationsor Twitter citations in the first week (tw7) to all JMIR articles (n=206) published between  April 3 2009 and nov 15 2011 plotted against ranked articles. Eg top tweeted article for 97 tweetations, the 10th article for 43 tweetations, the 102th ranked got 9 tweetations.
  • should be primarily seen as metrics for social impact (buzz, attentiveness, or popularity) and as a tool for researchers, journal editors, journalists, and the general public to filter and identify hot topics. 
  • Twitter and research impact

    1. 1. Digital Enterprise Research Institute Twitter and research impact Marie Boran Copyright 2011 Digital Enterprise Research Institute. All rights reserved. Enabling networked knowledge
    2. 2. Digital Enterprise Research Institute  A review of: Eysenbach, G., 2011. Can Tweets Predict Citations? Metrics of Social Impact Based on Twitter and Correlation with Traditional Metrics of Scientific Impact. Journal of Medical Internet Research, 13(4), p.e12. Enabling networked knowledge 2
    3. 3. A little background…Digital Enterprise Research Institute Impact Factor as a measure of scientific impact: The Good, the Bad and the Ugly. Enabling networked knowledge 3
    4. 4. Sick of Impact Factor?Digital Enterprise Research Institute  Imperial College London researcher Stephen Curry: „the stupid, it burns.”  impact-factors/  “dependency on a valuation system that is grounded in falsity.”  “we need to find ways to attach to each piece of work the value that the scientific community places on it though use and citation.” Enabling networked knowledge 4
    5. 5. What are altmetrics?Digital Enterprise Research Institute  Alternative web-based social metrics  Scientometrics from online social activity centred around scholar‟s work  Self-publishing: blogging, uploading, tweeting, sharing  Impact measured via: articles viewed, shared, downloaded, „retweeted‟, „liked‟, etc. “Scholars are moving their everyday work to the web. Online reference managers Zotero and Mendeley each claim to store over 40 million articles (making them substantially larger than PubMed); as many as a third of scholars are on Twitter, and a growing number tend scholarly blogs. These new forms reflect and transmit scholarly impact […] That hallway conversation about a recent finding has moved to blogs and social networks– now, we can listen in. - manifesto A ” From: Enabling networked knowledge 5
    6. 6. Eysenbach (2011)Digital Enterprise Research Institute  Study objectives:  Feasibility of measuring social impact/public attention to scholarly articles through social media  Relation between dynamics, timing of tweets about a scholarly article (aka tweetations) and journal citations  Evaluating accuracy of resulting metrics in predicting highly cited articles Journal of Medical Internet Research top articles, ranked by tweets Enabling networked knowledge 6
    7. 7. MethodsDigital Enterprise Research Institute  Journal of Medical Internet Research  Highly-cited, open access journal  Articles published between issues 3/2009 and 2/2010  Thomson Reuters 3-year impact factor of 4.7  Citation counts (SCOPUS and Google Scholar)  Twitter citations or „tweetation” – must mention journal article URL  Only tweets with URLs linking directly to the journal article are captured. Does not count links to blogs or newspaper articles mentioning research. Note: Eysenbach is the editor-in-chief and publisher of JMIR Enabling networked knowledge 7
    8. 8. Methods (cont‟d)Digital Enterprise Research Institute  Tweets captured: all sent and archived by JMIR between July 24, 2008 and November 20, 2011  Classification: “highly-cited” articles - top 25th percentile of each issue (by citation counts)  “highly-tweeted” - top 25th percentile (ranked by tweetations)  Adjusted for increasing popularity of Twitter over time & older articles have higher citations. Enabling networked knowledge 8
    9. 9. ResultsDigital Enterprise Research Institute  55 articles  4208 tweetations  Average 14 tweetations per article  Majority of tweets published on or day after article published (see graph)  First 30 days: “network propagation phase”  30+: “sporadic tweetation phase”  Observed 80/20 rule (Pareto principle)  Highly tweeted articles 11 times more likely to be highly cited than less-tweeted articles  75% of highly tweeted articles were highly cited in comparison to 7% of less- tweeted articles Enabling networked knowledge 9
    10. 10. Results (cont‟d)Digital Enterprise Research Institute  Citation and tweetation patterns  Scopus and Google Scholar citations tested for agreement  Eysenbach observed “distribution […] typically observed for citations” Enabling networked knowledge 10
    11. 11. FindingsDigital Enterprise Research Institute  First systematic, prospective, longitudinal article and journal-level investigation of how mention (citations or tweetations) of scholarly articles in social media accumulate over time  First study correlating altmetrics to citations  Online buzz around articles is measurable  Tweets are “surprisingly accurate” predictors of future journal citations Enabling networked knowledge 11
    12. 12. LimitationsDigital Enterprise Research Institute Via  Complementary, *not* a replacement for Impact Factor  “Tweetations” as buzz, attentiveness, social impact Enabling networked knowledge 12
    13. 13. ConclusionsDigital Enterprise Research Institute  Proposes “twimpact factor” (twn) as metric of impact in social media, where n is cumulative number of tweetations within n days after publication  “The cumulative number of tweetations by day 7 (perhaps as early as day 3), could be used as a diagnostic test to predict highly cited articles.”  Tweetations as proxies for social impact of scientific research  Can be applied to other social media and non-scholarly articles to measure issue impact on social media user population + = Twitter + metrics = wider perspective on research impact Enabling networked knowledge 13
    14. 14. Related researchDigital Enterprise Research Institute • Priem, J. & Costello, K.L., 2010. How and why scholars cite on Twitter. Proceedings of the 73rd ASIST Annual Meeting, 47(1), p.1-4. • Priem, J. & Hemminger, B.M., 2010. Scientometrics 2.0: Toward new metrics of scholarly impact on the social Web. First Monday, 15(7) Enabling networked knowledge 14
    15. 15. Happy Christmas!Digital Enterprise Research Institute Enabling networked knowledge 15