The Impact Factor, Eigenfactor, and Altmetrics: From Theory to Analysis


Published on

Altmetrics is an emerging area encompassing broader assessment of scholarly impact through downloads, links, and online conversations to fill gaps in assessing research. Bibliometrics is the traditional form of measuring the impact of scholarly research through citation rates. The Research & Instruction Librarian for Sciences and the Scholarly Communication Librarian at Wake Forest University will compare bibliometrics and altmetrics, and discuss their applications in science information literacy and research assessment in higher education.

Published in: Education, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • Now is classified by the National Library of Medicine MeSH broader term“Bibliometrics” Narrower Term “Impact Factor”
  • Each journal can belong in multiple subject categories
  • Variation of IF range among disciplines Ex. medical v. mathematics journals
  • Dr. Kim has written 39 articles so his maximum h-index could possibly equal 39. According to a citation report in Web of Science database, Dr. Kim’s h-index is 12, therefore 12 of his articles have been cited at least 12 times each.
  • Eigenfactorconsiders citations from more influential journals than lesser known journals
  • Users may not guess correctly to which subject category the journal was assigned.
  • As scholars have migrated to online spaces for research purposes, various platforms capture previously invisible activities such as reading, saving, discussing, and recommending research outputsTraces of these activities are being observed to “inform new metrics of scholarly influence and impact—so called altmetrics” (Lapinski, Piwowar, Priem)Short for alternative metrics, Jason Priem coined the term, and in “Altmetrics: A Manifesto” he co-authored, he fully proposed altmetrics in 2010; defined as “altmetrics is the creation and study of new metrics based on the Social Web for analyzing, and informing scholarship.”
  • “Aims to measure Web-driven scholarly interactions” (Howard, quoted in Galligan & Dyas-Correia)“Altmetrics are new measurements for the impact of scholarly content, based on how far and wide it travels” (Galligan, quoted in Galligan & Dyas-Correia)“Altmetrics go beyond traditional citation-based indicators as well as raw usage factors (such as downloads or click-through rates) in that they focus on readership, diffusion and reuse indicators”“More importantly, they are diverse, tracking impacts all across a quickly changing landscape populated by: diverse products…; diverse platforms…; diverse audiences…” (Lapinski, Piwowar, Priem)Measures: usage, captures, mentions, social media, citations
  • “Sometimes conflated, but not interchangeable” (Tananbaum)Public Library of Science, also known as PLoS; Scopus; Nature Publishing Group; and Highware have all incorporated article-level metrics into their platformsUnlike the bibliometrics measures Sarah discussed, which are usually tied to a journal’s impact, ALMs “disaggregate an individual article’s impact from the publication in which it appears” (Tananbaum)Another way to think about this is that ALMs track the impact of a single article and altmetrics, when aggregated, track the impact of multiple research components, from publications to data to presentations to products
  • ImpactStory is one tool for tracking altmetricsHeather Piwowar and Jason Priem cofounded; recently received a huge NSF grant (300K) and also is funded by the Alfred P. Sloane FoundationOpen source altmetrics tool; nonprofit companyDraws from Facebook, Twitter, CiteULike, Delicious, PubMed, Scopus, CrossRef, scienceseeker, Mendeley, Wikipedia, slideshare, Dryad, and figshareNormalizes metrics based on a sample of articles published the same yearRaw and percentiles compared to other articles
  • Plum Analytics is another tool for tracking altmetricsProprietaryMarketed to universities and research institutions, not directly to researchersMetrics for nearly two dozen types of outputsCustom reports intended to quantify departmental productivity, support grant proposals, and address other impact-related questions
  • Altmetric.comProprietaryAdopted by publishers, including Springer, Nature Publishing Group, Scopus (Elsevier), and BioMed CentralTracks social media sites, newspapers, and magazines for mentions of scholarly articlesScore created for each article; quantitative measure of the quality and quantity of attention an article has receivedBased on three main factors: number of individuals mentioning the paper, where the mentions occurred, and how often the author of each mention talks about scholarly articles
  • PLOS ONE article-level metrics
  • Since I am the library liaison to the Biology, Chemistry, and Physics departments, I compared the top ranked scientific journals by 2-year Impact Factor in Journal Citation Reports.
  • DORA drafted in December 2012The San Francisco Declaration on Research Assessment (DORA), initiated by the American Society for Cell Biology (ASCB) together with a group of editors and publishers of scholarly journals, recognizes the need to improve the ways in which the outputs of scientific research are evaluated. The group met in December 2012 during the ASCB Annual Meeting in San Francisco and subsequently circulated a draft declaration among various stakeholders. DORA as it now stands has benefited from input by many of the original signers listed below. It is a worldwide initiative covering all scholarly disciplines. We encourage individuals and organizations who are concerned about the appropriate assessment of scientific research to sign DORA.A number of themes run through these recommendations:the need to eliminate the use of journal-based metrics, such as Journal Impact Factors, in funding, appointment, and promotion considerations;the need to assess research on its own merits rather than on the basis of the journal in which the research is published; andthe need to capitalize on the opportunities provided by online publication (such as relaxing unnecessary limits on the number of words, figures, and references in articles, and exploring new indicators of significance and impact).
  • DryadCurated general-purpose repository for data underlying scientific and medical literatureIntegrated data submission for a growing number of journalsIndividual and institutional membership options available$80/deposit fee for individualsData submitted under a CC Zero license; given DOIsRuns on Dspace, served by NC State UniversityFigshareRepository for figures, datasets, media, papers, posters, presentations, and filesets; given DOIsSupports multiple formatsPrivate and public spaceCollaborative spaces in the worksDatasets shared under a CC Zero license; everything else under a CC-BY licensePreserved in CLOCKSSFree to join; funded by Digital ScienceGitHubworld’s largest open source communityCollaborative project spacePublic and private options available
  • ZoteroOpen source, cloud based citation management systemMore easily facilitates sharing of citationsMendeleyWas independent, recently acquired by Elsevier, so we’ll see what happensEnables full text article uploadsMore social aspect than Zotero, but also a citation management systemF1000For biology and medicineOA journal and OA database of posters/presentationsRecommended top articles by top faculty and researchers worldwideCiteULike- Yet another social citation management tool
  • Academia.eduOver 4.8 million usersOver 1.6 million papers and 888,000+ research interests addedShare papers, see analytics on papers and profile, follow others in your fieldResearchGate“built by scientists, for scientsts”Over 4 million membersShare papers, find collaborators, track stats (views, downloads, citations of your work)SlideShareNot as overtly geared toward research, but used by many researchersMarketing angleCan follow othersCan get usage stats on views, downloadsCan upgrade to Pro account for additional benefits and features
  • The Impact Factor, Eigenfactor, and Altmetrics: From Theory to Analysis

    1. 1. + The Impact Factor, Eigenfa ctor, & Altmetrics: From Theory to Analysis Sarah Jeong, Research & Instruction Librarian for Science Molly Keener, Scholarly Communication Librarian Wake Forest University October 18, 2013
    2. 2. + What we’ll discuss  Bibliometrics  Altmetrics  Application  Tools for the Future
    3. 3. + Bibliometrics THE KNOWN
    4. 4. + Alan Pritchard first coined the term “bibliometrics”
    5. 5. + “In the early 1960s Irving H. Sher and [Eugene Garfield] created the journal impact factor to help select journals for the Science Citation Index…[Garfield] expected that it would be used constructively while recognizing that in the wrong hands it might be abused.” (Garfield 1999)
    6. 6. + Eugene Garfield (1955) first mentioned the concepts of Science Citation Index and Impact Factor in Science  Impact Factor (IF) = “a measure of the frequency with which an „average article‟ in a journal has been cited in a particular year or period” 2005 IF of a journal = 2005 cites to articles published in 2003-04 number of articles published in 2003-04  “The journal impact factor is a good predictor of the quality of journals as measured by citations to primary research articles. It is, however, a poor indicator of citations to specific papers or of the future performance of individual researchers.” (Nature Materials 2013)
    7. 7. + PROS Impact Factor • • • • One of the oldest quantified metrics 2-year & 5-year citation windows Many journals advertise their IF Widely used & recognized
    8. 8. + Impact Factor CONS • Citations need context • • • Unidentifiable + or – citations Self-citations Review articles are favored • Metric for journals not authors • Time varying IF  • Limited to JCR
    9. 9. + “For the few scientists who earn a Nobel Prize, the impact…of their research is unquestionable. For the rest of us, how does one quantify the cumulative impact…of an individual‟s scientific research output?” (Hirsch 2005)
    10. 10. + h-index shows the broad impact of an individual’s work h-index developed by a physicist (Hirsch 2005) Ex. Dr. Kim’s h-index = 12 12 of his articles have been cited at least 12 times each
    11. 11. + PRO  Considers the impacts of both journals and authors CONS h-index    Unidentifiable + or – citations h-index increases with age so comparing productivity of younger researchers is problematic Calculated in Web of Science but need comprehensive citation report of all author’s publications
    12. 12. + Dr. Bergstrom and his colleagues “have developed a way to use the network structure of citations to improve on simple citation counts in measuring the scientific influence of academic publications.” (Bergstrom 2007)
    13. 13. + Eigenfactor developed by Dr. West and Dr. Bergstrom at Univ. of Washington  “Eigenfactor scores are scaled so that the sum of the Eigenfactor scores of all journals listed in Thomson's Journal Citation Reports (JCR) is 100.”
    14. 14. + PROS  Weighted metric with different weights for journals  Excludes self-citations  5-year citation window Eigenfactor CONS  Limited to journals in Journal Citation Reports  Journals assigned to a single subject category (Jacso 2012)
    15. 15. + Altmetrics THE NEW
    16. 16. reading recommending What are altmetrics, anyway? saving Altmetrics discussing
    17. 17. + Altmetrics measure…  How far and wide content travels through the scholarly* web  Web-driven social scholarly interactions  Twitter  Facebook  Blogging  Bookmarking * It‟s not just scholars who are engaging: clinicians, practitioners, and the general public are reading and sharing, too!
    18. 18. + Altmetrics vs. Article-Level Metrics  Related, but not interchangeable  Article-Level Metrics present picture of an article’s true impact via data points   Scopus  Nature   PLoS Highwire Altmetrics track other types of output in addition to articles, including datasets, presentations, and software
    19. 19. + So what’s the verdict? PROS New and emerging Gaming still possible      Content-level, not containerlevel CONS Context is critical Immediacy  Social sharing  Incorporates traditional metric measures, too
    21. 21. +
    22. 22. + Figure 2. Biochemistry & Molecular Biology Journals 40 35 30 25 20 15 10 5 0 2012 Impact Factor 2012 5-Year Impact Factor Eigenfactor Cell 31.957 34.366 0.585 Annual Review of Biochemistry 27.681 31.964 0.051 Nature Medicine 24.302 27.139 0.164 Molecular Cell 15.28 14.902 0.228 Molecular Psychiatry 14.897 13.985 0.042
    23. 23. +
    24. 24. +
    25. 25. + Assessing “impact”  Tenure & Promotion    Expectations Tenure-track vs. tenured New models & modes of scholarship   Digital Humanities San Francisco Declaration on Research Assessment (DORA)
    26. 26. + Tools for the Future THE ALTMETRICS “GRAB-BAG”
    27. 27. + Data sharing tools
    28. 28. + Citation & collaboration tools
    29. 29. + Social sharing tools
    30. 30. + References - Bibliometrics  Bergstrom, C. (2007). Eigenfactor: Measuring the value and prestige of scholarly journals. College & Research Libraries News, 68(5), 314–316.  Beware the impact factor. (2013). Nature Materials, 12(2), 89–89. doi:10.1038/nmat3566  Eigenfactor Score and Article Influence Score: Detailed Methods. (2008). Retrieved from  Garfield, E. (1955). Citation indexes for science. Science, 122(3159), 108–111.  Garfield, E. (1999). Journal impact factor: A brief review. Canadian Medical Association Journal, 161(8), 979–980.  Hirsch, J. E. (2005). An index to quantify an individual’s scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572. doi:10.1073/pnas.0507655102  Jacsó, P. (2010). Eigenfactor and article influence scores in the Journal Citation Reports. Online Information Review, 34(2), 339–348. doi:  Jacsó, P. (2012). The problems with the subject categories schema in the EigenFactor database from the perspective of ranking journals by their prestige and impact. Online Information Review, 36(5), 758–766. doi:10.1108/14684521211276064  Pritchard, A. (1969). Statistical bibliography or bibliometrics? Journal of Documentation, 25(4), 348–349.  Van Noorden, R. (2013). Brazilian citation scheme outed. Nature, 500(7464), 510–511. doi:10.1038/500510a
    31. 31. + References - Altmetrics    DORA –  Galligan, F., & Dyas-Correia, S. (2013). Altmetrics: Rethinking the Way We Measure. Serials Review, 39, 56-61. doi: 10.1016/j.serrev.2013.01.003  Galloway, L.M., Pease, J.L., & Rauh, A.E. (2013). Introduction to Altmetrics for Science, Technology, Engineering, and Mathematics (STEM) Librarians. Science & Technology Libraries, epub ahead of print, 11pp. doi: 10.1080/0194262X.2013.829762   Kwok, R. (2013). Research Impact: Altmetrics Make Their Mark. Nature, 500, 491-493. doi: 10.1038/nj7463-491a  Lapinski, S., Piwowar, H., & Priem, J. (2013). Riding the Crest of the Altmetrics Wave. College & Research Libraries News, 74(6), 292-300.   Tananbaum, G. (2013). Article-Level Metrics: A SPARC Primer. SPARC. 14pp.
    32. 32. + Questions?
    33. 33. + Sarah Jeong Thank you! Molly Keener