Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Measuring research impact with bibliometrics


Published on

Updated 30/01/2015

This session included discussions around the value of bibliometrics for individual performance management/promotion and the REF.

What are bibliometrics?
Journal metrics
Personal metrics
Article level metrics and altmetrics

Published in: Education
  • Be the first to comment

Measuring research impact with bibliometrics

  1. 1. Measuring research impact with bibliometrics Friday 30th January Tanya Williamson, Assistant Librarian
  2. 2. Overview Learn about the most common bibliometrics and the tools used to measure research impact and influence. 1. What are bibliometrics? 2. Journal metrics 3. Personal metrics 4. Article level metrics and altmetrics Optional practical: Have a go at finding a journal's impact factor, and calculating a researcher’s h-index. • Practical – Find journal metrics using journal citation reports • Practical – Find a researcher’s h-index using Google Scholar or Web of Science
  3. 3. • Bibliometrics are quantitative methods of analysing research publications and citations • Based on the assumption that citations = impact What are bibliometrics? Citation counts in Web of Science Visualisation of citation networks from InCites Journal Citation Reports Graphic from
  4. 4. An attempt to compare the influence and impact of journal titles in a particular discipline. Can be used to inform where to publish, which could in turn improve research impact. Commonly used metrics available through Journal Citation Reports, part of Web of Science • Journal Impact Factor • 5 year Journal Impact Factor • Immediacy index • Eigenfactor score and • SCImago Journal rank Journal metrics
  5. 5. Journal Citation Reports category rank An example: Journal of Allergy and Clinical Immunology ALLERGY category rank
  6. 6. Journal Impact Factor JIF 2011 2012 2013 2014 Citations to 2011 publications Citations to 2012 publications The JCR year Items published Items published 2015 175 125+ 250 350+ = 2 The Journal Impact Factor is the average number of times articles from the journal published in the past two years have been cited in the JCR year. The Journal of Important Discoveries Other Journals in WoS
  7. 7. The Journal Impact Factor is the average number of times articles from the journal published in the past two years have been cited in the JCR year. Also available: 5 year Journal Impact Factor Journal Impact Factor An example: Journal of Allergy and Clinical Immunology
  8. 8. • The Immediacy Index is the average number of times an article is cited in the year it is published. • Good for comparing cutting edge research field Immediacy Index An example: Journal of Allergy and Clinical Immunology
  9. 9. • Based on source data from Web Of Science • Not influenced by journal self- citation Eigenfactor and Article Influence An example: Journal of Allergy and Clinical Immunology Demo of Journal Citation Reports
  10. 10. SCImago journal rank An example: Journal of Allergy and Clinical Immunology • SCImago Journal Rank uses source data from Scopus • Based on the Google PageRank algorithm • Weights citations from high prestige journals and attempts to balance the influence of the size of a journal Immunology and Allergy category rank
  11. 11. • Don’t work well for every discipline due to different publishing patterns and coverage of different publication types • Mustn’t be used to compare different disciplines Limitations of journal metrics Top scorer in Premier league is Luis Suarez with 31 goals Top scorer in NBA is Kevin Durant with 849 goals Can we conclude that Durant is a better sportsman?
  12. 12. • Focusing only on Journal Impact Factors could lead to you overlooking smaller, specialist publications • Dependent on the coverage in the Web of Science database • Can we infer individual impact from journal impact? Limitations of journal metrics
  13. 13. This video: Bibliometrics for the Individual, Prof Dermot Diamond Measure your Research Impact, Bibliometrics toolkit on YouTube Bibliometrics for individuals
  14. 14. Key metrics • Citation counts - uses a particular dataset to count how many citations an author has received • H-index, devised by: J. E. Hirsch in 2005 • There are several ‘improvements’ on the h-index, e.g. the g- index, the e-index, the w-index, the hfa index… Personal metrics Easy explanation: If a scholar has 20 articles which have each been cited 20 times, s/he has an h-index of 20
  15. 15. • Web of Science Personal metrics Demo of Web of Science
  16. 16. Google Scholar • A way to find your own h-index • The author needs to have a Google Scholar profile Personal metrics Publish or Perish • Uses Google Scholar data • Author and journal metrics • Accompanying book and website
  17. 17. • Name ambiguity – getting a comprehensive list is not easy, even in Web of Science • No one metric can capture all citations or ‘impacts’ • Early career researchers will be at a disadvantage • Citations ≠ endorsements of quality! Limitations of personal metrics Resistance against the h-index, ImpactStory
  18. 18. • Citation analysis in Web of Science, Google Scholar • Publishers’ interfaces often include citing articles Article level metrics Publisher’s site Google Scholar Web of Science
  19. 19. “Article-Level Metrics (ALMs) leverage the acceleration of research communication made possible by the networked landscape of researcher tools and services. Also by incorporating the manifold ways in which research is disseminated, these article impact indicators are made available rapidly after publication and are continually updated.” PLOS One Article Level Metrics Information
  20. 20. Alternative metrics or altmetrics ImpactStory: like a living publications CV. Adding metrics all the time watch social media sites, newspapers, government policy documents and other sources for mentions of scholarly articles to compile article level metrics. • Shows the broader, societal impact • More than just articles
  21. 21. Remember that highly cited article from before? Limitations of article level and altmetrics Source of data Citation to an article Web of Science 2021 Google Scholar 2937 Publisher’s website 1478
  22. 22. • Publish or Perish • Measure Your Research Impact toolkit • Conference Rankings for Computer Science: • The Ultimate Guide to Altmetrics (Researchers edition) The University is currently considering other tools, such as Scopus, SciVal and Altmetric for Institutions. Thanks to Scott Taylor from University of Manchester for permitting me to borrow the sporting analogy from his presentation with Stephen Pearson at the Northern Collaboration Conference 2014. collaboration-conference-2014-40398613 Follow up resources
  23. 23. 1. Navigate to Journal Citation Reports from the Library Databases A-Z 2. Choose a subject category to compare journals against different metrics  Practical
  24. 24. Find a scholar’s H-index in Google Scholar 1. Go to Google Scholar 2. Search for a topic of your interest 3. Choose a scholar who has a Google Scholar profile … Outside of the session, you can look into Publish or Perish software available from which ingests data from Google Scholar and allows you to analyse on a variety of journal and personal metrics.  Practical